I have spent the past 10+ years working with Blackboard as a system administrator at multiple higher education institutions in the US. In 2009, Blackboard presented me the Blackboard Catalyst Award for my work within the Blackboard Community. In 2013, I authored the first technical book for Blackboard Learn Administrators published by Packt Publishing.
Here is the place I post my musings, issues, and finding as I work through the sometimes frustrating, humorous, but always interesting world of Blackboard.
One of the many topics that my book, Blackboard Learn Administration, talks about is upgrading and installing Blackboard Learn. However I didn’t really go into detail about the process in preparing to upgrade or install a new version of Blackboard Learn for your institution. Well this blog post will hopefully erase that neglect. Let’s review how I prepare for a Blackboard Learn upgrade.
Select a version
Blackboard Learn has been rather stable in their release structure. Recently they announced the move from four releases a year to two releases a year. This might be a good and bad option for institutions depending on the release date and maintenance schedule. I try to recommend the latest version released prior to our testing window. The testing and security scans can take weeks. This along with implementing the new version means it will be about four to five months old by the time it’s put on our production environment.
The biggest concern deals with the amount of time Blackboard will support the version with regular patches for security and other issues that are found. Recently, Blackboard has moved to supporting versions for 18 months after their release. I like this plan because it gives the institution a guaranteed year of support with a version. I’m fond of the yearly upgrade. Some might disagree with this plan, but it seems to be the best based on the following factors.
A yearly upgrade gives a lot of time for security and user testing.
While the selected version is supported and patched, it doesn’t utilize “bleeding edge technology”
Reduces training and feature fatigue for faculty, staff and students.
Additional resources from Blackboard and the community are available
An upgrade can require a lot more than just installing new applications. After many years of performing Blackboard Learn upgrades, I’ve learned that all the parties possibly impacted by an upgrade should take part in upgrade planning, whether that would be the support staff, database administrators, networking teams, server administrators, or security teams. These people may be called on by you, the Blackboard administrator, to help with planning the upgrade or making needed changes. I’ve found a Gantt chart useful to show the upgrade plan. I hand this out at a meeting with all these parties. It’s a big help to get everyone on the same page and have a schedule that we all know and can use when planning this or other projects. I’ve also taken to scheduling monthly meetings with all these stakeholders. While I do cancel some of them, it still gives a regular schedule to discuss changes or issues that are coming up that could influence the planned upgrade.
During the first meeting I just mentioned, one of the topics we talk about is upgrading not only Blackboard, but the supporting software. An upgrade is a great time for server and database administrators to apply security patches or move to a new operating system or application software supported by Blackboard. With our version already chosen, we can look at what software supports the application and quickly plan for this. We might also plan to implement additional hardware components such as load balancers or new IP configurations. These types of changes can be added to our Gantt chart and the update pushed to all affected parties.
When preparing for an upgrade, not does it include changes to the latest version of Blackboard. It will probably include new tools or 3rd party integrations, even editing tool names or improving the application’s customized layout for some. These other items could create more havoc for users if they add to an upgrade without planning.
This is why our team creates a set Blackboard configuration. This means that we all agree early in the upgrade testing process what tools should be installed, what items will be upgraded, enabled, etc. This helps the testing, security, and documentation teams know what the official changes will be. While the set configuration may change, this gives the teams a good place to start and allows for possible changes to be brought to the meetings I mentioned earlier.
The biggest challenge when preparing for an upgrade, in my opinion, is user testing. Does everything still function as we expect in the new version of Blackboard Learn? What the best way to tackle this? The Educational Technologies at the University of Missouri has a team of students who work with Blackboard and we have them help us in our user testing. Each year I create paper copies of several tasks that we expect to do in Blackboard. Each task list becomes associated to an operating system and browser supported by Blackboard. A few of these are available below to review. The student picks up the task list and, using the browser and operating system listed on the front, can complete the tasks listed. They sign off on each task either stating that it works or not. If it doesn’t work, the student writes out an explanation that will help our team figure out what’s going wrong.
In the end, an upgrade plan can help organize the move to a new Blackboard version. It also provides planning for additional services that can impact your upgrade. I recommend picking a version that will give you a few months of testing and do the following:
Look at what upgrade version will meet your needs based on topics such as how long is it supported by Blackboard, etc.
Bring all affected parties to the table, tell them, keep them on schedule.
Find what hardware and software changes can be made to improve support of the Blackboard application
Have all interested parties agree to a set Blackboard configuration for the upgrade for testing purposes.
Find available options to have users test tasks within different browser and operating system combinations supported with the new version.
Best wishes to your (hopefully) successful upgrade!
Greetings everyone! Sorry that I haven’t been able to post but I have recently been dealing with a Blackboard production environment that has spent much of its time in various rings of hell. This has happened on and off since the start of the semester (about 14 weeks ago as of the writing of this post). This has been a very difficult time, but there has been one ray of sunshine for us, our implementation of Kibana, ElasticSearch, and Logstash proved invaluable during troubleshooting.
In my opinion, these three applications are the holy trinity of logging for any application like Blackboard Learn. They allow an admin to unlock the information kept in logs at a cost that most IT budgets can handle (the three tools are open source). I learned about these tools from our central server group (CSG) who had heard about the tools during PuppetCon. (See the video below)
They had started testing them in their “skunkworks” area. During a meeting, I lamented the difficulty searching log files on six different application servers. These difficulties made me create the log collection script which I blogged about earlier this year. One of the team members suggested Logstash, ElasticSearch, and Kibana.
The next week I was at my server administrator training at Blackboard HQ in Washington, DC. During that week, one of the attendees from SUNY mentioned these tools as well. These two discussions had sold me. Back on campus, I quickly moved to test out how helpful and easy it would be to implement these applications within our Blackboard Learn environments.
The three tools Kibana, ElasticSearch, and Logstash all have different parts that they play in the log puzzle.
Logstash – This application brings the logs and event data from one or many systems to one location. The open source product runs on top of ElasticSearch.
ElasticSearch – This application is the engine of Logstash. It collects, indexes, parses, and searches the log data.
Kibana – The frontend of Logstash and ElasticSearch, this application is the Google search for all your logs. The user can be as basic or as advanced in how they want to search to find log information. The latest release of Kibana adds more visuals based on collected data.
We were already using shared network storage to hold all the log files from our production, quality assurance, and development instances. This meant that 95 percent of our files wouldn’t be difficult to collect. We could simply mount the shared storage to collect the data. We then had to use a shipper for the other 5 percent that don’t sit with the network storage, such as tomcat logs. The shipper runs on the application server and sends all data written to that file for indexing and storage. We spent a few weeks making sure that the shipper didn’t put any additional load on the application servers. Next we had to learn how to grok.
Grok is a pattern set used by the Logstash application to breakdown logging input and make it searchable. (See Grok Patterns) We created multiple fields for items such as email subject lines, IP addresses, etc. that appeared in the logs. The fields gave us the ability to leverage the data collected and the application’s indexing abilities.
Once all the data was being collected, groked (or parsed), and indexed. The Kibana application stepped into the spotlight. While Kibana has a simplistic interface, it is powerful and sometimes cryptic. I had to create a Kibana Cheat Sheet for myself and my team on how to use the web application to find logging data.
Once implemented, a few team members started to use the information to collect data about our environment. However our fall semester started out with our environment crashing on a regular basis, nearly as well scheduled as a subway train schedule. The only humor was that we would regularly be able to call the time for these failures. “Will the 11:03 Blackboard train wreck be on time today?” but I digress. During the following weeks we worked with Blackboard Support on various fixes. The use of Logstash, ElasticSearch, and Kibana really helped us find information and learn more about what was going on within our environment. We were able to find ActiveMQ and file locking issues quickly and discovered issues with session spray using our searches. (See some screen captures of Kibana below.)
The only negative thing I can say about Kibana is that it doesn’t have built in user management or the ability to alert users when specific data appears in the logs. GrayLog2 is an open source project that uses ElasticSearch and Logstash. However it does have the components that Kibana is missing. We might look at moving to it in the future. For right now, Kibana meets our needs.
In the end, these three tools have greatly improved how we can use data to address issues and fix changes within our Blackboard environment. I hope that it helps you when trying to decide how to empower your log file data.
Happy Back to School to everyone! I know that the start of the new school year creates many challenges and issues for Blackboard Learn Administration. For that reason I’m teaming up with my publisher, Packt Publishing, to give away a digital copy of my book Blackboard Learn Administration to three lucky winners!
If you haven’t read about the book, it covers many different topics. Including:
Create, install and update a Blackboard Learn instance
Create and manage courses with Blackboard Learn
Create and administer user accounts within Blackboard Learn
Brand Blackboard Learn with your organization’s other web presences
Integrate publisher building blocks and information systems within Blackboard Learn
Blackboard’s annual users conference kicked off with its technical and geeky side first as The 10th annual Blackboard Developers
From Left to Right: Mike McGarr, Stephen Feldman, David Ashman, and Gary Lang (Image from Blackboard)
Conference started with a day and a half of sessions focusing on four major tracks:
Extending Learn using Building Blocks
Cross-platform and the Cloud
It began with a morning keynote led by Stephen Feldman Vice President of Development, Performance and Security Engineering and Infrastructure at Blackboard and Mike McGarr Director of DevOps for Learn at Blackboard playing co-hosts at the conference’s keynote events. The opening keynote offered insight into the company’s push to implement a new directive from last year that leveraged building blocks to deliver and manage tools that have normally be built into the core.
The first big announcement was the move from the current edugarage.com website to developers.blackboard.com. Mark O’Neil has been heading up this project and the move over to the new URL won’t be swift, but will build on the important documentation and resources that many admins and developers have come to expect. The next announcement was the four different tracks within this year’s conference: Extending Learn using Building Blocks, Performance, Security, & Monitoring, Cross-platform & the Cloud, and Enterprise Integration. Gary Lang, the new Senior Vice President of Product Development took the stage. As of that day, he had been with the company only about ten days. Lang discussed his vision for Blackboard Learn and the creation of a “developer surface area” which will allow developers to integrate with all the different product platforms within the Company.
David Ashman, Chief Architect of Blackboard Learn, came to the stage next to talk about the work that the product team has been doing over the last year. If you have been under a rock or haven’t been following the changes within the Blackboard Learn product. The company has started to make changes to the way tools are integrated by using Learn’s APIs to make changes and improvements to the product without major upgrades. This gives institutions to implement the new tools without major downtime to upgrade all the code within the application. This is a big relief to many, however it still has its teething troubles as programmers have to increase what APIs are available and how building blocks can communicate with the product and other building block tools. In the end, the opening keynote really launched the conversations a bit more for attendees.
Jeremy Wiebold talks with attendees during a presentation. (Image from Blackboard)
One of the best presentations, in my humble opinion, was Nick McClure’s presentation to beginning Blackboard admins (see the presentation online thanks to Blackboard & Echo360). I have always said that empowering and educating the system administrators and support teams for Blackboard Learn instances greatly improves the end users experiences with local support and helps Blackboard’s support personnel quickly diagnose the issues that they face. This is one presentation that should be a requirement for every new admin and should be repeated every year.
The next session was my co-presentation with Jeremy Wiebold about the SIS Integration at the University of Missouri (Presentation Video). The room was much bigger than what I expected and our session was well attended. We got a lot of questions about how we made the move from our snapshot environment to the new SIS Frameworks. There were a few other presentations about SIS Frameworks at DevCon and Blackboard World, each presenter having their own spin on the topic which gave attendees a wide range of views and opinions on how to implement SIS Frameworks. Below is a copy of our slide deck from that presentation. We had a lot of questions from that presentation and I spent quite a bit of the afternoon discussing the SIS Framework process to other attendees.
After lunch, we then got the chance to hear Gene Kim, co-author of The Pheonix Project (Available on Amazon.com and the first 170 pages for free.)
and The Visible Ops Handbook (Available on Amazon.com) give our afternoon keynote. Gene has given many talks and he is an engaging and thought provoking speaker. Below you can find a sample of his presentation recorded by my VIP blogger and colleague Melissa Stange on her Google Glass. He also gave a similar speech during PuppetConf which can be found here on YouTube.
While I haven’t had time to start on the book. Reviews (available at Amazon.com) tell the too often repeated situation in IT departments of an urgent project, a set in stone date, and a lack of planning. The book’s narrative follows the failure and lessons learned from the project’s ashes. In the end, Kim’s point is that many IT Division find themselves in the middle of projects or implementations with timelines that don’t include all effected parties in the conversation. This creates products that have bugs, issues, and force teams to make shortcuts. In the end, Kim’s talk really touches to the need for more Kumbayah than silos within IT. Kim’s publisher gave away 100 copies of The Pheonix Project, one of those copies is mine and was signed by Kim.
The afternoon included a presentation by Matt Saltzman who is a member of Stephanie Tan’s Security team for the Blackboard Learn product. Both Stephanie and Matt are my security heroes when it comes to Blackboard Learn. My presentation was up against Stephanie’s this year so I missed hers (you can see it online), but I wasn’t going to miss Matt’s (See his presentation online). Matt discussed securing your Blackboard Learn environment. This was one of the chapter topics in my book. While Matt did go over a lot of what I discussed in the book, I was able to learn a bit more about port scanning and what information can be collected just from a port scan. He also helped me have more confidence in the recommendations that I made to readers. Which always makes an author feel a little better.
We ended the night with a few laps around the racetrack. The indoor track at Pole Position Raceway took me on a ride which I can say I’ve never experienced in my life. The event was enjoyable for most with video games, food, and an open bar (unless you were racing – no D&D). The evening ended early with a bus ride back to the hotel and prepping for the next day.
Day two started with much more collaboration. I found myself missing out on presentations to discuss options and issues with other attendees or catching up on some emails back at work. One presentation I did get to sit in on was the MBeans exposed! presentation (video available here) by Noriaki Tatsumi and Danny Thomas. They gave a great presentation on how to monitor the Blackboard application with MBeans and JConsole. These situations make me so thankful for the partnership with Blackboard and Echo360 who record many of the presentations given during DevCon and Blackboard World. It was great to see this tweet from Dan Rinzel (dan2bit) on Twitter.
The final keynote announced the winners of the Catalyst awards along with the introduction of a new track for next year, User Experience. It also included some rehashing of things discussed during the event. Mike and Steven both asked for more client presentations for next year’s event and announce the planning for an un-conference which should happen in Winter 2014. The event will be a virtual conference so more people can participate.
For some of you reading this post, this isn’t your first Blackboard World rodeo. For those new to the fold, users annually get together in what I call a cross between a conference, celebration, college homecoming, and alumni event which focuses on the many ways we use, maintain, and upgrade Blackboard products. I thought over the past few days about what my expectations are for the event, what attendees need to hear from Blackboard, and some don’t miss events during the week as well.
While Blackboard World (and the Client Appreciation Party) will clearly miss Blackboard’s former CEO and co-founder Michael Chasen, attendees welcome a new leader to the event as Jay Bhatt takes the reins. I know very little about Mr. Bhatt except from what is available in his biography. (and some googling) While we have come to learn the thoughts and ideals of CTO and President of Academic Platforms, Ray Henderson, I hope and expect that Mr. Bhatt will give attendees a clear understanding of what his vision is for the company and what it means to us.
Last year’s event in New Orleans contained many announcements, most were about the increasing acceptance of industry standards and tightening integration with publishers. I would believe that this continues to be a main focus during this year’s Blackboard World. The company’s wide variety of products and services require this type of integration. We should also get a preview of what new tools, technologies, and advancements the company is making in its different product lines.
Over the past few weeks my blog posts and listserv conversations voiced major concerns regarding the quality, improvement, and issues found in new versions in the Blackboard Learn product. Blackboard, with all due respect, can sometime have a hard time being innovative while ensuring a quality product. They aren’t alone in this struggle which includes most major software companies, however I have overheard admins look at releases and state “There’s more flash than substance.” (or just banging their heads into the table saying “They – missed – the -point -again.”) I’m not sure how, but the company needs to address these issues and concerns that seem to be growing to become the “gorilla in the room” before it turns from whispers into institutions walking away.
Blackboard Mobile Learn was introduced at Blackboard World 2009 and was welcomed by many as how Blackboard would bring its flagship product to the young mobile market. Smartphones and tablets have skyrocketed over the past few years, but Blackboard Learn’s Mobile Learn app has languished. Over the past four years, the application added many new options, but it doesn’t offer the ability to leverage all the product’s tools within Blackboard Learn. The company shows little interest in improving the web experience within the product if users don’t purchase the app (and the app doesn’t seem to be working to represent the complexity and flexibility of the product). I would love to see a mobile roadmap announcement with a projection to bring full functionality with every tool within Blackboard Learn either within the app or using a mobile browser. This is a challenge worthy of the company and the millions of mobile students and increasing numbers of mobile instructors.
2. Issues that must be addressed
While these expectations address what many users expect (or hope) to hear during the conference, there are some glaring issues that the company needs to face and address for its customers. First address the current product roadmap. Some in the Blackboard community have spoken out about issues and concerns with the current service pack / release program for Blackboard Learn. Some institutions can fall six or eight service packs behind the current release between upgrades. While Blackboard recommends upgrading to the latest and greatest version of their software, institutions state the lack of quality and stability in recent releases gives them reason to stay back and accept known issues and bugs within the application. Blackboard should use the conference to announce a major listening program made up of webex, surveys, along with face-to-face meetings to discuss how the company should support and address these concerns with measurable and attainable goals that will allow them to be held accountable.
The current patch, building block, and service pack programs can be a nightmare to understand. When an issue develops within the application, it is hard to know if the issue will be resolved in a patch, building block, or service pack, and the issues could be resolved in different ways for different versions. Blackboard must create a structure that will flag each bug, allow for investigation, then notify and post how each issue will be fixed for what versions. This could be something as simple as a matrix of versions and fix options. Developing a clear picture on how each issue will be addressed helps administrators and support staff at institutions quelch the pitchforks and torches. This process should then be publicized and feedback collected for continual improvement.
As the Blackboard Learn product line moves into the modularization of its tools, administrators need to see a consistent plan for these tool building blocks. If the company plans to continue to use them, they should fall into similar patterns and expected releases like service packs and cumulative patches. When an issue is a bug and the company has no plans to fix it, Blackboard Support personnel should give this information and state that if the institution feels this issue should be patched, they can create a comment in the case supporting their view with examples, if possible. This type of information provides support and product development the input to understand how tools within Blackboard Learn are used “in the field” while showing that the company is listening.
Ray Henderson, during his annual report at Blackboard World, measures and grades the successes of the company on many aspects, however the user community has little tangible evidence to the claims. Last year the company announced major purchases to improve product quality. During the year, administrators have heard from numerous company representatives stating that product quality continues to improve. It would be a welcome addition to Ray’s report card to have measurable analytical data that will back up these facts. A good example would be a table showing the number of known bugs (externally and internally reported) in all supported versions of Blackboard Learn. This information could be collected and kept within the Behind The Blackboard, but this can show how the company is improving the quality as newer versions come out. Even sharing estimates of how much programming work is being done to fix bugs in comparison to new product development would be great. No matter what happens, the vail that many see around the product’s quality needs to be lifted for those supporting the product.
3. Things I don’t want to miss…
My Blackboard World schedule of events has already started to become packed, but there are a few things that I wouldn’t want to miss. First are the three presentations I am participating in during DevCon and Blackboard World. Our SIS programmer, Jeremy Wiebold and I will be presenting twice about how we moved from using Blackboard Snapshot to the new SIS Frameworks at the University of Missouri. I am also co-presenting with Melissa Stange and Francesca Goneconti about Blackboard’s Social Tools feature. So please feel free to come to our presentations, but don’t miss out on many of the other great presentations during both events.
I also recommend that if you have questions, issues, or just need to talk about something that is going on with the Blackboard product you use, either reach out to someone within the Blackboard community or stop and ask one of the many employees at the Blackboard booths or during events and ask them if you can schedule some one-on-one time with them during the conference. I’ve found that Blackboard welcomes constructive feedback to improve the products and services they offer and you never know if your conversation will get the issue resolved. This type of interaction along with the discussions at feedback and user experience sessions really allow us as users give constructive feedback on how the product may work in future releases.
I’m leading a Birds of the Feather session on Thursday, that are great chances to talk with people about different topics many wonder about. These conversations help each user to network and gain insight into what did work and what might go wrong which greatly helps increase the knowledge of the user community as a whole. In general, I find myself spending a large amount of time during the conference discussing issues, scheduling one-on-ones with other attendees to really learn what other institutions are doing. Sometimes these conversations can form partnerships with institutions to create building blocks or share other data. In the end, it’s a win-win.
I hope to use this post to help me digest and review how Blackboard World went. During the conference, I will work to post a daily recap of each day, which (I hope) highlights the important aspects that you may have missed. Two or three weeks later, I hope to have my review posted here as well.
I'm a Blackboard Learn Administrator that designs and creates excellent online education environments that meet student, faculty, and institution needs. I'm a Blackboard Catalyst Award Winner and the author of Blackboard Learn Administration from Packt Publishing. I enjoy photography, video games, & cooking.
Order My Book…
Blackboard Learn Administration By: Terry Patterson