communication data science
los angeles, ca
One of the City of Austin Fire Department's most-used application is QlikView, their web-based Business Intelligence (BI) system, which provides analytics and reporting. BI enjoys very high user adoption—an unusual and remarkable success when compared with typical BI implementations. However, the most-used BI application—the "Battalion Chief/Company Dashboard"—has been in use without a facelift since 2008.
How might we increase the usefulness and usability of the BI visualisations included in this app?
QlikView apps are a data visualisation tool. More apps are published as people continue to request certain data, but old apps are not updated. Furthermore, user value has not been measured as pushing out new apps has been prioritised over seeking user feedback on existing apps.
Over the course of two weeks, I drove out to stations and interviewed ten people about their QlikView usage. The majority of the people I conducted interviews with were users who worked in operations, including firefighters, lieutenants, captains, and battalion chiefs.
During my one-on-one interviews, I asked users which apps they used most and had them guide me through how they used them whilst thinking aloud and explaining their reasons for doing so in that way.
The most popular apps were the BC dashboard, HR dashboard, and Seniority apps; the BC dashboard provides most of the information users need on the daily, and the HR dashboard and Seniority apps are mostly used out of curiousity, for example to figure out promotions and vacation selection. They are also the apps that most users have permissions to view.
The BC dashboard is mainly for battalion chiefs, who use the app to cross-check data for data quality. However, lower ranking ops also check the BC dashboard often to see what the higher ups see and stay off their radar.
To manipulate data visualisations on the front-end, users can manipulate the left sidebar selectors. A common use of this feature is to see if there's any data missing, for example indicated in bright red in inspections, or shown as missing in CE completions. Users found this function intuitive and it is the main way they interact with the app.
However, that is the extent to which users understand how to use the apps. They miss the full capabilities of the apps, not realising how much they can customise on the front-end to personalise visualisations. Developers regard this feature as powerful, and users don't realise it exists.
Ops don't have the same technological intuitions as people in headquarters or the common technology user. If they have free time, they don't use it to play around with apps and discover new features, but rather use it to train and prepare to save lives. For example, they don't have the intuition to check out the Read Me tabs if they have questions, thinking that that's a note for developers rather than users. They get confused by the browser toolbar, app toolbar, and app navigational tabs, sometimes refreshing the whole browser in order to view a previous page that could have been navigated to within the app.
Users don’t know what’s available to them. They’ve done things their own way and will continue doing it that way until someone shows them otherwise. Even certain features that are located front and center are completely overlooked, such as the turnout time box chart on the BC dashboard. However, once learned, their retention is promising.
Not a single person I interviewed received training for QlikView, whereas I spent my first morning at AFD headquarters doing online QlikView training. Users need to learn what is available and how to use it. Thus, the two main focuses should be exposure and training. Introducing tech options early in the career is critical. It could be helpful to have a definitive guide that indicates which ranks have what responsibilities, which tools are available to aid them with those responsibilities, and teach them how to use those tools.
Another aspect that could aid the learning process is to work on consistency between the different apps, i.e. background colours, toolbar icons and functionalities, logo placement. All the apps are developed by a team of developers who do not have a shared style guide. Creating a design standard and sticking to it can speed up the development process and also help users know what to expect and learn how to use the apps more efficiently and effectively.
Some low-hanging fruit to take care of are creating a visualisation for physicals like there is for CE completions, add a filter by engine (rather than by station, like it currently is), and to allow users to view total runs in their career.