Video transcript: Mental health: Effectiveness of the planning to discharge people from hospital
Title: logo for the Office of the Auditor-General
Many people in New Zealand face mental health issues, some of which are serious enough to require hospitalisation and highly specialised care to support people through some of the most difficult periods in their lives. How do the people who provide mental health services measure how much of an improvement they make in the lives of their patients.
To find out, we undertook a performance audit focused on people who have spent at least one night in an inpatient unit in a mental health hospital. We wanted to understand how well these people are supported in their transition back into community-based services to give them the best chance to live with as little disruption as possible in the presence of their mental illness.
To help us do that, we used millions of anonymised patient records to build our own data model which placed people at the centre. We did this because we were determined that the focus of our work should be the people, rather than the systems which serve them.
Our data model allowed us to analyse how well the numbers reported by the sector reflect any real improvements in the service. The two indicators which we studied in detail were re-admission and follow-up rates. We discovered that these measures are heavily impacted by a small number of patients, a factor which can skew the overall picture and make trend analysis difficult.
However, we also consider that there is great potential to do more with the available data to create better measures that more accurately represent service outcomes beyond aggregated averages and tallies.
By making our thinking transparent and by incorporating feedback from clinicians and analysts in the sector, we constantly refined the quality of our analysis to develop the best independent perspective we could.
In keeping with our aim to focus on outcomes for people, we built upon some innovative thinking being done by a team of analysts at one of our DHBs.
By plotting the intensity of a patient’s interactions with the service, they could show the extent to which peoples’ lives were being disrupted by their mental health conditions, as well as indirect evidence of whether things looked like they were getting better or worse.
They called this kind of picture a “patient journey”.
When they combined the journeys of many people in a single picture using a heat-map to show the more intense spots, they discovered that there were many variations in patient experiences and that studying these patterns could help clinicians better understand the needs of their patients. They could also help service designers think of ways to more accurately measure outcomes.
Our audit team adapted the patient journeys concept and began to develop interactive visual maps of the lives of the people we were studying. Starting with an empty timeline stretching out over the four years for which we had data, we began to colour in the various types of interactions between patients and the different service teams. We added markers for some of the existing indicators and also included interactions with non-mental-health services (such as visits to Accident & Emergency wards, or treatments for chronic conditions such as diabetes).
When we drew these pictures for all 20,000 people in our study, some very interesting and informative patterns emerged. Most people spent short periods in hospital, but a small number seemed to stay for months or even years at a time. Many people received regular care both before and after a hospital stay, but there were others who seemed to arrive in hospital abruptly and then seemingly disappear just as suddenly. These pictures can’t capture a person’s complete story, but they did help us to ask better questions.
Multiple patient journeys viewed together resemble a kind of “fingerprint”: each individual story unique but also a thread in a much wider narrative weaving thousands of peoples’ lives together through their interactions with mental health services.
By comparing the fingerprints of different DHBs we could see how, despite some significant differences, many larger patterns remained similar. Each DHB, regardless of size, has a small proportion of patients with complex needs whose interactions look very different to those of most patients.
In fact, these people with highly complex needs make up a small and identifiable group of fewer than 100 people nationwide. These are highly impactful patients with unusually long hospital stays covering months or even years at a time, yet they are not well represented by the existing performance indicators which don’t differentiate between groups of people with distinct needs.
Some health services elsewhere in the world have adopted a data science approach toward using patient data to cluster similar patients so that they can better classify risks in an attempt to predict critical events such as relapses before they happen.
Our summary view is that the mental health sector collects a wealth of valuable data about the people it services. There are many energetic and innovative people working in the sector who are trying to find ways to use this data to improve upon how services are measured, designed, and ultimately delivered for the benefit of the people they serve. We hope that our work has shed a helpful light on a few of these innovations and that the sector finds ways to improve its support for the people who developed them. Doing so will move the sector closer to realising the untapped potential of its extensive data collections.
Mental health services are highly complex and demanding, and that’s unlikely to change. We believe that the collaborative work on this performance audit is a showcase for how agencies can combine their efforts, share their data, pool their expertise, and act collectively to improve services and build better lives for the people of New Zealand.