Star ratings at odds with compliance data
Report exposes flaws in the system designed to increase transparency and support informed choice.
Non-compliant homes are scoring four or five stars for compliance against the government’s own guidelines, bringing into question the accuracy of the overall rating system, says a report.
Dr Rodney Jilek, managing director of Aged Care Consulting & Advisory Services Australia – which published the report earlier this month – told Australian Ageing Agenda the compliance scores contradict Department of Health and Aged Care advice.
“The guidelines are so clear about the compliance ratings,” he said. “Basically, to get a 5-star compliance rating you need to have no non-compliance for three years. For a 4-star rating you need to have no non-compliance for 12 months. Therefore, anyone on the non-compliance register can’t have a 4- or 5-star rating – not possible.”
The star rating system – which came into play in December 2022 – was established to increase sector transparency and to help older people and their families make informed choices when selecting an aged care facility.
Under the government’s star rating system:
- 1 star indicates significant improvement needed
- 2 stars show improvement needed
- 3 stars equal an acceptable quality of care
- 4 stars suggest a good quality of care
- 5 stars highlight an excellent quality of care.
Aged care services are awarded a star rating based on four sub-categories: residents’ experience (which accounts for 33 per cent of the overall rating), staffing (22 per cent), quality measures (15 per cent), and compliance (30 per cent).
The compliance component of the star rating is measured against a provider’s performance in relation to government regulations and standards as outlined in the Aged Care Act 1997. This information is submitted to the department by the Aged Care Quality and Safety Commission.
However, mining data from the commission’s Non-Compliance Decision Log, the 158-page report – The failure of the aged care star rating system – shows facilities found non-compliant rated as “good” or “excellent”.
“We targeted the homes off of the non-compliance register mainly because they are supposed to be the worst operating homes in the country – they’re all non-compliant – so you would reasonably expect that they would represent the lowest ratings,” said Dr Jilek – also founder and managing director of Community Homes Australia, a registered disability provider specialising in dementia care.
“But it’s clear that there was a significant number of homes that have either 4- or 5-star ratings. Given the compliance rating makes up 30 per cent of the total star rating, that means a significant number – around 150 homes out of 500 – were automatically rated incorrectly.”
The report shows for the 12 months from 1 November 2022, of 501 homes listed on the non-compliance register, 68 services were rated as 5-star for compliance and 81 services had a 4-star compliance rating.
“The concern is we’ve got a minister spruiking the success of the star rating program and saying that this is a way for increased transparency, and people can use the star ratings to make informed decisions and comparisons about aged care. What we’ve found is that’s not actually true – you can’t do it because the ratings are not correct,” said Dr Jilek.
He told AAA he saw two possible explanations for the incorrect ratings: “One – there’s just an unacceptable level of human error. Or that it’s been done deliberately to skew the results so that they are reporting higher than they actually are.”
AAA contacted both the office of the Minister for Aged Care Anika Wells and the commission for an explanation, but had – at the time of publishing – received no comment.
“We haven’t got a very robust system.”
The report points to other flaws in the government’s star rating system. Such as the resident experience component, which Dr Jilek describes as “completely useless”.
“It’s an annual survey that takes into account 10 per cent of the residents in a home. The whole methodology is flawed because the provider has certain – if not complete control.”
Dr Jilek also dismisses the quality measures and the staffing elements of the ratings system. “They are all completely unvetted, provider-supplied data,” he said. “There’s no checks and balances to say that the data is correct. There’s no validation process.”
The discrepancies surrounding the compliance scores follow recent revelations that care minute reporting for the November-December 2022 quarter was incorrect. That issue also raised questions about the accuracy and value of star ratings because of the delay from data collection to star rating publication with ValleyView Residence CEO Mark Sheldon-Stemm calling the system worthless.
“The star ratings that are about to be published do not reflect the up-to-date quarter (July to September 2023) and the latest resident survey. Publishing out-of-date information is misleading and defeats the purpose of any rating system,” Mr Sheldon-Stemm told AAA in December. He said “the system should be scrapped as it does not serve the purpose for which it was intended.”
Dr Jilek agrees. “We’ve now got the care minutes they’ve said is wrong. We now know the compliance rating is wrong. And we also know that the resident satisfaction survey – those results are not representative of the cohort of residents. So we haven’t got a very robust system,” said Dr Jilek. “Its stated goals were to improve transparency and allow people to make a like-for-like comparison of services – you cannot do that at all. It completely fails.”
In light of the findings, Dr Jilek told AAA that the government should abandon the star rating system altogether. “It needs to be pulled and they need to go back to the drawing board and work out a more appropriate robust way of ranking service quality so that people can make informed choices … At the moment, the system is saying that – even if you fail the most basic minimum standards of accreditation – your home is still acceptable. The question that we pose is: acceptable to whom?”
Comment on the story below. Follow Australian Ageing Agenda on LinkedIn and Facebook, sign up to our twice-weekly newsletter and subscribe to AAA magazine for the complete aged care picture.
Having just had direct experience in finding an Aged Care facility for a relative I can only say I agree totally that the star ratings and general assessments are very flawed. In addition, the processes re financial costs, exceptions, additions etc are complex, long winded and difficult to understand. Pity help anyone with English as a second language ! As a former public servant, accustomed to long winded, elaborate texts I found myself struggling to extract the information . My other family members had a similar complaint.