Dallas reporters use AHRQ data to measure patient safety
Filed under: Health data, Health journalism, Public health, Public records, Tools
The Dallas Morning News continues its 19-month investigation into patient safety at UT Southwestern Medical Center and Parkland Memorial Hospital.
The project, “First, Do No Harm: An investigation of patient safety in Dallas hospitals,” is behind the website’s paywall but The Dallas Morning News has granted AHCJ members access. To find out how to access the stories, please click here and log in as an AHCJ member.
Among the latest reporting:
Dallas Morning News reporters Ryan McNeill and Daniel Lathrop took advantage of AHRQ’s Patient Safety Indicator (PSI) software, typically used internally by hospitals, to process 9 million publicly available patient records from Texas hospitals, all of which came from between
Parkland, the prominent local hospital that has earned scrutiny on numerous prior occasions, was just the most notable of a number of area hospitals that came up short (and generated headlines), but our interest lies more with the reporters’ investigative methodology as well as the path they’ve blazed for broader hospital quality reporting.
All their work was done in consultation with experts in the field, including academics, government officials and hospital administrators. An outside review indicated McNeill and Lathrop used the software properly, and their results were in line with a similar public analysis. But that’s not to say it was a simple process.
The newspaper spent six months analyzing nearly 9 million state hospital discharge records using Patient Safety Indicators, or PSI, software. This highly sophisticated system was designed for the federal government as a tool to measure potentially preventable complications among hospital patients.
The PSIs do not present a complete safety picture because they are based on administrative data — a summary of diagnoses, procedures and outcomes derived from patients’ medical charts, as opposed to a complete review of all medical records.
It’s not a perfect measure, but it’s one of the best available.
PSIs “reflect quality of care inside hospitals,” according to the Agency for Healthcare Research and Quality, a division of the U.S. Department of Health and Human Services. It released the PSI software in 2003 and periodically updates it, most recently in August. The News used that version for its final analysis.
The software analyzes the administrative data that nearly every hospital in Texas reports to the state. No patient-identifying information is included.
The results on 15 PSIs are statistically “risk-adjusted” because some hospitals treat a disproportionate share of unhealthy patients, who face a greater risk of potentially preventable complications. Rates from eight of the indicators are used to determine a hospital’s patient safety “composite score.”
The AHRQ has just started posting some PSI measures on Hospital Compare, and the Texas health department plans to follow suit in 2013, but reporters looking to get their hands on a broader swath of the data will still have to follow the Dallas duo’s do-it-yourself approach.
The reporters’ work drew criticism from the Texas Hospital Association, which said the methodology was “not intended for use in public reporting.” McNeill refutes its claims in a blog post. Daniel K. Podolsky, president of UT Southwestern Medical Center, also sent a letter criticizing the reporting. George Rodrigue, managing editor of The Dallas Morning News, published a point-by-point response to Podolsky’s letter.
Data shows disconnect between patient perception, hospital performance
Filed under: Health data, Hospitals, Hot Health Headline, Public records
Sifting through Medicare hospital rating data, USA Today reporters Steve Sternberg and Christopher Schnaars found an enlightening disconnect between patients’ subjective ratings of hospitals and hospital performance on quantitative measures such as death and readmission rates.
“This is a very important finding,” says Donald Berwick, director of the Centers for Medicare & Medicaid Services, adding that though patient-survey data offer critical insights into how it feels to be a patient at different hospitals, patients’ perceptions don’t tell the whole story.
The story is packaged with an infographic that allows readers to look up ratings for local hospitals.
AHCJ resources
- Updated hospital data allows reporters to identify ongoing problems: The release this month of federal data on hospital quality is a good reminder for reporters to give their local hospitals a checkup. To give you a head start, Charles Ornstein, senior reporter at ProPublica and AHCJ’s president, has done some preliminary analysis and points out states in which hospitals fared well and the states where hospitals did poorly.
- Deciphering hospital quality data
- Sorting out hospital rankings
- Making sense of hospital quality reports
- Analyze patient satisfaction surveys to evaluate local hospitals
Series reveals gaps in communication of hospital inspection results
Filed under: Government, Hospitals, Hot Health Headline
Jodie Jackson Jr. of the Columbia (Mo.) Daily Tribune took an in-depth look at patient safety at University Hospital, part of the University of Missouri Health Care system.
Jackson found that inspections, by CMS and the FDA, have repeatedly turned up systemic practices that compromised patient safety. At the same time, the Joint Commission awarded the hospital a full accreditation, raising questions about why the agencies don’t share information.
In a blog post, Jackson, a Midwest Health Journalism Program Fellow, says he has “examined some 700 pages of documents and have had national infection control leaders examine the reports that formed the basis for the series.”
- Patients in Peril?
- Hospital infections seen as ‘winnable battle’
- Inspection reports, hospital at odds
- Wash your hands
- ‘Superbug’ spurs patient’s activism
- Years after surgery, struggle continues
Critics point out issues in patient satisfaction ratings
Filed under: Health data, Hospitals, Hot Health Headline
On the heels of a government proposal to tie hospital incentive payments to patient satisfaction ratings, a few outlets have started looking at the validity of such measurements.
At HealthLeaders Media, Cheryl Clark reports that regional differences in tendency to be satisfied (the numbers show that New Yorkers are harder to please than Midwesterners and New Englanders, for instance) mean that any absolute number thresholds issued by the feds would penalize hospitals in parts of the country where folks are less likely to respond well to surveys.
And on KevinMD.com, William Sullivan, D.O., J.D., takes a few swings of his own, first taking aim at the ratings’ sampling and statistical grounding, then moving on to what he says is hospitals’ over-reliance on percentile quality ratings.
The problem, according to Sullivan? Overall patient satisfaction is quite high, thus doctors’ ratings cluster tightly around the low 90s on a 100-point scale. That means even a small shift in absolute rating will cause a huge jump in percentile. On at least one system, a 4-percentage-point absolute drop will take a doctor from the 90th percentile to the 50th. And, thanks to the aforementioned sampling issues, that drop can be caused by a handful of particularly ornery patients. Patients who, Sullivan writes, are thus given massive leverage.
With our employment and our compensation hinging on every “5” we can get, doctors are being coerced into giving patients whatever they want, regardless of medical appropriateness. When we cater to satisfaction scores more than we cater to proper medical care, we are violating our oath, devaluing our education, and potentially harming our patients.
AHCJ Resource:Analyze patient satisfaction surveys for your local hospitals
“Numbers can be a start - not the end - of a story,” the AHCJ website notes. Remember that patient satisfaction scores only mean so much. Sometimes the best doctors have gruff demeanors while those with inferior skills have great bedside manners. Patients may not recommend hospitals to friends because they dislike the food or think their roommates were too loud. But if patients report that doctors or nurses didn’t communicate well, that very well could affect the care the patients received. Using data can give you a valuable tip sheet to generate ideas and questions in your pursuit of a story.
For hospital overall survey results, AHCJ includes comparison of data first released in March 2008 then updated quarterly, allowing journalists to compare overall survey results over a lengthy timeline.
Yale doc knows the right data is out there
Writing for Forbes Magazine, Matthew Herper profiles Harlan Krumholz, the pioneering quality of care researcher and Yale cardiologist. During his career, Krumholz, 52, has been at the leading edge of everything from Hospital Compare and angioplasty delivery times to recent headline grabbers like the Vioxx suit and the 2009 study on radiation exposure during routine scans. The highlight of the profile comes when Herper highlights Krumholz’ knack for picking out just the right metrics with which to hold feet to the fire.
Harlan M. Krumholz, M.D.
By figuring out what to measure and how, he showed that even top hospitals were systematically underperforming, largely because no one was tracking the results.
…
Krumholz’s basic idea is that if you ask the right question and pick the right measurement, you can figure out a way to get the answer, often using billing records or existing databases. This frequently involves partnering with insurers or Medicare. He has a knack for focusing on performance metrics that hold hospitals accountable.
Mass. won’t post hospitals’ death rates
Filed under: Health data, Hot Health Headline, Public records
The Boston Globe’s Liz Kowalczyk reports that, two years after it was first proposed by a consumer group, the Massachusetts Health Care Quality and Cost Council has decided it won’t publish hospital-wide mortality rates. The problem, it seems, is the lack of an accurate, universal method of computing such numbers.
Health and Human Services Secretary Dr. JudyAnn Bigby, who heads the group that made the decision, said current methodology for calculating hospital-wide mortality rates is so flawed that officials do not believe it would be useful to hospitals and patients and could harm public trust in government.
It appears, Kowalczyk writes, that general hospital mortality rates just aren’t “ready for prime time” quite yet.
The council convened an expert panel, which worked with researchers to evaluate software of four companies for measuring hospital mortality. The problem was that researchers came out with vastly different results when they used the various methodologies to calculate hospital mortality between 2004 and 2007 in Massachusetts, and they could not tell which company’s results — or if any — were accurate.
Analysis of billing record data reveals hospital quality issues in Las Vegas
Using data from hospital billing records, Marshall Allen and Alex Richards of the Las Vegas Sun have been able to identify “hospital-acquired patient harm,” that is, events in which patients are harmed while in the hospital.
Medicare does not pay for these “never events” and so they are reflected in hospital billing codes. Such events include things like leaving foreign objects in a patient, bed sores, falls, infections related to catheters or surgical sites, blood clots and poor glycemic control.
Nevada – and 40 other states – collect such data for analysis, Allen and Richards report. In Nevada, the state had not yet analyzed the data so the reporters requested it and did the analysis.
The pair requested and received records for “every Nevada hospital inpatient visit going back a decade — 2.9 million in all. The information, coupled with interviews with more than 150 patients and health care insiders, has yielded a sweeping and detailed portrait of hospital care in Las Vegas.”
The project includes stories about patients who were harmed while hospitalized, the documents behind the reporting, data tables, interactive graphics and more.
Patient safety expert Pronovost is keynote speaker
Peter Pronovost, M.D., Ph.D., a professor at Johns Hopkins University and founder of the Quality and Safety Research Group, will be the keynote speaker at Health Journalism 2010. He will appear at the awards luncheon on Saturday, April 24.
Peter Pronovost, M.D., Ph.D.
Pronovost specializes in improving patient safety through methods that are scientifically rigorous but feasible at the bedside. In his new book, “Safe Patients, Smart Hospitals: How One Doctor’s Checklist Can Help Us Change Health Care from the Inside Out,” Pronovost tells of losing his father as the result of a medical error and his own journey from a researcher to an international leader in patient safety.
Pronovost joins a number of high-profile speakers. Conference participants will have the chance to attend newsmaker briefings featuring leaders from the Department of Health and Human Services, the Centers for Disease Control, the Food and Drug Administration:
- Thomas Frieden, M.D., M.P.H., director, Centers for Disease Control and Prevention
- Kathleen Sebelius, M.P.A., secretary, U.S. Department of Health and Human Services
- Jeffrey Shuren, director, Center for Devices and Radiological Health, Food and Drug Administration
A special track on assessing health reform is intended to help reporters understand the changes coming and better explain what’s ahead to their readers, viewers and listeners:
- Does comparative effectiveness research work?
- Outlook for the nation’s hospitals
- Is there a looming doctor shortage?
- What’s ahead for state and local governments
- The reporting challenge going forward
“Influenza! Lessons learned from a year of H1N1″ will feature experts on public health, infectious diseases, preparedness and vaccines:
- Jeffrey Levi, Ph.D., executive director, Trust for America’s Health
- Anne Schuchat, M.D., director, CDC’s National Center for Immunization and Respiratory Diseases
- Litjen Tan, Ph.D., director of medicine and public health, American Medical Association; co-chair, National Influenza Vaccine Summit
- Moderator: Maryn McKenna, independent journalist, Minneapolis
See the complete conference schedule.
Reports reveal problems in England’s NHS
England’s Care Quality Commission, a regulatory agency, has found that a quarter of the National Health Services hospital trusts fail to meet basic standards of hygiene, according to The Telegraph’s Andrew Hough.
Some of the failures included 36 trusts not providing areas to decontaminate instruments, three trusts failing to regularly flush unused water outlets while more than a dozen trusts failed to keep clinical areas clean.
As Hough reports, the revelations come just days after a BBC investigation found that hospital trusts have given incorrect information on their performance and quality of care.
Related
Conflicting demands on their job and being rushed or understaffed were common problems revealed by a recent survey of employees of England’s National Health System, as The Telegraph’s Rebecca Smith reports.
The NHS, according to its Web site is “the world’s largest publicly funded health service” with more than 1.7 million employees. The survey was done by the Care Quality Commission.
The CQC reports some improvements in job satisfaction, however:
Approximately half of all staff would recommend their trust as a place to work, and just under two thirds are happy with the standard of care provided by their trust. There has also been a substantial rise in the % of staff saying that they have had training in infection control.
Joint Commission finds improved hospital quality
Filed under: Conflicts of interest, Health data, Hospitals, Pharmaceuticals, Studies
The latest report from The Joint Commission, a hospital accrediting organization, finds that “overall, hospitals are following evidence-based standards for treatment of myocardial infarction, heart failure, and pneumonia,” as MedPage Today reports.
The report, which looks at 31 evidence-based measures, did find decreases in two areas: measuring oxygen in blood for pneumonia patients and administering antibiotics to pneumonia patients in the intensive care unit within 24 hours.
The report, “Improving America’s Hospitals: The Joint Commission’s Annual Report on Quality and Safety 2009,” (PDF) and those from three previous years are available on the commission’s Web site. Among the key findings:
- Hospitals accredited by The Joint Commission have significantly improved the quality of care provided to heart attack, heart failure and pneumonia patients over a seven-year period.
- Hospitals have steadily improved on individual surgical care performance measures – as well as on additional individual heart attack and pneumonia care measures - over a two-, three- or four-year period.
- Hospital performance on two individual measures of quality relating to inpatient care for childhood asthma is excellent after only one year of measurement.
- Improvement is still needed.
- Where a patient receives care makes a difference.
As ProPublica’s Charles Ornstein explains in his tip sheet, The Joint Commission does routine inspections of participating hospitals to ensure they meet the standards required for accreditation. It compiles public reports on each hospital, which are available on the qualitycheck.org Web site. These reports include the hospital’s accreditation status, as well as some data on hospital outcomes and practices.
It does not release its detailed inspection reports to the public, and many states’ open records laws specifically exempt the reports from public disclosure. In the past, these inspections have not been surprises, and the group has been faulted for being slow to act against hospitals with problems Also, The Joint Commission rarely takes punitive steps against hospitals, preferring to work with them to improve.
Tip Sheets
A road map for covering your local hospital’s quality
Study: Hospital quality comparisons are inconsistent
News: Congress requires Joint Commission to re-apply for accreditation privileges (Sept. 17, 2008)
-
Join us in Atlanta
for Health Journalism 2012
Fellowship applications
due Feb. 27
New at HealthJournalism.org
- Data gives depth to investigation of patient safety
- What reporters should know about Alzheimer’s and related dementias
- Apply for fellowships to attend Health Journalism 2012
- AHCJ asks Supreme Court to permit broadcast of arguments in health reform case
- Tips for print, online journalists to embrace broadcast opportunities
- Numbers reveal how often, or how rarely, states check doctors’ disciplinary records
- How I did it: Falsified charts in nursing homes linked to patient care
- Access Denied: Looking at a lack of health care availability
- Transparency & medical errors: Public reporting of patient harm is a potent force
