Hospital Digital Experience Index: An Experiment to Define Website Benchmarks

March 27, 2017

// By Jane Weber Brubaker //

A consumer’s experience on a website may be less than perfect at times. In most cases a badly designed website isn’t life-threatening, but in healthcare, being able to find the right information at the right time could actually make a difference. Are health system websites equal to the task? How easy do they make it for consumers to quickly find what they’re looking for?

HDX-15, the Hospital Digital Experience Index, is an experiment designed to investigate these questions. Created and conducted by digital experience agency ConnectiveDX, HDX-15 assesses the websites of the top 15 U.S. hospital systems, ranked by U.S. News & World Report.

The project evolved organically from the agency’s work with its own clients. “We often do competitive assessment when we’re working with our healthcare clients,” says Dave Wieneke, digital strategy practice director. “Our experience designers figured out that they could share good assessment practices and start rolling up knowledge from one engagement to the next.”

But why limit it to clients? Why not develop benchmarks that could serve the industry as a whole? Based on that premise, the team at ConnectiveDX created an assessment tool to systematically and objectively quantify the best digital innovation. Evaluating the “Best Hospitals” websites was a logical place to begin.

Wieneke worked with several healthcare organizations to validate the usefulness of the tool. Phoenix Children’s Hospital was an early beta tester. “I saw some potential in it when we first got started, but the value exceeded my expectations,” says Jared Johnson, marketing technology manager.

At HCIC 2016, Wieneke and Johnson presented an overview of the index and the results of their collaboration. We followed up with them to dig into the details of the assessment process and find out:

• How does the tool actually work?
• Who can use it?
• Can objective data on website functionality settle internal arguments about design priorities?
• Can benchmarks help the industry make progress faster?

How to Get an “A”

A perfect score on the Digital Experience Index is 100. That means a website would rank number one on all questions that require ranking, and score five on all questions scored on a scale from one to five. No one included in HDX-15 Index achieved that level of perfection. There were some notably high scores in certain categories, and the spread of scores illustrates that there is room for improvement, even among this group of recognized leaders.

Defining Usability

The HDX-15 Index looks at four website capabilities: clarity, functionality, use cases, and impact. It assigns a score for each category, as well as an overall score.

1. Clarity

Clarity looks at brand clarity, plain language, and geographic complexity.

Brand Clarity

How well does a hospital’s website express its brand clarity? NYU Langone promises on its homepage, “Your care starts here.” Penn Medicine proclaims that “Life is worth Penn Medicine.” UCSF Health conveys its caring and humanity without words, featuring a video of Duke, the Therapy Cat who comforts patients in the ICU.

Plain Language

This category also looks at readability of website content, a key factor in promoting health literacy. The average adult reads at an eighth-grade level, but one-third of the systems in the HDX-15 Index had key pages written at a graduate school (17th grade) level. If patients don’t speak English at all, how well do websites accommodate them?

Geographic Complexity

Simplifying geographic complexity is a major challenge for large health systems due to constant growth, consolidation, and expansion. Patients need clear information about how to get to the right location, and how to navigate to their final destination once they get there.

2. Functionality

How helpful is the Find a Doctor section? Can patients easily find the information they’re looking for using a website’s site search functionality? The index looks at how well a website helps a site visitor:

• Structure a request
• Receive results
• Filter the results
• Read or archive resulting pages

Functionality also looks at site speed. A page that loads too slowly on a desktop computer or a mobile device could drive patients away—to a competitor, for example.

3. Use Cases

This category of the index evaluates how easy it is to find information or complete certain tasks:

• Getting information for an emergency room visit
• Encouraging referring providers to recommend patients
• Communicating about services for patients traveling internationally or regionally

4. Impact

This category looks at four metrics:

• Unique visitors
• Time on site
• Google’s site authority score
• Dominance in local search results

Here is a summary of how the hospitals in the HDX-15 Index ranked, including only the top five in each category:

	Clarity	Functionality	Use Cases	Impact
NYU Langone	1	4	5
Penn Med	2	5
UCSF Health	3	1
NW Medicine	4
NY-Presby	5			3
Stanford HC		2	2
Cleveland Cl		3		5
Mayo Clinic			1	1
UCLA Health			3
Mass Gen			4	4
UPMC				2

Inside HDX-15

Here, we’ll look under the hood of the assessment methodology and see the types of criteria the assessors actually use to score and rank websites. These scoring rubrics guide the assessors by providing specific conditions that must be met to achieve each score level.

Here are some examples:

Scoring Rubric for Finding the Emergency Room

• A link to Emergency services is visible on the home page.
• Emergency services have a clear location and phone number as an obvious part of their section page.
• Directions to Emergency locations are easy to find.
• Additional information (such as when to use Urgent Care, wait times, specialties, and other expectations setting) is available.
• Wild card: Add or subtract for noteworthy simplicity or innovation, or unexpected or notable deficiencies. (Collect images of these for improving future design.)

Scoring Rubric for Referring Physician Engagement

• Content for referring MDs was found.
• A link for Referring MDs or HCPs is on the home page.
• The site describes service levels referring MDs and their patients can expect, seeking signs of collegiality and partnership.
• Observed description of chart sharing, or other signs of strong capabilities for care coordination.
• Wild card: Add or subtract based on other high or low points, e.g., positive: referring MD programs, newsletters, testimonials, and negative: dry, pro-forma, or high-handed copy.
  

Scoring Rubric for Reading Level

• Under 9^th grade reading level
• 9^th – 10^th grade reading level
• 11^th – 13^th grade reading level
• 14^th – 17^th grade reading level

Reading level is evaluated using the Gunning Fog Index. The number of words per sentence, number of punctuation marks, and number of words with three or more syllables impact the reading level.

The tool uses ordinal data to rank websites on attributes such as site speed, unique visitors, and time on site.

How Phoenix Children’s Hospital Uses the Digital Experience Index

The HDX-15 Index is a set of discrete data related to 15 top hospitals, captured at a point in time. Here, we’ll look at how another health system applied the methodology in its own context. Phoenix Children’s Hospital used the scale to compare itself to three key competitors. Digital experience designers assessed the four websites using the same methodology used in HDX-15, and came up with these results:

“We are actually right in the thick of those top 15 in terms of overall score,” says Johnson. Scores in each of the four quadrants showed where they were ahead as well as where they had opportunities to improve.

Data Helps to Resolve Design Disputes

Detailed feedback on site speed on mobile devices, which falls within the functionality quadrant, was particularly useful for Johnson in internal stakeholder meetings. PCH is preparing for a site redesign, and design considerations include impact on how fast pages can load. “We look at the website in most professional settings on a large screen. Ultimately mobile tends to be more of a checkbox,” Johnson says. The average person is willing to wait only three or four seconds for a page to load on a mobile device, he notes, but the average site takes 18 to 20 seconds. “Having that number in the index measurement made it a lot easier for us in conversations with senior leadership to say, ‘Site speed is important, and here’s why.’”

Wieneke notes that the report gave the PCH team new insights on where to focus. “They were able to say, here’s an area where we’re already strong. Maybe we need to focus on either an area where no one else is, or an area where we think we need to be stronger,” Wieneke says.

Vision for the Future

Johnson hopes the benchmarking tool will begin to establish consistency across the industry. “I’d love to see it adopted as more of a widespread standard,” he says. “As hospital marketers, we want to be able to have greater defined user standards so that anyone, anywhere knows where to find certain information. We can do a better job as an industry improving things.”

Wieneke wants to put the benchmarks in more people’s hands, refining and improving them over time. He envisions a free “home edition” of the tool: “I think the more open we are with it, the more useful it will be rather than black-boxing it up and renting access to it.”

For Johnson, usability is all about engagement. “Most Americans are out there just begging for somebody to take them by the hand and guide them through this new landscape,” he says. “I definitely think benchmarking has an opportunity to help us define more clearly how to do that.”

For more information, download the Hospital Digital Index report.

Jane Weber Brubaker is editor of eHealthcare Strategy & Trends.