Identifying high quality providers in the presence of heterogeneous preferences

Why is it so difficult for health care payers to identify a “best” provider?  A paper by Gutacker and Street (2017) explains:

There are two key elements that complicate assessment of how well public sector organisations are doing their job (Besley & Ghatak, 2003; Dixit, 2002). First, they lack a single overarching objective against which performance can be assessed. Instead, they pursue multiple objectives, and this requires performance measurement along a range of performance dimensions. These objectives may conflict, such that higher performance along one dimension may come at the expense of performance along another. Second, they typically serve several stakeholders, including those using services, tax-payers, regulatory bodies, and politicians. The values that stakeholders attach to objectives are often not known and unlikely to be identical.

This is a fundamental issue whenever decisions are made by a centralized authority rather than by individuals.  One solution is simply to weight all the objectives and create a composite score.  Then providers can be compared based on this composite.  However, these weights may not reflect individual stakeholder preferences.

The solutions that the authors propose is a multidimensional performance assessment framework.  In theory, this approach is simple.  To take a concrete example from the paper, assume that stakeholders care about hospital length of stay, waiting times, readmission rates, and patient reported health status after treatment.  If a hospital is superior along all dimensions, then clearly that hospital is better than one that ranks worse on all these dimension.  Hospitals that are better than other hospitals along all dimensions are dominant, those worse on all dimensions are domintated, and those with mixed differences are considered non comparable.

A complicating factor is what is the magnitude needed for a hospital to be considered “better” along a given dimension.  Clearly, if Hospital A has 25% fewer readmissions than Hospital B, clearly Hospital A is the higher quality provider along this dimension.  However, if Hospital A has only 0.01% fewer readmissions than Hospital B, then in practice they are likely the same quality.

One way to determine if a hospital is dominated is simply do univariate statistical tests (e.g., t-test, Chi-squared) along each dimension and then only consider a hospital as dominant (or dominated) if the statistical test rejects the null hypothesis along all dimensions.  The novel approach proposed by Gutacker and Street is that they conduct “multivariate hypothesis tests of parameters of interest that take into account the correlation between dimensions and achieve correct coverage probabilities.”

The clear benefit of this approach is that hospitals considered dominant are clearly the best and those considered dominant are clearly the worst.  The drawback of this approach is that the majority of hospitals will be in the non-comparable group.  In the Gutacker and Street example of hospitals in England as long as the dimensions 5 of the 252 hospitals (2%) were considered dominant and 8 of the 252 (3%) were considered dominated at the 90% confidence level.  Thus, 95% of hospitals were non-comparable.  Even if we decrease the confidence level to 50%, 24 out of 252 (10%) were dominated and 30 out of 252 (12%) were dominated, meaning that 78% of hospitals were non-comparable.  This is better than the univariate approach that doesn’t take into measure correlations were by only 99% and 95% of hospitals were not comparable at the 90% and 50% confidence levels respectively.  Further, the approach assumes that all the quality dimensions of interest to all stakeholders are included in the analysis.  As the number of quality metrics increases, the likelihood a hospital is dominant (or dominated) will fall and the method becomes less informative.

Nevertheless, given the different stakeholder preferences, using a multidimensional performance assessment framework is a potentially appealing approach.

Source:

• Gutacker N, Street A. Multidimensional performance assessment of public sector organisations using dominance criteria. Health Economics. 2017;1–15. http://ift.tt/2xe1VKk

Identifying high quality providers in the presence of heterogeneous preferences posted first on http://ift.tt/2sNcj5z