Resetting the Scoreboard

Resetting the Scoreboard:

Why CBO Should Abandon Its Flawed Analysis of the
Center for Medicare and Medicaid Innovation

Executive Summary

Congress created the Center for Medicare and Medicaid Innovation (CMMI) in the Affordable Care Act and vested it with extraordinary powers.  CMMI can conduct demonstration projects in the Medicare, Medicaid and Children’s Health Insurance Program and expand those projects nationwide without congressional approval.

The Congressional Budget Office (CBO) believes that CMMI will achieve substantial federal savings.  It bases this conclusion not on analyses of projects that CMMI has undertaken, but on faith in the CMMI process.  CBO assumes that process will produce money-saving ideas and that the center will scrap failed projects and expand successful ones.

“The savings that CBO expects to result from the center’s activities,” a senior CBO official said in congressional testimony, “stem largely from the judgment that successful demonstrations will be expanded and achieve savings.”

The statement’s circularity – CBO “expects” CMMI to achieve savings because CMMI will “achieve savings” – is but one way which the agency’s analysis of CMMI departs from its long-established methods of preparing estimates.  In addition to assuming that CMMI will sometime in the future conceive, launch and nationalize successful projects, CBO conjured a numerical factor to convert its assumptions into dollar estimates.  It then embedded these numbers in its Medicare baseline, the yardstick against which it measures legislation. 

CBO’s unique approach to CMMI thus colors its analysis of legislation designed to achieve Medicare savings.  CBO believes that any bill that would overlap with any ongoing or possible future CMMI demonstration would increase Medicare spending above baseline levels.  Even if Congress offers up a proposal that would reduce spending relative to the statute, CBO will score it as a spending hike if it believes that CMMI might someday test a similar policy.

CBO thus ascribes unobserved and unobservable savings to projects that CMMI has not yet undertaken (and may never undertake), quantifies these savings through the application of an arbitrary numerical factor, incorporates the savings into its Medicare baseline, and measures the budgetary effects of legislation against this revised baseline. 

This paper traces the history of Medicare demonstration projects and shows how CMMI’s authorities differ from its predecessors.  It then examines CBO’s assumptions about CMMI, carefully tracing the reasoning that has led to its conclusions.  It then shows how recent events, including the Trump Administration’s cancellation of CMMI projects that CBO believed would save money, expose flaws in CBO’s assumptions and reasoning.  It concludes with recommendations for CBO, Congress and the executive branch with respect to CMMI.

Introduction

The Center for Medicare and Medicaid Innovation (CMMI) was created by the Affordable Care Act (ACA) to “test innovative payment and service delivery models to reduce program expenditures … while preserving or enhancing the quality of care furnished to individuals” enrolled in Medicare and Medicaid.[1]

Congress has invested CMMI with considerable resources and unprecedented authority to conduct demonstration projects.  The ACA allocated $10 billion to CMMI for fiscal years 2011 through 2019 and an additional $10 billion for the ten-year period beginning with FY 2020 and every 10 years thereafter in perpetuity.[2]  The statute authorizes CMMI to conduct demonstrations that increase federal spending, at least in their early stages.[3]  Participation in such demonstration projects need not be voluntary.  CMMI is further authorized to extend such demonstrations nationwide under certain criteria. In effect, Congress has authorized CMMI to amend the Medicare and Medicaid statutes without an Act of Congress.[4]  The statute also exempts most of CMMI’s actions from judicial and administrative review.[5]

The Congressional Budget Office (CBO) believes that CMMI will eventually achieve fairly substantial Medicare savings.[6]  CBO bases these estimates on what it calls “judgments” about the likelihood that the Secretary of Health and Human Services, will at some point in the future come up with money-saving ideas and apply those discoveries nationwide, while at the same time scrapping projects that don’t achieve savings.  Any congressional action that CBO believes might limit the Secretary’s flexibility with respect to demonstrations – including legislation that would codify demonstrations that CMMI has undertaken – would result in additional Medicare spending.[7]

This paper will compare CMMI’s demonstration authority with authorities the Secretary had prior to the ACA’s enactment. It will also examine CBO’s judgments that have led it to conclude that this new demonstration authority, unlike its predecessors, would produce substantial savings to the federal government.  Finally, this paper will point out flaws in the agency’s judgments and suggest ways in which it can arrive at better grounded estimates of its impact on Medicare spending.

Medicare Demonstration Authority Pre-CMMI

The Department of Health and Human Services (HHS) has long had legal authority to conduct Medicare demonstration projects.  Section 402 of the Social Security Act of 1967 authorizes the HHS Secretary (through the Centers for Medicare and Medicaid Services) “to develop and engage in experiments and demonstration projects.”[8]   The statute directs the agency to use this authority to “increas[e] the efficiency and economy of health services … without adversely affecting the quality of such services.”

In addition to being limited to changes in method of payment, such demonstration projects have typically been confined to certain geographic areas or subgroups of Medicare providers and beneficiaries.[9]  They also have generally been time-limited, commonly two years.[10]  The Office of Management and Budget (OMB) additionally requires that section 402 demonstrations be budget neutral, although it has made exceptions.[11]  OMB also requires that provider and beneficiary participation be voluntary.[12] The HHS Secretary had no authority to expand a demonstration nationally.  Doing so has generally required congressional legislation.

While the Centers for Medicare and Medicaid Services (CMS) has made extensive use of its section 402 authority, Medicare demonstration projects undertaken prior to the creation of CMMI were often congressionally mandated.  Congress, for example, created a hospice benefit that initially sunset after three years.[13] More recently, Congress directed CMS to conduct a demonstration project for durable medical equipment (DME) competitive bidding.[14] 

The competitive bidding program, though beset by continued controversy, provides a rare example of a demonstration project that has produced programmatic savings.[15]  For the most part, Medicare demonstration projects, whether mandated by Congress or initiated under the Secretary’s section 402 authority, have produced disappointing results. 

A 2012 CBO study of numerous major Medicare demonstration projects that were initiated prior to the creation of CMMI found that they failed to reduce Medicare spending:

In nearly every program involving disease management and care coordination, spending was either unchanged or increased relative to the spending that would have occurred in the absence of the program, when the fees paid to the participating organizations were considered….Results from demonstrations of value-based payment systems were mixed.  In one of the four demonstrations, [spending] … was reduced by about 10 percent…Other demonstrations of value-based payment appear to have produced little or no savings for Medicare.[16]

CBO attributed this failure to factors that extend beyond the demonstration projects themselves, assigning blame both to Medicare’s government-set fees and “the nation’s decentralized health care delivery system:”

Demonstrations aimed at reducing spending and increasing quality of care face significant challenges in overcoming the incentives inherent in Medicare’s fee-for-service payment system … and in the nation’s decentralized health care delivery system, which does not facilitate communication or coordination among providers.[17]

Nor were these demonstrations unique in their failure to reduce Medicare spending.  CBO’s Deputy Director Mark Hadley summarized the evidence from more than four decades of Medicare demonstrations, when he testified:

A small share of demonstrations resulted in savings, [but] most had little or no effect on Medicare spending, and some increased Medicare spending.[18]

How CMMI Demonstration Authority Is Different

CBO takes a different view of CMMI demonstration projects, projecting that they would achieve gross federal savings of $45 billion between 2017 and 2026.[19]  In reaching this conclusion, the agency points to key differences between CMMI and CMS’s longstanding demonstration authority.  Those differences are summarized in Table 1.

In authorizing the creation of CMMI, Congress invested the HHS Secretary with vast powers to test new Medicare ideas.  These new powers were designed to correct what were perceived to be flaws in the agency’s section 402 authority, flaws that were believed to have hampered the development and evaluation of changes that would reduce Medicare spending and preserve or improve health care quality.

The first difference is that participation in CMMI demonstrations can be mandatory both for providers and beneficiaries.  Giving providers and beneficiaries the option of participating in a demonstration project can cloud the evaluation process.  When participants self-select, it is possible, if not likely, that the outcomes of the demonstration cannot be readily generalized.

As CBO’s Mark Hadley put it:

When health care providers’ or beneficiaries’ participation in a demonstration is voluntary, and when they are able to determine how a demonstration might affect them before deciding whether or not to participate, analysts must determine whether those who opted to participate differ sharply enough from the others to make the two groups not comparable.[20]

Mandatory participation, which CMMI has used in many of its demonstrations, supposedly allows researchers to draw direct comparisons between those who participated in the demonstration and those who did not.  This, in turn, allows for making a more intelligent assessment of whether savings for a subset of Medicare beneficiaries would produce savings for the program if the demonstration were enlarged to include all beneficiaries.   

Mandatory participation alone, however, does not guarantee that a demonstration will yield generalizable results.  Some mandatory CMMI projects, for example, assign patients and providers to control and experimental groups based on geography, rather than on the characteristics of project participants.  Such geographical assignment creates methodological problems, whether or not participation is voluntary.

Ethical problems also can arise from mandatory participation.  If, for example, the Medicare statute requires CMS to pay administering physicians a certain amount for cancer drugs and the agency compels oncology practices in a particular geographic region to accept a lesser amount, this may affect the site – if not the quality – of care that a cancer patient receives.  The patient who is affected by these changes is given no opportunity to opt out of the demonstration and may not even be aware that he or she is part of a Medicare reimbursement experiment.  This is one reason why CMS in the past rarely compelled providers or patients to participate in a demonstration.

A second difference is that CMMI demonstrations are not congressionally mandated.  Prior to CMMI’s creation, a large number of demonstrations were undertaken at the behest of Congress.[21]  Although CBO is an arm of Congress, the agency believes that legislatively required demonstrations can and often have hampered the ability of the executive branch to test new approaches effectively.  CMS, for example, cannot modify such mandated demonstrations in ways that go beyond the statute and cannot terminate them if they prove unsuccessful.[22]  They are sometimes constrained to limit demonstrations by geographic area.  Most importantly, the agency cannot expand a successful demonstration nationally without Congress’s approval.[23]

A third difference is that CMMI demonstrations are largely exempt from administrative and judicial review.[24]  While this exemption is not plenary, it does extend to such things as selecting organizations, sites and participants in a demonstration and the authority to revise, expand or terminate a test.  Giving groups of patients or providers a process of challenging their participation in a demonstration project in court or before an administrative law judge could impose substantial barriers to the launch, alteration or expansion of demonstration projects.  Due process concerns aside, by stripping providers and patients of procedural remedies, the statute allows CMMI to more efficiently test, discard and operationalize innovative ideas.[25]

Fourth, the statute relieves CMMI of the obligation to ensure that its demonstration will not increase Medicare spending during its initial phase.[26]  The Office of Management and Budget has generally required that any demonstration project initiated by CMS be at least budget neutral.  Congressionally mandated demonstrations ordinarily carry a similar requirement.  This requirement may have prevented the agency from testing innovations that involve short-term costs, despite offering the prospect of longer term savings.

Finally, CMMI asserts the power to extend demonstration projects nationally without obtaining additional authorization from Congress.  Once so extended, such a demonstration can continue in perpetuity, which is tantamount to a de facto amendment of the Medicare statute.  This is an extraordinary prerogative for an executive branch agency and one that CBO believes will have a powerful budgetary effect.[27]  Indeed, the agency’s Deputy Director told Congress that CBO’s belief that CMMI will produce tens of billions of dollars in Medicare savings “stem[s] largely from the judgment that successful demonstrations will be expanded.”[28]

What CBO’s Estimate Is Not Based On

In estimating the cost of new legislation, CBO takes a generally concrete, fact-based approach.  If Congress, for example, proposed to further increase the tobacco tax, CBO (in conjunction with the staff of its sister agency, the Joint Committee on Taxation) would look at how the higher levy might affect baseline assumptions about smoking prevalence; health care expenditures; insurance premiums; labor force participation; workplace productivity; and behavioral effects among individuals, employers, state and local governments and health care providers.[29]  It also would attempt to estimate its effects on health and longevity, as part of its assessment of the proposal’s economic and fiscal impact. 

Similarly, if Congress considered a bill to change how Medicare reimbursed for rehabilitative services, CBO would seek to quantify how much more or less Medicare would pay for those services, whether the reimbursement changes would affect the volume and quality of those services, change their site of care or otherwise produce behavioral changes that might affect federal outlays.

Like all estimates, there would be considerable uncertainty.  Data is incomplete and sometimes conflicting, economic modeling is an imperfect science and human behavior is notoriously unpredictable.  But the estimates at least rest on data and quantifiable assumptions about specific legislation or executive actions.

CBO’s estimate of the fiscal effects of CMMI, by contrast, relies almost entirely on its faith in the process CMMI uses in testing innovations, without regard to specific costs or savings associated with specific demonstration projects.[30]  Before looking at that assumption in greater detail, it is important to see what CBO does not take into account when it forecasts that CMMI will reduce Medicare spending by $45 billion.  Table 2, appearing on the next page, summarizes those non-factors.

As the table illustrates, CBO’s method for estimating the budgetary impact of CMMI departs markedly from its accustomed methodology.  Ordinarily, CBO would be expected to examine specific CMMI demonstrations and estimate how much programmatic savings (and costs) they might achieve.  To arrive at such estimates, CBO might look, for example, at the results of similar demonstrations that the agency may previously have undertaken and decide whether the new projects would achieve better (or worse) results.[31] 

Instead, CMMI’s creation has inspired CBO’s flight from the quantifiable.  What is most striking about the quotes in Table 2 is the absence of anything that can be measured or even estimated: the success or failure of ongoing demonstrations, some sense of how, which or how many future projects might produce savings, and even the very prospect of observable savings.

What CBO’s Estimate Is Based On

The Four Judgments

CBO believes in the CMMI process.  That faith is encapsulated in a series of what CBO terms “judgments.”  It is a peculiar term for the agency to employ, but one it sprinkles liberally throughout its explanations of its CMMI estimates.  CBO’s Deputy Director used it 11 times in his prepared testimony for a September 2016 House Budget Committee hearing.  It invoked the word 22 times in 16 pages of responses to questions submitted for the record at that hearing.[32]

Ordinarily, CBO estimates rely on certain assumptions the agency makes in order to arrive at a cost estimate.  These assumptions, unlike its “judgments,” are quantifiable: for example, the effect on health insurance premiums of relaxing or tightening a particular insurance regulation,  the average per capita cost of covering childless adults under Medicaid, the number of seniors who will drop Medicare prescription drug coverage if premiums are increased, etc. 

These assumptions are the building blocks of CBO estimates.  They may be wildly wrong or pretty close to right (it is impossible to be perfect), they may be noncontroversial or they may incite criticism, they may be publicly available or closely held.  But CBO’s assumptions are generally quantifiable.

The judgments that underlie CBO’s estimate of CMMI savings are not.  Hadley laid out the key CBO judgments in his September 2016 congressional testimony:

CBO’s projections incorporate the following judgments:

• HHS will generally expand demonstrations that succeed, and the expansions will yield federal savings.
• Demonstrations that succeed will operate for four to seven years, on average, before HHS decides whether to expand them.  
• Demonstrations that do not succeed will operate for two to five years, on average, before HHS cancels them.
• The center will take some time to establish its procedures before achieving a steady state of testing, evaluating, and expanding demonstrations. Earlier sets of demonstrations may achieve slightly smaller savings than later sets, because the center is expected to learn more about conducting demonstrations.[33]

Of those four, Hadley identified the first as the most important.  “The savings that CBO expects to result from the center’s activities,” Hadley testified, “stem largely from the judgment that successful demonstrations will be expanded and achieve savings.”[34]

The most obvious point is the statement’s circularity: CBO “expects” CMMI to achieve savings because CMMI’s activities will “achieve savings.”  This circularity is essential to CBO’s judgment-based methodology: it proves only what it assumes at the outset to be true.

This circularity arises from the peculiar approach CBO has adopted toward its CMMI estimates.  As Joseph Antos of the American Enterprise Institute has noted:

CBO’s estimate is based on the assumption that the policies implemented in the demonstration will cut program costs or will be replaced by other policies that do.  CBO has declared that CMMI projects will be successful, even when they haven’t been imagined. [Emphasis in original.][35]

CBO’s conclusion that CMMI will produce programmatic savings is thus untethered from any analysis of discrete projects CMMI has undertaken and from any other quantifiable basis.  It instead rests on faith in the CMMI process itself.  That process – regardless of the particulars of the experiments it might conduct or decline to conduct – will achieve savings, CBO has declared.  It is a profession of faith, not the product of analysis.

The Last Judgment: Shielding CMMI from the Influence of Voters

CMMI’s faith in that process also involves a fifth and last judgment at which CBO hints, but never clearly articulates.  Specifically, CBO has incorporated into its baseline assumptions the judgment that CMMI will conduct its work beyond the reach of voters.  Neither presidential nor congressional elections will affect its activities or its decisions to terminate some projects and expand others.

The absence of such interference is a central, if unspoken, element of CBO’s approach.  Essentially, CBO envisions CMMI to consist of experts whose sole interest is to modify the Medicare program to achieve budgetary savings and enhance, or at least maintain, the quality of services beneficiaries receive.  CBO hypothesizes that these experts will conceive, execute and permanently extend successful demonstrations nationally, while identifying and terminating unsuccessful ones.  This can only happen, however, if these experts are free to operate undisturbed or untroubled by those elected to public office.  

CBO expresses this judgment in two indirect ways.  First, it assumes a certain continuity in CMMI’s work across administrations.

CBO’s projection of the budgetary effects of CMMI’s activities incorporates the judgment that future Administrations will continue to use the authority granted to CMMI and the Secretary of Health and Human Services under current law…CBO expects the types of demonstrations to change under new Administrations—perhaps dramatically—but it has no basis for assessing whether those different demonstrations would be more or less effective in reducing federal spending.[36]

This is a curious assertion, given CBO’s judgment that successful demonstrations will last four to seven years and will always be expanded.  Demonstrations of this duration are likely to be passed from one administration to the next, as in fact happened just months after CBO prepared the document from which this quote was drawn.  On the one hand, CBO expects potentially dramatic changes in the types of demonstrations a new administration might conduct.  On the other, it assumes that a new administration will continue demonstrations launched by its predecessor until it can determine whether they have succeeded.  CBO further assumes that a new administration will expand all successful demonstrations begun by a previous administration.  As we will see, this judgment also has proven unsound.

But what is especially striking is CBO’s judgment that Congress will not tamper with CMMI’s activities.  This is important because CBO assumes that any congressional intervention in a real or potential CMMI demonstration will reduce the savings that demonstration would otherwise have achieved.  Indeed doing so will make it more likely that an otherwise successful demonstration would fail, or at least be less successful, according to CBO.

Testifying before the House Budget Committee in September 2016, Mark Hadley said that any congressional action that would “limit HHS’s flexibility in designing and refining a demonstration” would “reduc[e] the likelihood that it would succeed and the magnitude of expected savings if it did succeed.”[37] 

CBO’s estimate of the savings CMMI will achieve thus rests on the assumption that Congress will in no way tamper with projects the agency undertakes.  CMMI is presumed to achieve optimal savings from its successful demonstrations.  Any congressional intervention in a CMMI demonstration will contaminate it, resulting in less savings than the agency’s pristine project would have achieved.

This applies not only to actual demonstrations but also to any project CMMI might theoretically undertake at some point in the future.  In a 2015 blog post, CBO commented on how it views potential interaction between legislation and projects that CMMI has or might one day initiate: 

CBO examines any legislative proposals that seek to enact approaches similar to ones that CMMI is testing, to determine whether HHS would do something different under the proposal from what it would do under current law…. To the extent that legislative proposals overlap with initiatives that CMMI is undertaking (or is expected to undertake), the potential additional savings would be reduced. [Emphasis added.] [38]

It is intriguing that CBO’s judgment, when applied to its analysis of budgetary effects of legislation bearing on CMMI, includes speculation about what CMMI “would do” or “is expected” to do.  These counterfactuals are somehow incorporated into the agency’s baseline assumptions about CMMI savings.  Any overlap between a proposed bill and a real or imagined CMMI demonstration could reduce those savings, thereby increasing Medicare spending above baseline assumptions. 

In sum, CBO believes that any bill that would overlap with any ongoing or possible future demonstration would increase Medicare spending above baseline levels.  Even if Congress offers up a proposal that would reduce spending relative to the statute, CBO will score it as a spending hike if it believes that CMMI might test a similar policy at some point in the future.

CBO has even found that bills to expedite expansion of existing CMMI projects would increase Medicare costs.  In 2017, for example, the Senate Finance Committee reported a bill whose stated purpose was to improve chronic care (CHRONIC Act).[39]  One of its provisions dealt with a CMMI demonstration that allows Medicare Advantage (MA) plans to vary cost-sharing and benefits for Medicare beneficiaries with certain chronic conditions in order to encourage them to use the services that are of the highest clinical value to them.[40] 

The CMMI’s Value-Based Insurance Design (VBID) model would exempt participating MA plans from the requirement that they provide identical benefits and cost-sharing to all enrollees.[41]  Participating plans can instead reduce or eliminate cost-sharing for patients with certain medical conditions.  An MA plan involved in the demonstration could, for example, provide that patients with diabetes get free eye exams or that patients with heart disease can get anti-hypertensive drugs without any copayments at the pharmacy counter.[42]

CMMI launched that project in seven states in January 2017, with the intention of including MA plans in three additional states in 2018.[43]  The bill would allow MA plans in any state to participate in the demonstration project in 2020.  It also would prohibit CMMI from terminating the project before January 2022.  After that date, CMMI could continue the demonstration only if it neither reduced health care quality nor increased Medicare spending.

CBO concluded that the bill’s VBID provision would increase spending by a total of $90 million 2020 and 2021.[44]  CBO attributes the additional spending – or forfeited savings (CBO is not clear on this point) – to the fact that the bill “would limit the Secretary’s flexibility to design and modify the demonstration.” 

The bill, however, allows the Secretary to modify the VBID demonstration.  HHS could, for example, add or subtract from the list of chronic diseases for which a plan may vary benefits and cost-sharing.  The bill only requires that CMMI allow MA plans in all 50 states be made eligible to participate in the demonstration project beginning in 2020, something the Secretary might do in any event.  In CBO’s judgment, however, such an expansion only saves money if CMMI acts on its own initiative, rather than in response to an Act of Congress.[45]

CBO’s Magic Asterisk

CBO’s score of the CHRONIC Act raises the question of how it produces dollar savings out of non-quantifiable judgments.  Why does CBO estimate, for example, that congressional expansion of a CMMI demonstration will cost Medicare $90 million and not $9 million or $9 billion?

The answer is that CBO has conjured a numerical factor to convert its non-quantifiable judgments into dollar estimates and then incorporated that numerical value into its Medicare baseline:

CBO estimates that the total effect of a set of demonstrations started in a given year will be an eventual reduction of about 0.1 percent each year, on average….For example, the set of demonstrations begun [during fiscal year 2016] are expected to reduce spending by a total of about $1 billion in 2026.[46]

Those savings are cumulative.  In every year, new projects undertaken are expected to yield eventual savings equal to 0.1 percent of Medicare spending.  As of September 2016, CBO was projecting that CMMI savings in fiscal year 2017 would equal $1 billion.  In 2026, cumulative savings will amount to $8 billion (since new successful projects, in CBO’s judgment, will be launched every year).[47]

The arbitrariness of this assumption is obvious (and inexcusable).  Because CMMI’s savings are set at a fixed percentage of Medicare outlays, the more Medicare spends, the more CMMI is presumed to save.  Since Medicare spending rises annually, CMMI’s savings are projected automatically to rise along with it – and at precisely the same rate.

Nor can CBO verify that the government realized these savings, even after the fact.  CBO, based on its math, forecast that CMMI would reduce Medicare spending by $1 billion in fiscal year 2017 (0.1 percent times Medicare Part A/ Part B spending in that year).[48]  That fiscal year has come and gone and we do not know whether those savings actually materialized.  Indeed, CBO tells us that question cannot be answered.   “Unlike the center’s spending,” Mark Hadley testified, “the reduction in spending on Medicare benefits will never be able to be observed.”[49]

CBO thus ascribes unobserved and unobservable savings to projects that CMMI has not yet conceived, quantifies these savings through the application of an arbitrary numerical factor, incorporates the savings into its Medicare baseline, and measures the budgetary effects of legislation against this revised baseline.  The methodology is reminiscent of something government budget analysts call a “magic asterisk” – a placeholder estimate of federal savings untethered from specific policy proposals.  These devices allow their architects to claim reductions in government spending without explaining how those savings are achieved and at whose expense.  Traditionally, CBO has policed the magic asterisk, forcing lawmakers to spell out the specific policies by which they would reduce federal spending.  In the case of CMMI, the agency itself is propagating a magic asterisk.

Case Study: Physician-Administered Drug Demonstration

In March 2016, CMMI announced that it would undertake a massive, five-year demonstration project designed to test new ways of reimbursing medical practices for the costs of physician-administered drugs.[50]  In December, the Obama Administration announced that it would not issue a final rule that would have launched the project, leaving that decision to its successor.[51] 

In an October 2017 Federal Register announcement, the Trump Administration formally extinguished the proposed demonstration.[52] CMS noted that the proposal had drawn more than 1,300 public comments, many of which “expressed concerns about the proposed model.”  Citing what it called “the complexity of the issues related to the proposed model design,” the Administration officially withdrew the proposal that its predecessor had tabled a little more than a year earlier.

The project’s failure to emerge from the larval stage exposes further flaws in CBO’s most critical judgments about CMMI.  It is thus worth discussing in some detail.

While Medicare Part D covers most drugs available at pharmacies, Medicare Part B pays for drugs furnished incident to a physician’s services that are not usually self-administered by patients.[53]  These include injected and infused drugs such as those used to treat cancer and other medical conditions.  Medicare Part B pays both for the administration of the drug (generally at a physician practice or outpatient hospital facility) and for the drug itself.  The Medicare Modernization Act established a reimbursement rate of 6 percent above the drug’s average sales price (ASP+6).[54]

The most recent Medicare Trustees Report (published in 2017, citing 2014 data) estimated that Part B spending on drugs comprised 6.7 percent of total Part B benefit spending, a share that has risen in recent years.[55]The Medicare Payment Advisory Commission (MedPAC) calculates that Medicare Part B prescription drug spending grew by an average of 9 percent annually between 2009 and 2013.[56]  Some attribute this increase, at least in part, to the program’s ASP+6 reimbursement methodology. Because Medicare reimbursement rises with a drug’s price, the 6 percent markup is larger for more expensive medicines. This, MedPAC has argued, creates an incentive for doctors to order up pricier drugs, but other factors such as development costs of new specialty drugs may also be at work.[57]  In any case, there is considerable evidence to support fiscal benefits from drug spending.[58]

CMMI published a proposed rule in March 2016 that set out to test new ways of reimbursing for Part B drugs.  Table 3 summarizes CMMI’s proposed demonstration. 

The agency would divide the country into four geographic areas.[60] During Phase I, which was to have been launched in the fall of 2016, Medicare would reimburse one group of providers at the ASP+6 rate.  These practices would serve as the control group.  Medicare would reimburse the second group at ASP+2.5 percent plus a flat fee of $16.80. 

Phase II was scheduled to begin no sooner than January 2017.  In this part, the two Phase I groups would be subdivided.  The control group would be split in two, with one continuing to be reimbursed at the ASP+6 rate and the other being reimbursed at ASP+6, while also using “value-based purchasing tools” employed by pharmacy benefit managers to control costs (e.g., varying reimbursement for a drug based on its clinical effectiveness for different indications, reimbursing based on reference pricing, outcomes-based risk-sharing arrangements with drug manufacturers).  The Phase I experimental group also would be subdivided, with some being reimbursed at ASP+2.5+$16.80 and the rest at the same rate, but also employing “value-based purchasing” methodologies.

Reaction to the proposal was swift. A number of medical societies, patient groups and drug manufacturers lined up against the proposal.[61]  The AARP and certain consumer groups joined with the insurance industry in voicing support for the demonstration.[62] They noted that some Medicare beneficiaries (primarily a small percentage of those without supplemental coverage) are required to pay 20 percent of the costs of Medicare Part B services.  Any reduction in Medicare reimbursement for physician-administered drugs thus would ease upfront cost-sharing burdens on non-covered seniors who consumed those drugs.

Most Members of Congress rapidly joined the ranks of critics.  More than 240 Members signed a letter to the then Acting Administrator of CMS, asking him to withdraw the proposed demonstration.[63]  Nor was congressional opposition limited to Republicans.  Democrats, too, circulated their own letter in opposition to the project.[64] Legislation was introduced to terminate the project.[65]

The CBO analysis of that bill afforded the agency the opportunity to opine on the Medicare savings that might accrue from a specific CMMI initiative.[66]  First, CBO indicated that both phases of the demonstration would save the federal government money, totaling nearly $2.3 billion over 10 years.[67]  But in keeping with its normal protocols regarding the budgetary effects of proposed regulations, it reduced that savings by half, to $1.15 billion.[68]  CBO then expressed the view that if Congress were to quash the demonstration, CMMI “would eventually ‘backfill’ $750 million of the lost savings by carrying out other demonstration projects.”[69]  No justification was given for this assertion. CBO calculated the net cost of the bill at $395 million ($1.145 billion - $750 million).

CBO’s analysis of the legislation once again betrays its unusual approach toward CMMI.  It is hard to imagine another case in which CBO would assume that an agency would find hundreds of millions of dollars in savings to “backfill” most of the new spending that a bill would produce.  If CMMI were able to save Medicare $750 million, why would it only do so if Congress pulled the plug on the Part B drug demonstration? 

The more consequential lesson, however, may be that the demonstration project never got off the drawing board.  The program’s failure to launch contravenes CBO’s most important judgments regarding CMMI.  First, the episode showed that CMMI is not immune to political pressure, particularly from groups representing patients and providers.  Second, although it did not enact legislation, Congress used other tools at its disposal to dissuade the agency from proceeding with an unpopular initiative.  CBO’s presumed firewall between Congress and the agency did not withstand the political heat.  Third, it demonstrated that CBO’s judgment that successful demonstrations would last four to seven years is mistaken.  New leadership at the agency may kill one of the most cherished projects of its predecessor. 

Finally, CBO believes that the project would have been successful, achieving fairly substantial Medicare savings.  On a policy level, this was not assured, given that CBO believed savings would occur immediately through utilization changes, and did not account for shifts in care to more expensive settings. Nonetheless, CBO, as we have seen, assumes the Secretary will always expand successful projects.  Two successive HHS Secretaries did otherwise: the first delayed the project’s launch; the second scuttled it. 

CBO’s most critical judgment about CMMI – that it will expand money-saving projects nationwide – was proven wrong with respect to a project that CBO itself believed would save money.

The Trump Administration’s New Direction for CMMI

In addition to cancelling the Part B demonstration conceived by its predecessor, the Trump Administration is changing the focus and direction of CMMI.  This, too, undercuts CBO’s judgments about the agency’s capacity to reduce Medicare spending, which require the continuity of successful projects across administrations.

CMS Administrator Seema Verma signaled the change in a September 2017 Wall Street Journal op-ed.[70] “This administration plans to lead the Innovation Center in a new direction,” she wrote.  “We will move away from the assumption that Washington can engineer a more efficient health-care system from afar—that we should specify the processes health-care providers are required to follow.”

Verma’s shot at Washington engineers was clearly aimed at the approach CMMI had taken during the Obama Administration.  Many of the agency’s largest and most far-reaching experiments involved requiring providers to follow certain processes in order for the agency to measure the quality and efficiency of medical care.  Verma’s op-ed repudiated that approach.

It also questioned the complexity and scope of some of CMMI’s largest demonstrations.  Projects that rewarded large institutions for hitting certain metrics (and that sometimes penalized them for falling short) “have encouraged consolidation in the health-care system,” she wrote.  The previous administration had favorably regarded such consolidation, since large provider systems were presumed to be both efficient and sufficiently capitalized to go at risk for the medical care of whole populations.

Verma called for a Copernican change in CMMI’s methodology.  Instead of focusing on demonstration projects in which the government shopped for value, she announced that CMMI would instead begin to focus on patient choice:

Consumers are a critical part of the health-care equation. We need to empower patients with information to seek value and quality as they shop for services. They also need incentives to be cost-conscious. Patients can define value better than the federal government can.

This last declaration – that patients can define value better than the federal government can – is a sharp departure from the approach that CMMI previously took. 

The genealogical line of “value based purchasing” demonstrations, for example, proceeds from the assumption that government could best define, identify and shop for value.[71] This focus on government over consumers was reflected in the design and evaluation of many, if not most, CMMI demonstrations. 

As of December 2016, for example, CMMI estimated that 18 million Medicare beneficiaries and individuals with private insurance, along with 207,000 health care providers, were participating in CMMI service delivery models and initiatives.[72]  Some of those beneficiaries were no doubt participating in multiple experiments, several of which may have overlapped.  It is highly unlikely that very many of them understood the implications of any of these innovative arrangements or how those arrangements might affect their care – assuming they knew or appreciated that they were part of an experiment.  Their participation in many of the projects was, in any event, mandatory.

Verma’s call for patients to play a more active role in determining the value of their medical care was a major component of the CMMI’s request for information (RFI) released shortly after publication of her WSJ piece.[73]  That publication outlined CMMI’s new “guiding principles,” which included choice and competition in the market, provider choice and incentives, benefit design and price transparency, and small scale testing.

In addition, the RFI announced that CMMI would put a new emphasis on what it called “patient-centered care,” which it defined as:

Empower[ing] beneficiaries, their families, and caregivers to take ownership of their health and ensure that they have the flexibility and information to make choices as they seek care across the care continuum.

Past CMMI projects very often involved contracts between the federal government and large providers to follow certain procedures and share risk based on outcomes.  The patient role (whether intentionally or not) was subordinated to that of payers and health systems – accountable care organizations, for example, would receive bonus payments if they adhered to certain protocols and reduced costs.

CMMI now seems headed in a very different direction, requiring more price transparency and quality information to consumers, who will make their own decisions about value.

A second important way in which the new initiative differs from the way CMMI has heretofore operated, is that it now will “focus on voluntary models.”  As discussed above, CBO’s judgments about CMMI’s budgetary effects has to do with the fact that it has generally required providers and patients to participate in its demonstrations.  CBO believes that mandatory demonstrations are more “powerful” and that voluntary projects often fail to yield generalizable results.[74]

This shift in focus already has led to the cancellation and scaling back of several CMMI projects undertaken by the Obama Administration.  In August 2017, the agency proposed cancelling four such projects; a final rule was issued in December.[75]  The agency based none of these cancellations on a conclusion that the programs would be unsuccessful.  On the contrary, the programs, according to HHS, would “improve quality and care coordination across the inpatient and post-acute care spectrum while lowering spending.”[76]  The projects, in the judgment of CMMI, would have met the statutory definition of success.  Nevertheless, the agency canceled them.

This is a significant departure from the way CBO assumes CMMI will operate. In addition to extending successful programs to the entire nation, CBO presumes that the agency will identify unsuccessful projects in a relatively short period of time and cancel them.  Between August 2017 and October 2017, CMMI proposed cancelling four projects that it believed would “improve quality” and “lower[] spending” and formally buried a fifth (the Part B drug demonstration) that CBO itself projected would produce significant Medicare savings. 

In addition to the cancellations, CMMI also proposed to modify the Comprehensive Care for Joint Replacement (CCJR) Model to make participation voluntary in approximately half the geographic areas in which the demonstration operates.[77]

This change also challenges one of CBO’s most important judgments.  As we have seen, CBO places great weight on the fact that CMMI projects can compel providers and recipients to participate in its demonstration programs.  Because CMMI can conduct mandatory demonstrations, CBO believes it is more likely to discover innovative models that will save Medicare money.

The HHS Federal Register announcement was less bullish on mandatory demonstrations:

We are concerned that engaging in large mandatory episode payment model efforts at this time may impede our ability to engage providers, such as hospitals, in future voluntary efforts.

This decision is not surprising, in light of CMMI’s new “guiding principles,” which announced a new “focus on voluntary models.”  It does, however, run directly contrary to one of CBO’s most important judgments that underpin its confidence in the ability of CMMI to generate tens of billions in Medicare savings.

CMMI thus is not merely moving to cancel projects, but is also beginning to express a preference for voluntary demonstrations over mandatory ones.  Without suggesting that this new direction is better or worse than the previous one, it is decidedly different and different in a way that should cause CBO to rethink its premise that CMMI’s projects and priorities will remain consistent from one administration to the next. 

Conclusion

CBO’s forecast that CMMI would reduce Medicare spending by $45 billion over ten years is flawed.  The agency has relied on nonquantifiable judgments and a “magic asterisk” (savings = 0.1 percent of Medicare spending) to produce its estimates. 

Those judgments are in some cases questionable, in others mistaken and in still others rendered obsolete by the new course on which CMMI has embarked.  Its most important judgment – that CMMI will expand demonstrations that prove successful – has proven wrong in at least one case:  CBO concluded that the Medicare Part B drug demonstration would yield savings; CMMI nevertheless killed it.  Since then, CMMI has proposed to cancel four other projects, despite the agency’s view that they would “improve quality” and “lower[] spending,” the two statutory metrics of success.  Those agency actions strike at the heart of the CBO judgment that CMMI would let successful projects continue for four to seven years before expanding them.

These cancellations also undercut CBO’s unspoken assumption, incorporated in its baseline estimates of CMMI savings, that the agency is a laboratory of experts quarantined from the political process. 

Whether it is constitutional – or even desirable – for an executive branch agency to make substantial revisions to federal entitlement programs unfettered by elected officials, it does not seem plausible.  The structure of American government gives elected officials authority to intervene in the ventures of unelected public servants and they will use that authority.  To assume, as CBO seems to, that CMMI’s decisions will not be influenced by elections and that the agency will operate without interference from elected representatives, presidents and representatives of patients and providers is untenable. 

Recommendations

• Congress should revisit CMMI’s structure and authority.  CMMI was created as part of the ACA, a broad and sweeping law of which the agency’s creation was a relatively small part.  There was little time for Congress to undertake a detailed examination of the law or its broader implications for constitutional governance.  Now that CMMI has been up and running, it is appropriate for committees of jurisdiction to look more closely at the agency.  In particular, Congress should examine whether it is appropriate for CMMI to compel patients and providers to take part in demonstration projects and whether it makes constitutional – or policy – sense to vest power in an agency to effectively amend the Medicare statute.  Congress also should follow up on the 2016 House Budget Committee hearings that examined CBO’s assumptions about CMMI’s fiscal effects.


• CBO should rethink its assumptions about CMMI.  CMMI’s “new direction” offers CBO the opportunity to embark on a new direction of its own. The agency’s cancellation of projects that would have continued had CBO’s judgments borne out, its focus on patient-centered models, and its preference for voluntary demonstrations all give CBO a reason to take a fresh look at its assumptions.  CBO also must account for the fact that the agency changed course with the change of administration, scuttling projects seemingly without regard to their prospects for success. 

  A new estimate of CMMI’s budgetary effects should rest less on faith in the CMMI process and more on empirical evidence.  CMMI’s costs are quantitative and real; the savings that may result from ongoing and future demonstrations are purely speculative. 

  In estimating such savings, CBO would do well to follow its established canons.  As Joseph Antos has recommended, it should base its savings estimates only on final rules, it should not ascribe savings to projects yet unannounced and it should not assume that CMMI will expand a project nationwide until the agency does so.[78]

  Such methods may seem too prosaic for an agency as unique as the CMMI.  But as experience has demonstrated, it is dangerous to assume that real savings will result from theoretical projects.  A clear-eyed and sober analysis of CMMI would bolster CBO’s indispensable role as arbiter of budgetary matters.

• HHS should undertake a rigorous review of whether it may require patients and providers to participate in demonstration projects.  CBO and others have argued that mandatory participation yields demonstration results that have more scientific value than those in which participation is voluntary.  That may be, although it is no guarantor of validity.  As discussed earlier in the paper, the particular characteristics of participants is a key factor and some of CMMI’s mandatory demonstrations assign participants to control and experimental groups based on geography, not on their characteristics.  In any event, mandatory participation raises ethical and legal concerns.  Ethical guidelines generally require that individuals who are involved in experiments be informed of this fact and given the option not to participate.  Legally, there are concerns that two Medicare beneficiaries be provided potentially different levels of medical services based merely on their ZIP codes.   The Administration has expressed a preference for voluntary over mandatory participation, but it should look more seriously at the question of whether and in what circumstances it is appropriate to compel patients and providers to take part in demonstration projects.


• OMB should conduct its own assessment of CMMI’s budgetary impact.  While there has been a fairly robust public discussion of the assumptions underlying CBO’s analysis of CMMI, there has been little discussion of OMB’s views.  OMB, in consultation with various federal departments and agencies, maintains its own baseline spending assumptions.  An OMB analysis of CMMI, whether prepared in response to congressional inquiry or at the direction of the President, could improve Congress’s understanding of CMMI’s potential for achieving Medicare savings.

Given the fiscal challenges faced by federal entitlement programs in general – and those affected by CMMI demonstrations in particular – it is vital that policymakers have realistic, analytically-grounded estimates, assumptions, and projections on which to base future decisions. It is time to evaluate how CBO has constructed the “scoreboard” for CMMI, so as to ensure that meaningful statistics are informing the true state of play.

---

About the Author

Doug Badger is a former White House and U.S. Senate policy adviser.  His career includes a stint at the President’s National Economic Council, where he developed a Medicare prescription drug proposal for President George W. Bush.  He represented the White House in negotiations with Congress that resulted in enactment of the Medicare Modernization Act, a law that established health savings accounts, the Medicare Advantage program and Part D of the Medicare program.

About the Taxpayers’ Budget Office

National Taxpayers Union Foundation publicly launched the Taxpayers’ Budget Office (TBO) in early 2017 to serve as a watchdog for Congressional Budget Office (CBO) processes, scoring, and transparency. Likewise, our project provides essential cost estimates and budgetary analysis for legislative proposals. Much in the way the Shadow Open Market Committee has provided with the Federal Reserve, TBO will provide a measure of balance, oversight, and thoughtful evaluation to CBO’s work. Through constructive recommendations, TBO will elevate and augment CBO’s mission so as to improve the quality of information available to policymakers and the public.