What the IRS's New Enforcement Budget Means for Taxpayers

IRS Workforce Levels

The table below shows the size of the IRS workforce since 1962. The data through 1986 is based on average annual employment levels reported in the 1988 Data Book. Data since 1987 reflect the total number of FTEs listed in each of the annual editions of the IRS Data Book. 

IRS employment peaked in 1991 with over 138,000 employees, which included over 29,000 seasonal employees brought on to assist with processing returns. Workforce levels since then have generally decreased in line with the rise of electronic filing and increased automation of IRS administrative processes. 

Unfortunately, the Treasury Inspector General for Tax Administration (TIGTA) found that the IRS could do a better job of automating tasks and maximizing efficiencies. Still, with technological progress and agency reform, the IRS should be able to provide more services with fewer taxpayer dollars.

Modernization of IRS systems and technology has always been a part of this equation, especially as it relates to labor costs. To give just one example, the 1997 report of the National Commission on Restructuring the IRS, which later became the basis for a landmark reform law, explicitly delineated several advantages of encouraging greater e-filing of returns:

“In addition to labor, overhead, and management costs to operate the returns processing pipeline, there are other cost reductions that could be realized with an increase in electronic filing, such as the following: 

• Heavy dependence on manual labor creates additional recruitment and training costs for IRS since it causes spike demands for low-cost labor that is not always available.

• Facility and physical handling equipment associated with paper filing is much higher than that used for electronic returns.

• After the data is manually entered into the database, the paper return is archived. Storage costs for paper returns are higher than those for electronic returns. Any subsequent need to access a return, such as an examination or collection, or requests by taxpayers for a copy of the return, results in retrieval costs as well.”

NTUF laid out a package of reforms needed at the IRS to improve its operations and become more efficient, including reducing barriers to electronic filing, bringing in private sector experts by reconstituting and revitalizing the IRS Oversight Board, and implementing Government Accountability Office recommendations for the IRS to more effectively make use of its existing funding. In addition, simplifying the tax code not only reduces taxpayers’ compliance burden, but it also makes it easier for the IRS to administer with a smaller workforce.

Will the New Employees Work on Enforcement or Taxpayer Services?

Fact-checkers have also pushed back on the 87,000 new IRS “agents” data point because some of the new hires would also work on taxpayer services, which have been languishing at the agency. While it is true that some resources will be devoted to improving services provided to taxpayers, it represents a small portion of the $80 billion. After all, the administration’s plan was called the “Tax Compliance Agenda” and not the “Taxpayers Assistance Agenda.”

It is quite true that many techniques to improve tax compliance need not involve direct enforcement such as audits or criminal investigations. Returning once again to the IRS Restructuring Commission’s 1997 report, Commissioner Ernest Dronenburg, a former leader with California’s State Board of Equalization, observed that “a .5 percent increase in voluntary compliance resulting from taxpayer education and changing attitudes would increase revenue in my state by over $400 million annually. Conversely, doubling our current audit coverage from 3% to 6% would produce less than half that amount.”

Nonetheless, Treasury’s chart indicates that the additional employees would effectively yield higher tax revenues through the combination of additional direct revenue from taxpayers who are not currently in compliance with tax law, and through “revenue protected” program integrity efforts such as reducing improperly claimed tax credits. Given the return on investments figure calculated by the administration, taxpayers could reasonably conclude that the bulk of these new IRS employees would be involved in enforcement rather than taxpayer assistance. 

The text of the document also heavily stresses compliance. Below is a chart showing the frequency of use of certain words and phrases in the document, illustrating the focus on compliance over services. “Compliance” was used 120 times throughout the document (including the title of the report repeated 23 times). Meanwhile, the word “assistance” was used just twice. The word “services” appears eight times in the agenda, including two appearances in the phrase “taxpayer services.”

The focus on compliance is also clearly evident in the distribution of the IRA’s $80 billion in funding, illustrated in the chart below. Over half of the amount goes to enforcement while just four percent of the amount goes to taxpayer services. The amounts under “other” include $400 million for the Treasury Inspector General for Tax Administration, $153 million to the U.S. Tax Court, $105 million for Treasury’s Office of Tax Policy to promulgate regulations, and $50 million to Treasury for IRS oversight.

Increased Enforcement Will Not Just Impact “Tax Cheats”

President Biden and Treasury Secretary Yellen had promised that no one earning less than $400,000 per year would be impacted by the new enforcement measures, and that new audits will only target a small class of wealthy taxpayers who are not paying their “fair share.”

The number of millionaires being audited has dropped off significantly since 2015, when nearly 40,000 millionaire returns were reviewed. In 2020, 11,000 millionaires were audited, and the figure rose to nearly 14,000 in 2021. This number will continue to rise with the new emphasis on enforcement, but additional taxpayers further down the income ladder will likely be swept up in the IRS’s net. In the wake of the IRA’s enactment, Republicans and other critics of the IRS expansion warned that new revenue agents will inevitably target the middle class.

Treasury’s Sarin told Reuters in response, “The speed and voracity with which (Republicans) are coming at this is really a testament to how important these resources are going to be - because there are many wealthy tax evaders that stand to lose a lot.” A college professor and columnist separately argued that taxpayers only need to worry about a supercharged IRS if they are “tax cheats.” And on NBC’s Meet the Press, Chuck Todd said, to the delight of his fellow panelists, “ … you know my answer to people afraid of more IRS agents is well, you know, then don’t cheat on your taxes.” This type of argument ignores the disproportionate impact that IRS enforcement activities can have on low- and middle-income taxpayers, as well as small business owners, who don’t have the resources to challenge enforcement claims. These taxpayers may also ultimately be innocent of what the IRS is charging, or may have made an honest mistake on their tax returns.

In reality, many additional taxpayers will likely be affected by stepped-up enforcement efforts. The IRS is expected to focus more scrutiny on the self-employed, whether as sole proprietors, limited liability corporations, or S-corps. Moreover, the IRS collects mountains of data from employers, contractors, and financial institutions and feeds them into the IRS databases. Any discrepancy among the data automatically generates tens of millions of notices annually that are mailed out to taxpayers. The more data the IRS collects, the more opportunities it will find to claim that taxpayers are underpaying.

It costs money and time for the IRS to conduct audits, and when the IRS determinations are challenged, the court cases can drag on for years. Targeting complicated tax arrangements of wealthy individuals and businesses can be more difficult because they have resources to obtain the best tax attorneys. For households making $75,000 or less, the IRS makes $10 for every $1 it spends in the collection process, and 50 percent of IRS audits fall on these households. Low-income earners are audited at five times the rate of all other taxpayers, due in part to the high record of improper payments for wrongly or incorrectly claimed refundable tax credits. Also, they are generally easier tax overpayments to reclaim because the taxpayers are less likely to contest the IRS.

One reason why people suspect that the IRS tax enforcement will focus on the middle class is that the Biden administration initially included a proposal to give the IRS access to nearly everyone’s financial transactions. The plan was to require banks and other financial institutions to report all transactions from any account with at least $600 in annual activity to the IRS. Because of its far-reaching implications on nearly all taxpayers, the proposed threshold was raised to $10,000, but eventually the idea was dropped. Separately, the American Rescue Plan Act lowered the 1099-K reporting threshold from $20,000 and 200 financial transactions to just $600 with no limit on the number of transactions. This was projected by CBO to increase tax receipts by $7.7 billion over ten years, but the substantially lower threshold is set to cause significant headaches for taxpayers who use payment services like Venmo or use eBay as an ongoing online garage sale.

Some households will, in theory, be protected by Treasury Secretary Yellen’s directive that the additional resources provided in IRA “shall not be used to increase the share of small business or households below the $400,000 threshold that are audited relative to historical levels.”

The order does raise some confusion for several reasons. For one, Yellen did not indicate what “historical levels” would be the basis for further audits. For example, in 2011, just prior to a multi-year drop in IRS funding, the examination coverage rate of individual returns reporting $75,000-$100,000 of positive income was five times greater than the 0.1 percent coverage rate reported in 2019. 

Other questions include:

• Who gets to define $400,000? Does that amount represent Adjusted Gross Income, Modified Adjusted Gross Income, or Taxable Income? Is it what a taxpayer reports on their return for the year the IRS wishes to conduct an examination, or is it what the IRS believes the taxpayer’s taxable income ought to be after proposed adjustments? If the latter, greater numbers of Americans than advertised will be subject to scrutiny, perhaps to no avail. 

• Does this directive create a “marriage penalty,” in the sense that the IRS could investigate a single taxpayer’s return reporting above $400,000, but could also look into a joint return where each taxpayer earns $200,000? Dual-earner households in large cities – many of which are located in blue states – could be surprised to find themselves under the microscope.

• What year will govern the audit decision – the year of the return under examination, or the taxpayer’s present circumstances? A business owner who earned $500,000 during the good times of 2019 might be shocked to get an audit notice next year if she finds herself struggling with half that cash flow today. In fact, high-income tax returns are also among the most volatile in reporting year over year incomes. 

• What happens if the directive is disobeyed or unintentionally violated? The Secretary’s ability to impound funds is highly limited. And if “accidents” happen, how can a taxpayer possibly be made whole? A Treasury directive does not carry the force of law in court, and regardless, no outside entity will be monitoring IRS audit rates in real time. 

Do these concerns amount to parsing words? Perhaps, but the operation of the entire tax system often amounts to this very exercise, as Congress proves with myriad technical corrections to various tax laws. For its part, the IRS has frequently denied taxpayer appeals, pursued exotic enforcement and litigation strategies, issued absurd rules, and reversed its own mind, based on interpretations (and contortions) of a few words. In this environment, taxpayers are entirely justified in wondering how exactly Secretary Yellen’s directive will translate to IRS personnel on the ground. 

Beyond these considerations, Secretary Yellen is asking the IRS to do something that is neither practical nor transparent. Money is fungible – nothing the IRS receives in annual appropriations, over and above the IRA funds, would be subject to the $400,000 taxpayer income limitation. The Service could also simply request from Congress, or to a more limited degree, redirect, more funds from what it would otherwise normally spend on high-income enforcement activities toward those lower on the scale. Furthermore, neither Secretary Yellen nor Congress demanded that any kind of public accounting be kept on how enforcement dollars will be used against certain types and classes of taxpayers. Additional monies in the IRA to the Treasury Inspector General for Tax Administration could be employed for such purposes, but here again, the legislation is silent on where TIGTA is supposed to look.

For definitional reasons above and others these vague protections would have been better implemented through the force of law, but Senate Finance Committee Chair Mike Crapo’s (R-ID) attempt to amend the IRA to prevent the IRS from using any of the funds to audit taxpayers with taxable incomes below $400,000 was defeated on a party line vote. However, Secretary Yellen’s directive did confirm that Biden’s pledge on increased tax enforcement against the middle class would have been broken. CBO revised its revenue estimate from tax enforcement downward by $20 billion, taking the official directive into account.

Over the decades, there have been far too many examples of poor treatment of taxpayers by heavy-handed IRS actions. Among the most egregious include the case of Tom Treadway, who lost his business after a bogus $247,000 tax assessment that was later thrown out on appeal ( the IRS also raided his girlfriend’s bank account), and Alex Council, who lost his business and took his own life after a badly handled IRS audit that was eventually overturned. Reforms and strong oversight will be necessary to guarantee that taxpayers are protected as the IRS cranks up tax enforcement.