Is IRS Tax Enforcement Colorblind?

Professor Jeremy Bearer-Friend of the George Washington University Law School highlights key areas of the IRS’s tax enforcement where racial bias can occur.

This transcript has been edited for length and clarity.

David D. Stewart: Welcome to the podcast. I’m David Stewart, editor in chief of Tax Notes Today International. This week: colorblind tax enforcement.

At many times in American life, we’re asked to volunteer information on our racial or ethnic background, but one place where these questions aren’t raised is on tax returns.

IRS commissioners from both political parties have used this fact to support the claim that the IRS is not racially biased in its tax enforcement. They don’t have the data, so any enforcement actions are separate from a taxpayer’s race or ethnicity.

However, some have pointed to patterns in how the IRS does its work and believe there are opportunities where racial bias can occur within the IRS’s tax enforcement.

This week’s episode is part of a series we’ve done examining how tax rules affect marginalized groups. Some of our previous episodes include the intersection of tax and racial inequality, LGBTQ rights, feminism, diversity in international tax policy, tribal taxation, and wealth and inequality.

Today we’re taking a closer look at whether it’s realistic to claim that tax enforcement truly can be colorblind. Joining me now to talk more about this is Tax Notes contributing editor Nana Ama Sarfo. Ama, welcome back to the podcast.

Nana Ama Sarfo: Thank you, David. It’s really great to be back.

David D. Stewart: Now I understand you recently spoke with a professor about this issue. Could you tell us about your guest?

Nana Ama Sarfo: Yes. I spoke with Professor Jeremy Bearer-Friend, who is an associate professor of law at the George Washington University School of Law. His scholarship focuses on several areas within taxation, but it includes the civic act of taxpaying and the IRS’s decision to omit race and ethnicity from federal tax data.

David D. Stewart: Could you give us a bit of an overview of what you discussed?

Nana Ama Sarfo: He’s been doing some really fascinating research on the IRS’s decision to exclude race and ethnicity information from taxpayer data. In April, he published an article in the NYU Law Review entitled, “Colorblind Tax Enforcement.”

Now, as you had mentioned, the IRS’s position is that racial bias cannot occur within its enforcement because it does not collect race and ethnicity data. That position is what Jeremy calls “colorblind tax enforcement,” and his article argues that it actually is possible for racial bias to occur across various steps of the tax enforcement process.

He explained some of the tax enforcement activities that are vulnerable to racial bias, why they are vulnerable, and ways in which the IRS can collect and aggregate race and ethnicity data so that it can guard against potential racial bias.

David D. Stewart: All right, let’s go to that interview.

Nana Ama Sarfo: Jeremy, thank you so much for joining us on the podcast today.

Jeremy Bearer-Friend: It’s fantastic to be here. I’m so glad you’re doing an episode on this topic.

Nana Ama Sarfo: Your research on colorblind tax enforcement within the IRS is so important. First, I’m hoping that you can provide some background for our listeners.

What inspired your research and why did you decide to investigate the ways in which race and ethnicity can influence tax enforcement outcomes?

Jeremy Bearer-Friend: Great question and a great place to start. I think that there’s a few points I’d like to begin with.

The first is to just be clear about the stakes of this issue. We have over 150 million households filing each year. Treasury is handling over $3 trillion of remittances. There’s a lot of personnel and citizens who are all tied up in this process and yet the existing practice has been not to report on race or ethnicity within tax enforcement.

While the other federal agencies that are responsible for enforcing the law, for example, Department of Justice or Bureau of Prisons, will share with the public basic descriptive statistics by race or ethnicity, that’s not something that our IRS currently does.

I wanted to then look at a theoretical question of, “OK, if it’s the case that these agencies aren’t reporting any information on it, is that really enough for us to just look elsewhere? Or should we actually dig deeper?”

Nana Ama Sarfo: Now, as you so clearly explained in your article, the IRS’s current and former commissioners did not think that the IRS should collect taxpayers race or ethnicity data. I thought that you present an extremely clear picture of just how entrenched this opposition is. On one hand, it is bipartisan and it can be found across various levels of the IRS.

Now, obviously you are not these policymakers and you are not in their minds, but I’m curious. From what you can infer, what might be causing this seemingly systemic unwillingness to evaluate whether racial bias is occurring within America’s tax enforcement?

Jeremy Bearer-Friend: It’s a really interesting question and of course it’s somewhat speculative. We can’t read minds. I will also say upfront that this is a policy area that is moving quite quickly. There have been some encouraging developments even since publication of this NYU Law Review article that are encouraging. Nevertheless, the public reporting is still not happening.

I think it is quite fair for leadership within the IRS to think of their institution as quite politically vulnerable. It’s an organization that was defunded over a decade. It’s also an organization that’s been the target of regular domestic terrorism. The IRS commissioner himself has Secret Service detail. So I think they have a sense that this is politically sensitive.

We want to protect our work and somehow race is a scary topic to discuss. If we were to raise that issue, it could be divisive and maybe it would jeopardize our future funding. This is, again, speculation. It’s not a justification I’ve heard said publicly but that’s my guess about some of the intuition.

Nana Ama Sarfo: Well, thank you for shedding some light on that. Along the lines of what you had mentioned that race might be a scary topic to discuss or delve into, we know that the IRS’s general stance is that collecting race and ethnicity data might cause disparate treatment.

But as you mentioned in your article, other legal disciplines like criminal law have embraced the reality that racial bias permeates all aspects of the criminal enforcement system. They have used that information to try and neutralize that bias.

What is your response to the IRS’s stance on collecting this data?

Jeremy Bearer-Friend: Well, this really gets right to the heart of the article and the question that I was exploring here. Is the decision not to ask taxpayers to identify their race or ethnicity on a tax return sufficient protection against racial bias and tax enforcement? Is it enough to have essentially a “don’t ask, don’t tell” policy on our forms to then ensure that our tax code is enforced equitably by race and gender? Here, I was looking at race and ethnicity, but of course you could include other axis of oppression if you wanted to.

To answer that question, I thought it was important to have multiple definitions of what would constitute racial bias in tax enforcement. I don’t believe there is consensus on that and I wanted to reach a broad audience to consider what categories of bias might reach different academics and also administrators.

The most clear picture of racial bias would be a racial animus model. I also looked at an implicit bias model and a transmitted bias model. I know over time we’ll get to dig into those and what those terms really mean.

But the bright line conclusion is that by deciding not to include a question about race or ethnicity on a tax form, nevertheless, there are many different opportunities for racial disparities to persist and perhaps even be aggravated in our tax enforcement systems. Until we improve our data practices, we’re not even going to know where to look to intervene and how to do so.

Nana Ama Sarfo: I think that’s a very powerful finding. The absence of information does not mean that there is an absence of discrimination or disparate impact.

But I also wanted to touch on the distinction between class versus race when we talk about racial discrimination or disparate impact in tax policy. I think that the distinction is so important because many times when people are presented with scholarship on race and tax, they tend to respond that racial bias or disparate impact in this area isn’t truly an issue of race, but it’s actually an issue of class.

What I appreciated in your article is that you painstakingly show how racial bias can appear within many different tax enforcement activities, and why and how that bias can occur. I’m hoping that you can summarize for our listeners some of your findings.

Jeremy Bearer-Friend: Great question. Yes, I think you’re exactly right. It’s helpful to be concrete here. What do we even mean by tax enforcement and really what do we mean by bias? We can think through maybe even some specific settings.

In the article on colorblind tax enforcement, I looked at seven distinct tax enforcement settings: summons enforcement — that IRS has broad summons powers — also civil penalty assessments, as well as penalty abatements, offers in compromise, appeals, collection due process, innocent spousal relief, criminal investigations, specifically, including referral to the Department of Justice.

So many different categories of enforcement beyond what was more well reported on, which would be the audit selection that already previously had enough commentary on it. I wanted to broaden the scope.

In all of those settings, the question I was looking at was, “OK, under what conditions might we see racial bias here?” Again, I was not accusing the IRS of any specific behavior. I was speaking theoretically.

If we have a shared commitment not to have racial bias in these systems, how might it be possible?

Some of the necessary elements here would be, to the extent that staff have discretion over the outcome, are there human beings who are making judgment calls about how much an abatement should be worth, or whether to accept a settlement, whether to approve innocence spouse relief, whether to refer to DOJ? There’s a lot of discretion that’s actually going into these processes. It’s not lawless. Of course, it must adhere to the Internal Revenue Manual itself, and then that which is enforcing the Internal Revenue Code.

But in the context of a summons, for example, a judgment call about whether to have included third-party summons, should we pull in outsiders to the taxpayer in addition to the taxpayer? How many tax years are we going to pull in?

You can decide really sort of how aggressive to be. Whether or not in those settings, there are certain cues that could nevertheless reveal racial or ethnic identity of a taxpayer separate from a question.

It’s well documented in a number of other areas that something like surname can be used as a proxy for race. Other sources of information that are also combined on a return with first name and surname, so family structure, even categories of employment, to the extent there is a phone or telephonic or in person interaction, video interaction, then you introduce potentially accents. You could introduce skin color or hair color. All of those cues end up appearing within the enforcement process, even without a question.

When you combine access to cues that identify race or ethnicity with enforcement that entails discretion, you could see disparate outcomes. That’s under both a racial animus model, where the bias is intentional, as well as an implicit bias model where the bias is not intentional. Nevertheless, we see disparities and outcomes.

Nana Ama Sarfo: You had mentioned how the discovery of discriminatory tax enforcement in the Netherlands caused a political scandal and led Prime Minister Mark Rutte and his entire cabinet to resign, which is an example of accountability within the system.

That anecdote made me wonder if there are any countries that actively collect their taxpayers’ race or ethnicity data that the United States can potentially look up to as an example. Do you happen to know of any?

Jeremy Bearer-Friend: We do see other countries ask about race ethnicity in other contexts. In the census context, for example, or even in safety net benefits, it’s common in Canada, in Ireland, and the United Kingdom for the question to appear.

But in tax filings, the latest development would actually be Oregon. Looking domestically, Oregon’s legislators introduced a bill to add race or ethnicity to the income tax form in that state. That was principally motivated to allow those who make the tax law to be able to understand how the burdens and benefits of that tax law are distributed amongst taxpayers in the state. That seemed important from a transparency standpoint, from a democracy standpoint.

I don’t know whether that will end up being law, but should Oregon adopt that policy, we might see other states follow suit.

Nana Ama Sarfo: Very, very interesting. That will definitely be a development to watch.

Now, speaking of domestic policy, as you very well know it’s difficult to talk about the IRS and not discuss its ongoing budget issues. The recently enacted Inflation Reduction Act will increase the IRS’s budget by about $80 billion over 10 years. A significant portion of that money is earmarked for tax enforcement.

As the IRS ramps up its enforcement activity, what tools does it already have at its disposal to help prevent racial bias and enforcement?

Jeremy Bearer-Friend: Luckily I think IRS is in a great position to do more on this, and so we’re in a quite optimistic period. We have a president whose very first executive order was a racial equity executive order and that included forming a racial equity data task force. That task force exists outside of the IRS, but nevertheless is trying to move the ball forward, which is encouraging.

But we also know that most agencies are conducting their own internal civil rights reviews in compliance with the Civil Rights Act. The IRS really doesn’t have to reinvent the wheel here. It can look at what are the best practices within the Social Security Administration, the Department of Housing and Urban Development, the Department of Education, or even the Department of Agriculture?

There’s so much accumulated work here that I think the IRS can start actually pretty far down the road.

It doesn’t have to be from scratch and I think the work begins with improving the internal data. Tax return information has very important and necessary privacy protections.

There are some additional challenges that the IRS might face on that from a data standpoint, but even if it could do more internally to evaluate how enforcement choices at a macro level sort of high up the chain might have disparate impact, I think that would be really useful information for leadership.

An example of that is not the frontline worker who is making decisions after reviewing a return, but someone high up in IRS chief counsel or maybe even the commissioner themselves deciding whether to prioritize audits of large pass-through business entities. Who has benefited from the vanishingly low audit rates and effective non-enforcement of tax offer? Do we want to preserve that under enforcement?

I think data can help with that kind of decision making.

Nana Ama Sarfo: Understood. Really important points you made there that the IRS is not starting at zero here. It has a pretty vast library of information that it can already pull from.

To close our conversation, if you had to identify a series of first steps that you would like the IRS to take in ensuring racial equity and tax enforcement, what would those steps be?

Jeremy Bearer-Friend: That’s always hard. It’s like choosing a favorite child or something like that. I don’t even have children, so maybe that’s overdramatic, but there’s so much I would love to see them do.

In the end, of course, it’s true, they can’t do everything. Where do they start?

Well, one area we haven’t focused on yet, but I think actually might be most appealing to the administration, would be looking at problems related to transmitted bias. This would be an area of tax enforcement where there’s actually no racial animus amongst any staff member that would be responsible for a disparate outcome in tax enforcement by race or ethnicity. There’s no even implicit bias that would produce this disparate enforcement by race or ethnicity.

But instead there are transmitted biases that have occurred outside of the tax context. For example, generations of redlining, generations of predatory lending, generations of discriminatory criminal justice enforcement.

If areas outside of tax enforcement nevertheless were contributing to disparate outcomes in tax enforcement, that would be something the IRS would not want to compound that injustice and they would want to do everything they could to make sure that they weren’t reinforcing that transmitted bias.

So I think that’s a very promising place for the IRS to start. Doesn’t require any finger pointing about bad actors within the IRS, but could have real material impact on taxpayers’ lives.

Nana Ama Sarfo: Well, Jeremy, I have to thank you so much for shedding light on all of this.

Jeremy Bearer-Friend: Thank you. My pleasure.