What if we treated wrongful convictions and bad police shootings the way we treat plane crashes?

In a Q&A, Quattrone Center Executive Director John Hollway explains how "sentinel event reviews" can improve the criminal justice system

---

[Disclosure: John Hollway is an associate dean executive director of the Quattrone Center at the University of Pennsylvania Law School. I’m currently a journalism fellow with the center, which is a paid, part-time position.]

When the criminal justice system goes terribly wrong, it’s rarely the fault of a single bad actor. A wrongly conviction typically includes errors or malfeasance by police, prosecutors, defense attorneys, and the courts, not to mention possible contributions from crime lab analysts and other expert witnesses. Even a bad shooting by a single police officer are usually the product of institutional failure. Was the officer trained properly? What was the officer’s personnel history? Should the officer have been fired for previous misconduct? Does the police department use an early warning system to flag potentially abusive or trigger-happy officers? If not, why not? If so, why wasn’t that officer flagged?

A sentinel event review, or SER, is an attempt to dig into and correct these institutional failures. The idea is to bring in all the relevant parties to get at the root of what caused an outcome that everyone agrees is unacceptable.

The inspiration for the idea comes from two fields outside of criminal justice: the National Transportation Safety Board (NTSB) investigations of plane and train crashes, and the morbidity and mortality (M&M) reports hospitals conduct after medical errors, such as amputating the wrong limb or administering the wrong medication.

NTSB and M&M reviews don’t look to blame on individual actors. Because of this, they tend to get better participation from the parties and institutions involved. But the reviews don’t confer immunity onto anyone, either. If there are individuals who merit blame and discipline, they can still be held accountable or liable by parallel investigations by other authorities.

The aim of a sentinel event review is to figure out what went wrong and why — to get at the systemic errors that allow catastrophic events to happen, and to make policy recommendations to reduce the chances those events will happen again.

John Hollway, executive director of Penn Law’s Quattrone Center, has designed and led SERs in several cities , including the review of the crime lab in Austin, the police response to the George Floyd protests in Seattle, false arrests of teenagers for a mass murder in Philadelphia, and two in-custody deaths in Tucson.

This exchange comes from a phone interview with Hollway last month. It has been edited for length and clarity.

RB: The inspiration for sentinel event review comes from fields like aviation and medicine, where you have these all-hands investigations of plane crashes and medical errors. But law enforcement — and policing in particular — tends to be a psychologically isolated profession that’s incredibly suspicious of critiques from outsiders. How can a SER board get past that skepticism and persuade law enforcement officials to cooperate?

JH: I think I’d start by asking if the isolation you’re talking about is cause or effect. Perhaps one reason people are suspicious of critiques because the critiques are focused solely on blame, often by people who haven’t walked in the shoes of the actors in question, and have not been focused on the systemic factors that contribute to undesired or unintended outcomes.  What aviation and healthcare show us is that starting from a perspective that is less blame-focused, and more system-focused leads to asking different questions, and generates different answers about how to prevent the recurrence of bad outcomes.  And those systems share a lot with criminal justice.  If I were to tell you our goal is to improve a system that's operating in a hyper dynamic environment with lots of different human interactions, imperfect data, things that are changing moment to moment, all in a high-stress environment where people's lives are at risk, would you tell me I'm talking about health, care, transportation, or policing?

The answer is yes. All three fit that description. The other thing that’s true in all three professions — and lots of others — is that people don't like to have their work second-guessed. They get worried and defensive when they think someone is accusing them of making a mistake.

So I think healthcare and and transportation point to the ability to be successful with these reviews — that you can get past the isolation and unwillingness to revisit a bad outcome out of the fear that you are personally going to be held responsible.

They’ve been doing this in transportation for 70 years. They've been doing it in healthcare for 40. We just haven't been doing it in criminal justice, but all the necessary precursors are there for this to be successful. We just have to convince people to make it part of the culture. It’s now just accepted that after a plane crash, the NTSB will be doing a review. It's just accepted that after a botched surgery or a bad clinical outcome, we're going to do a morbidity and mortality review.

That’s where we need to get in the criminal justice system.

When we think about wrongful arrests and convictions, we think of all the obvious issues — police incompetence, tunnel vision, prosecutor misconduct, inadequate defense, and so on. But I think some of the less obvious findings by SERs are interesting. In the Philadelphia review of the wrongly arrested teenagers, for example, you found that vacancies in supervisory positions at the police department were partly to blame for the wrongful arrests because there was less oversight of the investigating officers. Can you talk about some of the other less obvious findings from these reviews?

Oh, we get those in every case. Here’s a good example: In Tucson, one of the deaths in custody that we looked at involved a Hispanic grandmother calling 911 because her grandson was naked and apparently on drugs. He was interfering with her ability to move around the house, and she was afraid for his safety. But the Tucson 911 call center didn't have anyone who spoke Spanish.

I recall reading that in the report. It’s Tucson!

Right. So now you have an abuela who doesn't speak great English, talking to somebody who doesn't speak Spanish, trying to explain what is going on, and there is obviously a lack of fidelity in the communication. What then gets transmitted to the patrol car is not what she told the operator. What does get transmitted is that there’s an outstanding warrant on the grandson for a domestic violence dispute that happened a couple days before.

So now we have officers running in, thinking that they're going to arrest someone who’s using force and and beating people up instead of a naked guy whose grandmother called 911 for what was pretty clearly a behavioral health or drug issue. It changes the entire tenor of the interaction.

There’s also the Walter Wallace shooting in Philadelphia. There, the officers responded to what they knew was a behavioral health call. But they didn't have tasers, because even though Philadelphia was supposed to get tasers. At that point the purchasing process and budgeting only allowed tasers for a third of the department.

So the issue wasn’t that the Department had refused, or didn’t want tasers. It was that these specific officers didn’t have tasers because the city couldn’t purchase enough tasers quickly enough.

So the issue was that because they didn't have a less lethal option available, they used their guns?

Right. It turned out to be a bureaucracy problem, not a policy problem.

---

---

One of the stated benefits of SERs is that they can build public trust by showing that police agencies and DA’s offices are amenable to this sort of review — that they want to improve. But for the for the first review in Philadelphia, there were confidentiality agreements among the participants. It seems like there’s a balancing act between making participants comfortable enough to be candid, and possibly undermining public trust by keeping discussions confidential.

Trust is already pretty frayed between communities and lots of different agencies in the criminal justice system. I think it would be naive not to acknowledge that. Even when we’re completely transparent and there are no confidentiality agreements, like in the Tucson review, we still had people watch those videos and see very different things. So I don’t know if you’re ever going to have a situation where you please everyone all the time.

We granted confidentiality in the Lex Street [Philadelphia] review to make sure we could get the relevant agencies together to participate, but everyone participated with the expectation that once we all agreed on a consensus report, we would publish it.

Often you’ll have mistrust between the agencies involved. So let’s say you have a wrongful conviction case. Well, a wrongful conviction is often the result of multiple failures. You have an inaccurate arrest. You have a prosecutor who took and tried the bad case. You’ve got a defense lawyer who may have failed to recognize or challenge the problems with the case. You’ve got the judge who was responsible for overseeing the quality of the trial and ensuring that it was fair. And you’ve got a jury. With a wrongful conviction, all of those checks and balances have failed.

The SER process democratizes opportunities for improvement. Everyone can help diminish the risk of this outcome occurring again. If you’re one of the institutions involved and you don’t participate, you risk having all the other participants sit around and point the finger at you.

Lex Street was the first time we tried to do this, and in order to create an atmosphere where people felt safe to share, we agreed not to publish unless everyone who participated agreed to publish. The CDA [confidential disclosure agreement] allowed people to speak freely. They knew we weren’t going to take something they said and use it for some other purpose.

So in that case, CDAs were a useful tool to get people to start thinking about the philosophy of these reviews, and to begin to believe in it. But we typically don’t do the CDAs anymore.

When the Philadelphia review began, the wrongful arrests had taken place 13 years prior, and all the litigation had ended. As you’ve acknowledged, that made it easier to get buy-in from all the agencies involved. In other cities, you’ve had reform-minded prosecutors and police officials who seem more amenable to a project like this. Do you think you’ll be able to get buy-in with newer cases, or in jurisdictions with more traditional DAs and law enforcement leaders?

So you’re exactly right. The reason we went with the Lex Street massacre in Philadelphia was that all of the litigation was done. It was a high-profile case that happened under previous regimes for all the agencies we were looking at.

I’d say the single biggest barrier to these reviews right now is the fear that by doing them we’re arming plaintiffs’ council with ammunition to sue. This is why in healthcare they have peer review Protection Acts, and in transportation, the enacting legislation for the NTSB states that while their data is available, their conclusions can't be used as evidence in civil litigation.

So there are safe harbors for these other types of event reviews that we don't have in criminal justice. I think this is something that we really need to look at. We could create a safe harbor for this sort of review that wouldn't limit any of the other review or investigative processes. It would just allow an event review to operate in parallel.

So all of the accountability processes that we currently have, whatever you might think of them, they can all continue. But we could have a preventative, quality improvement, and safety review also going on. Right now, that sort of review isn’t happening because the threat of litigation chills it.

I'd like to explore that a little more. It seems like there are some interesting ethical questions, here. In both criminal and civil rights cases, plaintiffs often have a difficult time getting discovery. What happens if a SER board finds evidence that should have been turned over, but legislation or an agreement among the participating agencies says nothing uncovered in one of these reviews can be used in litigation? It seems like the sort of thing that could come up often if SERs become more common.

Well, I guess it could come up that way. It can come up a lot of different ways. Let’s assume we're doing a review with a DA’s office on a wrongful conviction, and somebody tells us, in the middle of that review, about something they didn't disclose that would qualify as Brady material. At that point, that disclosure might then be governed by different rules than the rules of our event review.

We could be obligated to report that to the responsible agency, and the agency would then handle it according to their policies. But again, that's something that can be addressed in the legislation, so that everybody knows their obligations going in. You can control how the safe harbor works. Any time you've got multiple investigations asking questions about the same event, there’s a risk that you might turn up some different facts.

So let's say you have an agreement about a safe harbor for information, particularly from police and prosecutors offices. A defense attorney in post conviction litigation or a civil rights attorney then subpoenas the review board for information. Would the board fight the subpoena? Or would that sort of thing be written into the legislation establishing the board?

So we have done this. In the past we have been subpoenaed. And we have turned over the information as a result of the subpoena. So that's that's how that has worked. But we have not turned over information without a subpoena.

We do think that the good these reviews can do is furthered by being confidential. But when did the review of the DNA section in the Austin Crime Lab, there were some defense counsel who subpoenaed what we found. We turned over the documents and they used that information. Because we were contracting with a state agency, it was part of the Texas Public Disclosure Act, so we turned it over.

The question then becomes, by turning over documents, did we chill any of these agencies from participating in the next event? I can't answer that question yet, because we haven't done another review in the same city. I think it's a risk, but it's not a guarantee.

How would you get around other barriers to transparency? If you’re looking at, say, a wrongful conviction that involves ineffective assistance of counsel, you could have attorney-client privilege issues. You could have work product issues. And in many places, police and prosecutor personnel files aren’t public record.

So these reviews are a voluntary exercise. There's no there's no law compelling anybody to do them. Our belief is that you get value out of the exercise, even if you don't have 100 percent of the information that you might want to have.

Lex Street is a good example. Those events happened in 2001. We looked at it 2015. There were people who were no longer in their roles. There were people who passed away. We never actually spoke to the the main prosecutor in the case. That prosecutor had already been the subject of allegations of serial Brady violations, witness manipulation, and all sorts of other misconduct, and has had more allegations since, and we never talked to him.

Were there gaps in what we were able to do because we couldn’t talk to the prosecutor? Yes. But that meant we had 25 great recommendations for change instead of 45. The review still moved the needle forward. But I can point to Lex Street and say, hey, we were able to make positive change from a case that happened 15 years ago.

We are now working on an exoneration where the guy was in jail for more than 38 years. The evidence that freed him was a a blood test conducted on the last day of his trial, though nobody knew about it at the time.  What we’re finding is that there’s a million different things we can learn from these cases, even if they’re old. I’d love to do all of these reviews right away when all the information is fresh, when all the people are still alive, and I’d love to have complete access to all of them. That's the ideal.

But I think you need to work backwards from the information you can get. Given the limitations like the ones you've mentioned, you have to bring people together, you have create an environment where people feel psychologically safe to talk openly, and you say these are the things we've learned, and here’s what we can do to improve.

Improvement is inherently a game of whack-a-mole. So we'll whack all the moles we can, and then move on to the next one.

Getting back to liability, there’s been momentum in the reform movement to abolish qualified immunity for police, and even some debate over the absolute immunity given to prosecutors. But it seems like there could be some tension between those policy goals and an SER. Reformers want more individual liability. You want to do this systemic review, but to do that you have to convince the agencies involved that this isn’t about blaming individual people.

There can be some tension. We’re on the same side in terms of wanting change. When I got started in this area, I talked to this to a guy named Chris Hart, who at the time was the chair of the National Transportation Safety Board, and was the only lawyer on the board. I told Chris that when I’m talking to people about this and I start talking about blame-free event reviews, I can see the window shades close in people’s eyes. But he put it in a way that I think can help people understand: Just because the NTSB does a blame-free review doesn’t mean a pilot can get away with showing up for work drunk.

There’s this concept of “just culture.” It’s been really well articulated by a guy in Australia named Sidney Dekker. The idea is that even reasonable professionals might create shortcuts in what they do that can lead to unanticipated risks and unwanted consequences.

That’s a problem, but it’s a different problem than a pilot showing up drunk. When we put people in positions where it's hard for them to succeed, the people who want to do the right thing will start to take shortcuts. And sometimes a shortcut can go wrong ways they didn't consider.

So I think it’s important to differentiate between intentional behavior, negligent behavior, reckless behavior, and accidental behavior. That’s essential.

So it’s possible that an event review will recommend policies to change the environment so officers make better decisions, but the officers responsible for the incident that inspired the review should still be fired for what they did.

The big challenge with qualified immunity is that the Supreme Court has chosen to look at these cases on a really anatomical level. I’d personally rather see this be a standard torts issue. We don’t want officers not doing their jobs because they’re afraid of being sued for unintentional behavior. So maybe we don’t hold them civilly liable.

But they can be disciplined in other contexts. We're not going to give you absolute immunity, or even the sort of quasi absolute unity that comes from this need to replicate a very specific fact pattern from a previous case to be held liable for a constitutional violation. I don't have a problem in my mind with some sort of elevated immunity. But it's gone way too far, and I think it could be kind of simplified into a common sense torts framework.

But we’re never going to reduce the frequency of these problems if we don’t get at the underlying systemic factors and try to be preventative. We can’t just rely on accountability and civil liability to do the trick.

As you say that, I’m reminded of the shooting of Levar Jones by South Carolina state trooper Sean Groubert.

Groubert had pulled Jones over and asked him to get out of his SUV. He then asked Jones to retrieve his ID. As Jones reached into the vehicle to do exactly that, Groubert panicked and opened fire. Jones was unarmed. Groubert was later charged and convicted.

I remember watching the video of that shooting and thinking that Groubert was clearly terrified of Jones, even though Jones posed no threat. I’m sure racial bias had something to do with that. Groubert had also been involved in a prior incident where he’d taken gunfire. And we know that from the academy on, police officers are inundated with this idea that every traffic stop could quickly turn into an ambush.

Groubert should never have been a cop, and it seemed correct to charge him with some sort of crime. But unless you address all of those other issues — the fear, the bias, the hiring standards, the training — charging him isn’t going to reduce the chances that an incident like will happen again.

I think that’s right. Whether you're talking about Michael Bell, Tamir Rice, or your South Carolina case, I think that without evidence to the contrary, we start with the proposition that nobody woke up that morning thinking, man, “I can’t wait to kill someone today. I can't wait for this to happen.”

So if nobody woke up wanting this to happen, I think the question is, why did it happen? Some of it might be because this police officer wasn't emotionally equipped to handle the situation, and therefore they acted in ways that are inappropriate, and we can't give them the chance again. But we should also ask, why wasn't that officer emotionally prepared to handle that situation correctly? What policies created that level of fear, and how can we address it so we don’t have another officer doing the same thing?

In the Tucson review, we looked at two different deaths in police custody. In one, the officers did everything they were supposed to do and they still got the bad outcome. They followed all the policies. They did the things they thought were right. They had reasons for everything they did. An independent review found that they used the appropriate amount of force for what they were dealing with, and subsequently did everything to try to protect the individual in their care, right up to having the paramedics come and administer vitals, after which the person said he was fine. It wasn’t until they were packing up and walking away that he went into cardiac arrest.

The other case involved the abuela and her naked grandson. In that one, the officers departed from policy, and they were administratively discharged by the time we had started our review.

You can imagine how these different situations should be handled differently. In the second case, we came up with suggestions that might have stopped those officers from going off the rails. But they still went off the rails in ways that were deemed unacceptable for them to continue with the department.

The cultural framework of these reviews should provide a way for management to make policies that reassure not just the community, but the other police officers watching. It’s the difference between a good cop in a bad situation — but a situation that might be preventable with the right policies — and a cop who makes decisions that are unacceptable. That distinction is really important.

We’re not trying to skirt the issue of personal accountability with these reviews. We’re trying to set conscientious people up to be the police officer, the prosecutor, or the public defender they wanted to be when they took the job.

It seems safe to say that these reviews are still in the proof of concept stage. So I’d imagine that when you’re choosing where and when to do one, part of the equation would be picking a jurisdiction where you’re most likely to get buy-in from local officials. But let’s say the idea catches on and they become widespread. For what sort of incidents would these reviews do the most good? It seems like they’d be less helpful when you have clear bad actors — where the blame is obvious — and more helpful where you have good intent but a bad outcome, where the blame lies with institutional failures that can be difficult to identify.

I think a sentinel event review can be helpful any time you have an undesired and unintended outcome. In cases in which the behavior was not intentionally bad, the question is how did we get here when nobody wanted to be here?

But then there are cases where you might have a bunch of rogue cops, or a case like Philip Nordo here in Philadelphia, who was just convicted and sent to jail for serially sexually abusing and manipulating suspects, leading to multiple exonerations. He was acting intentionally. So first you eliminate the immediate threat by taking an intentional bad actor off the street. But then the question is, how was he able to get away with it for so long? What were the circumstances that allowed him to abuse suspects? Where are the holes in the quality control process that allowed for such a failure of oversight?

We don't want other bad actors thinking they can get away with that sort of thing. Clearly this guy not only thought he could get away with it, he did, and for a long period of time. So I think, even in a situation with a clear and intentional bad actor, there are a lot of opportunities to improve the system. The question is where you choose to to plant your shovel and start digging.

With police shootings in particular, I think we spend a lot of time debating the wrong question. There’s all this discussion about whether the shooting was legal and whether the offer should be charged, but we don’t talk enough about whether the shooting was preventable — whether, regardless of legality, this is the sort of shooting we’re willing to tolerate.

Every officer shooting, every death in custody, every wrongful conviction, every on-the-job police injury. Put personal accountability aside. These are not events we want to happen. So we should be trying to reduce their frequency.

I think a lot of the time when the police and the community discuss one of these shootings, they’re not having the same conversation. The community is talking about public safety, while the police are talking about enforcing the law. So when we’ve managed to get the police and community together in these event reviews, we’ve realized that the police have been studying for a test that’s different than the one being administered in the court of public opinion.

The police say, “Look, we did what the law requires and what our policies tell us to do.” And the public says, “Yeah, but the way you’re doing it isn’t keeping us safe. You’ve sworn to serve and protect us, and you aren’t keeping us safe. So we don’t trust you.”

So I think these events can be opportunities for real healing — to bridge that communication gap so police understand the injury the community feels after one of these incidents. The community gets a better understanding of where the police are coming from and how they are trying to address community safety.

When we recommend a policy change, it’s based on feedback from both parties. So people understand why the change should be made. The police end up excited about it, about implementing something they they know the community agrees with.

I’ll give you a good example. In Seattle, we looked at the the way the city handled the George Floyd protests. The police had established a static line downtown in the first couple days. They were telling the crowd, “You may not go to these places.”

Well, the that might work with a Greenpeace protest. But these activists were protesting the very legitimacy of the police to tell them what to do.

The police are dressed up in all of their armor, and they’re drawing lines across the street. They are becoming all the things that infuriated the protesters in the first place.

In the months that followed, it became clear that the police didn't really know the protesters’ goals. Where do you guys want to go? Why do you want to get there? What is your intention once you’re there? The police are hearing about and are worried about antifa violence and Molotov cocktails. Meanwhile, the people in the protests don’t want to be told where they can go.

After bunch of conversations, Seattle created these intermediary positions [called POET officers]. These are officers are trained in negotiation, and then actually embedded in the crowd, though they are designated as law enforcement.

You want someone there if the protest veers off course and wants to go somewhere that wasn't scheduled — because, of course, these things aren't scheduled. The whole point of the protest is to disrupt, so why would they notify law enforcement first?

But the POET officers are there and trained to say, “How can we make sure this happens in a way that you get to go where you want to go, but without creating any danger for others?” Those officers then relay what’s happening to the command center, so the Command Center can make modifications.

So the police are giving the community more agency in the protest. But the community is acknowledging that they don't want to harm others. And so they’re finding a way to let protesters make their point without endangering other people — so someone doesn’t set fire to a closed Starbucks not realizing there are people asleep in the apartments on the floor above, and now the building's on fire and we've got loss of life.

It's a way to navigate some of those things, so that people on both sides of the protest are speaking the same language.

I’ve read the SER reports from Tucson, Seattle, and Philadelphia. And I know you’ve overseen or helped with others. Can you talk about some examples where city officials have implemented a review board’s policy recommendations?

Sure. So in Tucson — and Madison they’ll do this soon — they actually published reports about our policy recommendations one year after implementing them. “This is what we've done, this is where we've run into challenges, and why” — that sort of thing.

And I think that's great. It’s a great way to say this is not just another report that’s going to sit on a shelf. This is a process, and we’re delivering on it.

In Seattle, the police chief has has expressed support for the recommendations, and I know they are moving forward with the intermediary officers and some other recommendations.

Not every department has done that. After an event review of the crime lab issues in Austin, the city council voted to create a a separate crime lab that’s no longer a part of the police department. Once that’s up and running, it remains to be seen how many of our recommendations the new lab will implement. So it can depend on the structure of the system. And it can take some time.

This is also a voluntary process. There is no enforcement authority for these reviews. But the idea is that by including the agencies in the review and the process that results in these recommendations, you have buy-in from them. Journalists like you can go back and hold them accountable., You can say hey, you signed on to this report a year ago, but you haven’t taken any of the recommendations, why is that?

Right now, all of these reviews have been in response to criminal justice excesses — deaths in police custody, wrongful arrests, aggressive responses to protest. Do you think they’d be as effective at addressing failures — things like failure to prevent or properly respond to a mass shooting, failure to test rape kits, low clearance rates?

We try to talk about errors rather than failures. An error is any undesirable outcome that the system wasn't designed to create. And so in that sense a cold case is an error, right? We weren't able to solve the crime. So I’d love to do an event review of a cold case that asks the question, “Why didn't the original team solve this crime?”

You wouldn’t have the psychological risk of blaming the current actors, so the question becomes more of a constructive exercise, so you might get more participation. So what policies would have helped investigators solve this crime X number of years ago? And what can we learn that would improve our investigative practices today?

There are also cases where we know someone actually did commit a crime but for various reasons was never prosecuted. In one of the cases we looked at, somebody was convicted of attempted robbery for a carjacking, but the conviction was overturned because of speedy trial rules.  So the system’s inability to bring charges in a reasonable amount of time prevented a an allegedly righteous conviction for a violent crime. And a system like that is probably violating other people of the right to a speedy trial, too.

So yeah, I think I think all of the contexts that you've described are ripe for this kind of conversation. It’s up to the city or jurisdiction we’re working with to say, “I want to fix that problem.” And once they do that, the goal is to get all the participants to agree on what caused the error.

Is there a danger that the process could be abused? I could see demagoguing politicians call for a sentinel event review to criticize a policy like bail reform to draw attention to an aberrant case in which someone given low bail committed another serious crime.

That certainly isn’t the first place I’d start. But I get it — I can see how someone would call that an error and want to find out what allowed it to happen.

Look at this horrible case in Texas [note: this interview was conducted shortly after Francisco Oropesa killed five neighbors after one of them asked him not to fire his rifle while their children were sleeping]. One error there is that this guy who has been deported multiple times was able to get an AR-15. He should have been red-flagged on several occasions.

I don't know all those answers, I think blaming this event on his immigration status would really be missing the point, but we could start from the point of, “We don't want people walking next door and shooting five people,” and work backwards from there.

I also think there’s also a question about whether police responded promptly to the distress calls, and if not, why not.

Right. So you put all of this information into the the fishbone diagrams we use, and you show that it's almost never just one thing. It's all of these things coming together leading to this bad outcome.

There’s been some interesting discussion about the role insurance companies could play in criminal justice reform. There have already been some instances in which municipal insurers insisted on reforms after large settlements for police abuse or wrongful convictions. Some have suggested that instead of qualified immunity, police officers should be required to carry personal liability insurance, and that insurers would then insist on training, behavior, and policies that would lead to fewer constitutional violations.

But you can also see how that could go wrong. You could also end up with insurers lobbying to make public records more opaque, or to make it more difficult to sue. What role do you think insurers could play in these systemic reviews?

Liability questions are going to be inevitable in any attempt to sort out why an incident occurred. As I think you’re aware, Joanna Schwarz had done some really interesting and thoughtful work on this.

But I think cities and their insurers should want to participate in these reviews. For however many thousands of dollars you’re paying for an event review, you’re hopefully preventing who knows how many multimillion dollar settlements down the road.

I’d like to see a city’s participation in one of these reviews allowed as admissible evidence that the jurisdiction is trying to do the right thing — that officials are at least trying to figure out how to minimize these incidents. I’d also love to see insurance companies reduce premiums for cities who participate in reviews, or make participation a condition of getting municipal insurance.

I agree with you that there are some difficult questions here. Who’s paying for what? What leverage do the insurance companies have? What do you do with a self insured city?

But you also have what I’d call the Amtrak tunnel problem, where you have a tunnel that has been in need of repair for 50 years, and New York can always blame New Jersey and New Jersey can always blame New York, and it's never this particular mayor's problem.

Have you thought about working with plaintiffs’ attorneys to make these system reviews part of settlement agreements in civil rights cases?

If reform is a priority for the client, I’d of course welcome that. But I wouldn’t want to push for it. You don’t want to put someone in the position where the city says, “Sure, we'll agree to a review. But we're going to take the cost of that out of the award to an exoneree.”  That’s not where the equities should lie.

So I think if a victim wants to insist on reviews as a condition of settlement, that’s great. But it should be up to the client — it should be up to the people who were wronged.

You mentioned that the cost of one of these reviews pales in comparison to a multimillion dollar settlement. How much do these reviews cost? I’d imagine it’s pretty case-specific.

Very case-specific. If you’re reviewing an in-custody death after a 10-minute interaction with police, that’s going to be very different than going to Madison, Wisconsin, to review 90 days of protests and the police reaction to them.

It really depends on the complexity of the case. Any wrongful conviction occurs over months or years, and that is going to take more time and be a lot more involved than a police shooting that unfolded in a 4-minute interaction from start to finish. There are a lot more documents to review, and you've got everything from the arrest through the trial and appeal. We were able to complete the Tucson review in a couple months.

But if you’re saying cities will save money by doing a review that could prevent judgements or settlements down the road, I’m guessing we’re talking about tens or at most hundreds of thousands of dollars?

That’s right.

The DOJ literature on SERs suggests that “near misses” may be better candidates for reviews than incidents with clear bad outcomes. That makes some intuitive sense, but could you flesh that out a bit?

This is something they do really well in aviation. When there’s a near-miss, people tend to say, “Good catch! We really kind of dodged one there.” But in aviation, they try to learn from them.

Near-misses are good candidates for event reviews because they had all the necessary prerequisites for a bad outcome but didn’t actually result in one. So the people involved are less likely to back into this defensive crouch because they’re worried about losing their jobs, getting sued, or other consequences. The air traffic controller doesn’t go to jail if the planes don’t crash.

But that doesn’t mean the system worked. There was some luck. The factors that led to the near-miss are still there, so the continued risk of a bad outcome is still there. But the people involved are more willing to talk about what may have gone wrong. So they’re great opportunities to introduce the concept of system reviews — to improve things before you get a tragedy.

Flawed forensic testimony has been a major contributor to wrongful convictions, and there’s a big debate right now about the role experts should play in the criminal justice system. In some of these reviews you’ve looked at incidents involving topics where prosecution and defense experts can be in sharp disagreement, like excited delirium or false confessions. Presumably review board itself would have to consult with experts in such cases. In cases like those, how does an SER board determine which experts to consult?

When we had some some discussion of excited delirium in Tucson, here’s what we did: We talked to clinical experts about what was going on in the body, about the chain reaction that led to cardiac arrest. We didn’t need to address excited delirium and whether it’s a valid diagnosis. Instead we asked them, what recommendations can we make to stop the dominoes from falling that cause someone to go into cardiac arrest?

Excited delirium can sometimes be used as a buffer against accountability for police officers. What we tried to do was, we said look, there are situations where the human body goes into cardiac arrest, and there are contributing factors that lead to that. In this case, we know the contributing factors were whatever toxins might have been in the person's body, the exertion going on in this person’s interaction with the police, how the police treated this person once they had him handcuffed, the equipment they were using, the medical assistance the person received, and at what point the police transitioned the control of the individual over to the medical professionals.

If you want to call that excited delirium because there was a sudden onset of cardiac arrest after all of those factors, that’s fine. I’m more interested in how we can interrupt that chain of events before they lead to an unnecessary death.

So we tried to find people who were knowledgeable about these processes. We also spoke with a psychiatrist who said something particularly interesting — she said she and her colleagues actually go through training on how to deal with people having a behavioral health crisis, and she and her colleagues never do this armed. When the police are called because someone is in crisis, after they apprehend the person they drop that person off with her. And she now has to figure out how to deal with this person without any weapons. And she does it. It can be done. So what can we learn from that?

That makes sense. I’ve always thought even even if excited delirium is a real thing, so what? If there’s this condition people can slip into while interacting with police that can cause unnecessary death, that isn’t an excuse to allow preventable deaths. It’s a reason to adjust your training and policies accordingly so those deaths don’t happen.

Right. It's pretty clear that this person had a heart attack. But we drop back into accountability and liability mode, debating whether or not the police caused the death. So then you get into this conversation where excited delirium becomes a potentially made up diagnosis that then gets used to shield police for bad behavior.

What we should be asking is, what factors contributed to this person dying when that's not the outcome we want?

There's always a risk that experts can be wrong, and I think you have written more persuasively than most on experts who refuse to go back and reevaluate data when science moves forward, after they’ve clearly been shown to be wrong.

So I generally try to get more than one expert on a topic to talk to the group. And then we just do the best we can. But when you do work on wrongful convictions, it instills in you some intellectual humility. You have police, prosecutors, and 12 people who were certain that the evidence proved someone was guilty beyond a reasonable doubt — and sometimes you have appeals courts, too. And you’ve seen how they were all wrong. So I’d never suggest that because we've done these event reviews, we've gotten everything exactly right.

But there’s value in probing what drove the participants’ decision making. I’m going to close with an Eagles metaphor because I’m in Philly and it's the easiest way to make my point.

When Jalen Hurts throws an interception, there’s a big difference between how the fans on the 300 level see it and what Jalen Hurts and his quarterback coach discuss when they look back at the tape. When the two of them are breaking it down, frame by frame, the coach is asking Hurts: What did you see? Why did you roll out here? Why did you stop there? What defense did you think you were playing against?

Should Hurts take responsibility for the interception? Of course. But you also want to prevent the same circumstances from allowing it to happen again. And that's a very different conversation than the fans in the 300 level who are just like, “Man, we really need to bench Jalen Hurts.”

---