Is Fact-Checking Working?

Much time and money have been spent on combatting misinformation through fact-checking. But it’s not clear whether it has any impact at all.

Fact-checkers are taking credit for the circulation collapse of the doctored Nancy Pelosi video that circulated widely online after being aired on Fox news in the US and tweeted about by President Donald Trump on May 24th 2019. After it was highlighted by Washington Post and others, views and shares fell dramatically. Facebook was urged to remove it but declined saying that the company “does not require posts to be true.” Instead they flagged it as false and reduced its distribution. YouTube removed the video altogether. This ambiguous response was typical of the constant skirmishes seen in the ongoing war against online dis/misinformation.

Calling out the Liars

The Brexit vote in the UK and the 2016 elections in the US revealed a global war on truth and, as attention became fixed on the dangers of dis/misinformation, platform companies, foundations and journalists searched for fixes that they could put in place quickly. Misperceptions about issues of public importance have always existed but somehow 2016 seemed to be a watershed. Politics was infused with claims that were widely understood to be false, yet brazen lying was the order of the day. In the 1980s false claims had been met by so called “truth-squadding” so perhaps it is no surprise that one of the ideas that emerged was to expand fact-checking efforts across the globe.

On the face of it, promoting journalistic fact-checking sounded like the perfect antidote. If dis/misinformation had tricked voters and encouraged them to vote for candidates on the basis of lies (Trump saying Mexicans were criminals, Brexiteers saying that Brexit would free up 350 million pounds a week for the National Health Service) then surely telling the truth and calling out the liars would be the best way to address the problem.

Foundations like Luminate (part of the Omidyar Group) Open Society Foundation, Gates Foundation and USAID together spent millions to support fact-checking organizations and expand their work globally. Even Facebook got involved spending in the low millions to support groups all over the world that had signed on to the International Fact Checking code of principles.

Three years and millions of dollars later, have these efforts worked? The answer: it’s hard to know. 

It depends on what you want fact-checking to do and whether that can be measured. Like the famous Journalism of Outrage model that said there were three ways investigative journalism can have an impact (personal, discussions in congress etc., and then policy results) so too for fact-checking. Citing a 2012 paper, by academics Brendan Nyhan and Jason Reifler, fact-checking funder Tom Glaisyer said there are three main ways that fact-checking can have an impact. 

• Changing people’s minds. To provide an effective counterweight to misinformation about any given issue, fact-checking must present the relevant facts to the right audiences in a way that encourages them to question misleading claims. 
• Changing journalism. A goal of many fact-checkers—and especially of media critics who fact-check news reports—is to encourage journalists to not just report competing claims but to assess them, and to challenge politicians who attempt to mislead the public. 
• Changing the conversation. By exposing political deception, fact-checkers and journalists in general may exert pressure on political figures to retreat from misleading claims—and perhaps discourage them.

Changing Hearts and Minds?

Have fact-checks changed hearts and minds? The evidence is mixed.

First, there is far more false information available online than can ever be fact-checked by small teams of fact-checkers around the world. Second, fact-checks are not necessarily seen by the people exposed to the false information in the first place. Also, because it takes a long time to verify information, or prove that it’s incorrect, the false information can circulate while the verification process is underway. Studies of rumors on Twitter find that they continue to circulate long after they’ve been debunked. And people tend to believe something if it comes from people in their group even if that information is wrong. Indeed, providing accurate information does not actually make a difference when people are partisan. In 2016 Trump supporters often said they didn’t mind whether he has a fact wrong because he is speaking a larger truth or telling their story. 

This is borne out by findings that Trump supporters may not withdraw their support for the candidate even when they recognize his statements are false. 

Even those working with fact-checkers in the trenches of misinformation are struggling. “Fact-checking has value but we’re still trying to figure how exactly how it impacts audiences. It’s complicated. Research shows that fact-checking has different impacts on different segments of the audience, depending on the topic, format and context” said Claire, Wardle, one of the co-founders of First Draft News, who has been working with a number of fact-checking groups around the world.

So if fact-checking isn’t changing hearts and minds then is it at least providing data that Facebook and Twitter and You Tube use in their algorithms and thus demote false stories so they won’t be widely shared? Facebook, which has put the most effort into thinking about fact-checking, says yes but doesn’t provide details as to when or how.

How Platforms Respond

The European Union has resisted regulating content on the platforms in part because they have the same worries that the US has about suppressing free speech. Instead, the EU has been pushing the platforms to handle the problem themselves. In October 2018, Facebook, Google and Twitter said they would sign onto the Code of Practice against disinformation with the EU Commission, and in 2019 began submitting monthly reports about their activities. These reports are sketchy and don’t provide a lot of concrete information as to what the platforms are doing to suppress false information. The platforms’ reports focus mostly on fake accounts and political advertising and don’t say much about how debunking lies affects what the platform shows audiences. And the numbers the platforms provide about accounts taken down are meaningless, because the numbers don’t provide insight to how many inauthentic accounts survive. In fact, high numbers of account removal could just represent an acceleration of the creation and removal rather than any progress. If, say, Twitter removes 10% more fake accounts in April 2019 but membership rose 20% then it’s possible they are falling behind in their efforts to remove bogus users not expanding them.

The Twitter and Facebook reports mainly detail what kind of content and policies are unacceptable on their platforms but fail to explain how they are addressing their problems. For instance, Twitter’s April report states that, “people who don’t feel safe on Twitter shouldn’t be burdened to report abuse to us.” Yet, 62% of abusive content that is acted on is flagged by Twitter users.

Facebook repeatedly uses vague language that indicates how far the company is from finding a solution but notes that it’s holding a lot of meetings: “One promising idea to bolster [fact-checkers’] work, which we’ve been exploring since 2017, involves groups of Facebook users pointing to journalistic sources to corroborate or contradict claims made in potentially false content. Over the next few months, we’re going to build on those explorations, continuing to consult a wide range of academics, fact-checking experts, journalists, survey researchers and civil society organizations to understand the benefits and risks of ideas like this,” read one if its spring reports.

Google’s reports are a bit more in-depth, citing new tools like the “Fact check explorer,” new funded media literacy programs like “One World at School,” and new journalist fact check training initiatives, like the 2,743 European journalists they have trained in person to use digital verification tools between January and April 2019.

This lack of information from the platforms has left fact-checkers, and everyone else, dismayed and raises the question as to how serious the platforms are or whether funding fact-checking was essentially a PR move designed to forestall regulation.

Alberto Rabbachin, from the DG Connect’s Social Media and Media Convergence Unit at the European Commission in Brussels, says: “We don’t have detailed information about the number of news items that they have demoted. The numbers we have are the numbers written in the reports that the platforms provided us during the close monitoring of the implementation of the Code of Practice on disinformation. No more than that. We are always asking for more details and numbers in order to allow an independent assessment of the effectiveness of the actions that are being taken. This is something to develop further.”Mr Rabbachin says that further work is needed. “We are happy because in terms of transparency of political ads and closure of fake accounts they have done something and we can see what they have done. It remains more difficult for us and for independent assessors to concretely assess the impact of their actions.”

Lies: Still Thriving

Meanwhile several reports have come out showing that fake accounts still abound on Facebook with some saying that even when they are pointed out, Facebook doesn’t take action. It may be that Facebook is using debunking by fact-checkers to lower the rankings on the newsfeed but it’s not clear precisely how often.

In any case, Facebook acts only on debunking provided by the groups they partner with. That’s left groups on the outside wondering whether they should join the club. “Not being on Facebook means there are a lot of user conversations we are not aware of, conversations riddled with disinformation and disinformation. As fact checkers, it is essential that we broaden our remit to be in that space,” said Russell Skelton founder of the ABC/RMIT fact-checking organization in Melbourne. However, he notes that, for ABC, their media partner, it is a tricky space to be in. “The national broadcaster cannot be part of Facebook’s fact checking initiative while objectively reporting on developments within and around the largest social media company in the world. That presents a clear conflict,” Mr Skelton said.

But taking money from Facebook has implications, too. Groups funded by Facebook spend more time debunking images and videos on social media than on statements by public officials. This is having the effect of transforming the work of the fact-checking organizations and making them more like the groups that specialize in verification such as Bellingcat and Storyful.

“Their work has shifted over to the misleading online content Facebook is paying them to debunk, which is a lot of shitty memes and videos. There is a crisis in the field. We went into this because we care about truth and facts. So the money has shifted the field,” said one senior person in the fact-checking world who asked for anonymity due to close partnerships with Facebook.

So if it’s not clear how much fact-checking is changing public opinion or causing the platforms to down-rank false content what is fact-checking doing? In some countries, fact-checking is naming and shaming politicians and causing them to be more truthful.

During the 2019  Australian election campaign ABC/RMIT Fact Check had some notable success. The Prime Minister stopped saying emissions were going down, which had been a key part of the government’s campaign mantra, and the Opposition leader stopped conflating the number of traineeships and apprenticeships. There were other instances of where politicians stopped repeating incorrect claims. “We know from our analytics that our fact-checkers were extraordinarily popular and the average read times were well above the average for ABC News Online,” Mr Skelton said.

More Allies Needed

It may be that the major impact of fact-checking has not been the building of trust or the suppression of online content, that has been debunked and flagged, but the impact on the profession of journalism. Supporters of the Factchecking movement say that the international alliances and training that has taken place have trained journalists in how to be accurate and gotten them to collaborate often when they hadn’t previously.

Brazilian journalist Debora Ely was part of the Brazilian fact-checking alliance, Comprova, that brought 24 news organizations together for 12 weeks before the 2018 presidential elections. Facilitated by First Draft and funded by Facebook and Google, the collaborative effort included major dailies and local newspapers who monitored and checked rumors, reports on misinformation being shared across social networks, websites and private messaging apps. Members of the public sent in questions via WhatsApp and could then share the debunking results written up by Comprova. Jair Bolsonaro won and the attacks on the media left the Comprova partners deflated but Débora Ely remains determined. “I would do it again,” she said. “Despite everything because we don’t know yet the solution [to online mis/disinformation] and since we don’t know the solution we still have that feeling that something has to be done and this is the answer we have now.”

Truth squadding might have worked in 1980s America but Débora Ely and her compatriots seem ill matched against algorithms wielded across the globe that are daily more powerful at targeting information – often false – at the vulnerable. Fact-checking as a distillation and refinement of truth-seeking journalistic practice is undoubtedly a good, but alone it is hard to see these fighters for truth win the battle without additional allies, and armaments.

With thanks to Chloe Oldham, Barnard 2022 for her research help.

Anya Schiffrin is the director of the technology, media and communications specialization at Columbia University’s School of International and Public Affairs.

Photo by Joël de Vriend on Unsplash