Measuring Islamophobia on Twitter

Yesterday we released new research measuring the scale and scope of Islamophobia (keep reading… we’ll return to that) on Twitter. It was part of an attempt to get some kind of a handle on how much is happening on social media; when it happens, and who it happens to.

The research was reported widely across the BBC and elsewhere, and has received a lot of reaction on social media. Some of it, predictably, was very nasty. None of that needs a response; it was, in a darkly eloquent way, making our point for us.

However, other criticism was sane and reasoned. People wondered what definition of Islamophobia we used, whether it was the right one, and how it was applied to analyse Tweets. They worried that in using it, we confused and conflated criticism of an idea (Islam) with abuse of people (Muslims), and that words like Islamophobia are used to shut down any kind of criticism of Islam at all. These are all, of course, legitimate concerns. The lengthiest came from the National Secular Society (you can read it here), and was written by Benjamin Jones, which I’ll use as a jumping-off point to deal with the main points of criticism raised.

At Demos we want to respond to fair criticism because it’s good for the research we do, and part of healthy public debate. So here it is.


Some criticism focussed on a single word that we used in the report: Islamophobia. The objection with our use of Islamophobia as the centrepiece concept for the research was two-fold: first that we didn’t define the term itself (and it was therefore unclear what we were counting), and second that the concept itself was the wrong one to use. I’ll deal with each in turn.

First, definitions.

With respect to Benjamin and others, I don’t think it’s quite fair to say that we “have totally failed to define what ‘Islamophobia’ actually means”. The way that we do social media research is to look at the data that we collect, and inductively generate the categories and analytical frames based on what we find. At least on Twitter and at least within the data that we collected, we said that we’d found four categorical kinds of Islamophobia. This was all included in the section called ‘defining Islamophobia’ within the report. We might have been clearer here, (and there was a box within the flow-chart showing the algorithmic architecture that certainly could have been clearer), so let me be so now.

The first two categorise of Islamophobia were, I think, uncontroversial:

  • The first was hateful abuse directed towards Muslims themselves, and especially those using derogatory slurs and descriptors;
  • The second was abuse that conflated Muslims generally (as people) with sexual violence

Here are some more examples of both. I’ve changed them slightly to stop the senders being retrospectively identified. The meaning is unchanged.

@xxx go eat curry you bastardised pack

It is very very hard to get lower than muslim scum, somehow the leftys managed to do it – <LINK>

@XXX @XXX fuck ur sand nigger parents #hack

How did that paki cunt end up mayor?

@xxx we have sand nigger and spics here — You guys all stick together – Animals stick together

Trump: My temporary ban on Ragheads entering the U.S. is a “suggestion” – <LINK> (NO, DO NOT BACK DOWN FROM THIS)

@XXX @XXX @XXX @XXX @XXX u will see when a muzzie rapes your daughter or wife u spineless weasel

American Women! How to deter male Muslim attack. In ur purse carry a spray bottle Pigs blood! Announce its pigs blood! Also draw ur Glock!

It was the final two categories that, I think, some found particularly problematic. These were:

  • A category we called ‘Islam is the enemy’, which contained claims (quoting the original paper) that ‘it is a fundamental injunction of Islam for all of its followers to be engaged in a violent struggle against non-Muslims and the west’;
  • and (similar) especially in the wake of a terrorist attack, the apportioning of blame for the attack not on the terrorists themselves, or on Islamist militancy, but on the Muslim population generally.

Examples here include:

@xxx @xxx Islam allows you to do anything to infidels. In fact it encouraged jihad to pay for your sins

1.5 billion Muslims Running 50 countries. They only do jihad terror becs they are marginalised. Give them another country.

@xxx There are 6 million muzzies in France. If only 10% of them are animals (we know its more) That’s 600,000 savage muzzie killers.

I feel sorry for the true people of London the Muslim take over has begun we must not let it go any further dame scum @XXX

Send these fucking muzzie scumbags back to sandholes they belong it and then bury them 4 good, those murdering pricks!

There are no moderate Muslims – Every Muslim plays a role in civilizational jihad from infiltrating government to crashing welfare programs.

There are no peaceful Muslims: there are only ones that have not been called to jihad yet

The objection with these final two categories was that they reflected how the term Islamophobia itself is muddled and unclear. Here, the worry was that we were treating the criticism of ideas (of Islam) the same as nasty abuse directed towards Muslims, and that, in doing so, closing down’s the ability of people to criticise Islam at all. To quote Benjamin:

“In their report Demos selects some tweets it included in the study, which they presumably think are good examples of their methodology in action. A tweet stating “Morocco deletes a whole section of the Koran from school curriculum as it’s full of jihad incitement and violence The Religion of peace” is treated the same way as a tweet saying “I fucking hate pakis” in their methodology.” “One of these tweets criticises an idea. The other is racist. One describes and mocks a belief system, the other (verbally) attacks people. Demos’ methodology treats both of these tweets in the same way.”

The term Islamophobia itself began to gain currency in the mid 1990s, thanks in no small part to an influential paper by the Runnymede Trust called Islamophobia: A Challenge for Us All (the executive summary is here). Whilst they did not coin the term, the Commission that authored the report did go some way to codifying and defining it. Chaired by Professor Gordon Conway, the Commission involved academics, religious leaders, journalists and so on. The Runnymede Report itself admitted the term is ‘not, admittedly, ideal’. They worried, too, that it would be used to demonise and stifle legitimate criticism of Islam. For this reason, they took pains to define the distinction between how a phobia – dread – of Islam was different from a legitimate criticism of Islamic beliefs. For our research here, the most important distinctions the Commission made between an Islamophobic expression from a legitimate criticism of Islam are:

  • Seeing Islam as a monolith. This is the view that Islam is undifferentiated and static; the use of sweeping generalisations that ignore the tensions, disagreements, debates and profound differences that exist within community (or rather group of communities). This is especially important for the research that we did, because one of the kinds of Islamophobia we identified was moving from the general to the particular, where a terrorist attack is seen as either an illustrative example, or proof, to condemn all Muslims generally (under the assumption that they’re all the same).
  • See Islam, as a faith, as an aggressive enemy. This would constitute not the view that opposes theological militancy, Islamism, or particular brands of Islamism, or ideologues within particular brands of Islamism. However, it would include the worldview that Islam, writ-large and across the board, regardless of the different interpretations and traditions it contains, is inherently committed to terrorism, violence, and an implacable hostility to the Western world. Expressions included here were those that claimed that Muslims supported terrorism and terrorists not because of anything that they had said or done, supported or condemned, but because they were Muslim.

Making a distinction between an Islamophobic criticism of Islam and a non-Islamophobic one is really important. To be clear, in our research a criticism of Islam was not, by definition, taken to be Islamophobic, and was not understood by the researchers of our report to be. However, one that treated Islam as a single, hermetically sealed-off block, and one that, en bloc, is violently hostile to the West would be.

Finally, what about the mixing together of the criticism of ideas with the criticism of people? First, this is what concepts do; they bring together things that can be disparate in order to make sense of them. Putting things into the same conceptual bracket doesn’t mean putting those things on the same moral level. There’s obviously a big moral gulf between a racist Tweet, and a racist murder, yet both fit within the concept of racism. For exactly this reason, we weren’t trying to draw any kind of moral equivalence between the different kinds of things that we analysed. How could we? The research used structures of algorithms to filter tens of millions of Tweets into hundreds of thousands. Within this data, there will be Tweets of radically different kinds of seriousness and concern. We weren’t trying to say that all Islamophobia is equal, and that all these Tweets have some kind of equal moral standing. Nor did we actually – anywhere – say that.


I accept that there will be differences of opinion over whether some of the quotes used here are truly Islamophobic. A point made in the original paper, and worth making again, is that interpretation is involved here. But that isn’t a special feature of this particular research – this is something that is done across social science. The act of analysing a Tweet, of putting what someone says into a category, is necessarily an act of interpretation. It’s possible for there to be more than one legitimate interpretation. Could someone have interpreted individual Tweets differently? Absolutely. Our job is to be as transparent as we can be about how we were going about making the interpretations that we did.

That’s not to say that the research is completely subjective. Social science has developed ways of managing and mitigating researcher bias, and there’s several that we used here. The first is that we get more than one researcher to blind code (so they aren’t looking at the other researcher’s coding) a random sample of Tweets into those categories, and measure the extent that they agree or disagree. Another is that we try to get a wide range of researchers from a varied degree of background to do the coding. We did both for this research, but there’s no getting around the fact that other people’s interpretations may be genuinely different. This is all part and parcel of social science.

Freedom of Speech

The crux it comes down to is this. Many of the critics, I think, just don’t like Islamophobia as a concept at all. I accept that Islamophobia, as a concept, is a controversial one, and I accept that many may not like the definition it has. That doesn’t mean, however, that the definition that people don’t like can’t be robustly researched. Nor is it a kook, fringe term. The United Nations held a Conference on confronting Islamophobia, it is used by third sector organisations around the world, and it is of widespread use within peer reviewed scholarship, including the study of Islamophobia online (here, here, here, here, here, here are all examples, and I could point to hundreds of others).

To me, it isn’t the lack of some kind of workable definition that is the problem. It is Benjamin’s point about how it is used as a concept to shut down criticism and end the debate; often exactly in the moments when the debate is what we need the most. This is a real danger, and here I completely agree with Benjamin and others when they make that point.

“The National Secular Society was instrumental in abolishing the vestigial blasphemy law in this country, but now I fear that our culture is returning to the legal protection of ideas, and Islam specifically.”

Whilst I share his worry, I also can’t see how this criticism can be tied in any way with our report at all. Here it’s worth restating two parts of our paper in full.

First, the intention of the research: “There is no suggestion of any illegality of any of the content measured: the purpose of the research was not to look for content that was illegal, and it does not suggest that the content that was found was illegal. This research is not seeking to inform how laws should be enforced on social media. This research, and Demos’ broader research agenda, seeks instead to inform the broader question of how people from different races, religions, sexualities and genders are spoken about on social media, and the extent that people from different backgrounds face abuse and hostility.”

Second: “At Demos we believe it is important that the principle of internet freedom should be maintained; and that it should be a place where people feel they can speak their mind openly and freely. However, racist, xenophobic, Islamophobic and misogynistic abuse can curtail freedom, and the capacity to speak and act freely online, as much as it can be an expression of it. It is important, as society confronts the ways that social media acts as a new platform for the expression and dissemination of these of kinds of views, to understand as best as possible the scale, scope, nature and severity of these kinds of practices: when they happen, who they happen to, and why. This is what this research hopes to contribute to.”

There is nothing in the paper or the research about a blasphemy law, or illegalising Islamophobic expressions generally. We did not argue and do not think blasphemy laws should be introduced. No religion is above criticism.

Certain ways of talking about Islam – as a hermetically sealed-off monolith intent on destroying the West – are free expressions of speech, but I am worried that they can also curtail it for other people. Being on the receiving end of abuse may cause Muslims not to speak up or out, or to drop off social media altogether. For this reason, I believe it’s important we know how much of this is going on, and we know what the nature of it is. In some cases it is important to confront it, but not by making illegal what is said. This is about confronting one idea with another – with allowing Islamophobic views to be aired, and then opposing them. In my point of view, Islamophobia is real. It exists, it is a problem, it is not a crime, and it’s well worth researching, despite all the difficulties and complications that it entails.