on the use and abuse of statistics, Instrumentalizing Minorities edition

on the use and abuse of statistics, Instrumentalizing Minorities edition

This is what happens, Larry. This is what happens when you present data from one question from a two-year old survey in a sexy shaded map and let the Twitters have at it.

The tweet and article in question from Pew Research Global, here.

This sexy map and the corresponding article (which I’m sure considerably fewer people actually read) have got some rave and not-so-rave reviews on the one social network that rots my brain more than any other:

  • Ukraine Twitter? Happy. These numbers (apparently) prove that “Ukrainians are the least anti-Semitic [people] in eastern Europe,” the “most accepting of Jews,” and “more tolerant to minorities than any of its neighbours.”
  • The national news agency of Azerbaijan? Happy. Armenia, apparently, is “the most anti-Semitic country in central and eastern Europe,” they tweeted Thursday morning.
  • Armenian/Armenian-watcher Twitter? Unhappy, if not confused, with some suggesting the high number for Armenia could be rooted in lack of Israeli recognition of the Armenian Genocide and/or Israeli supply of weapons to Azerbaijan.
  • Polish nationalist Twitter? Unhappy. “Dirty manipulation,” since western European countries like France weren’t surveyed, where a Holocaust survivor was murdered last week in what French police are calling an anti-Semitic crime.
  • Islamophobes? Unhappy, because apparently all anti-Semitism is the result of Muslims and these numbers don’t jive with that theory.

This map – but, honestly, more the fevered reaction to it over the last day on social media – has got me pretty unhappy too, but for some different reasons

First (and relatively minor, in fairness) lot of people are suggesting this data’s new. It’s not. This report was released almost a year ago, based on surveys Pew did in central and eastern Europe in 2015 and 2016 on religious belief and national belonging. I still have a copy of this report on my mobile from last year.

Secondly, I’m not a fan of the question(s) and the way they’re presented in the survey.

This is the exact question that was asked. Identical questions were asked about Muslims, Catholics, Orthodox Christians and Roma, in that order.UntitledI don’t like this question. Yes, I understand what it’s basically drawn from (Bogardus social distance scale questions), but look at it in the context of the preceding survey questions and it’s just….weird. (Page 18, here).

The preceding questions are all about religious observance – here’s the three that precede Q59.

So we jump from these right into a question that’s basically HEY WHAT DO YOU THINK ABOUT JEWS

There’s no seguing into or introduction of the battery of questions on Jews, Muslims, Catholics, Orthodox Christians and Roma; we move from “what do you think about your religion?” to “so tell me what you think of these minorities and/or people different from you.” It’s an odd jump, in my opinion, and one where more than one respondent must have wondered what the hell was going on.

Worse, though, is the question itself. Again, I get that these are somewhat standard questions – I’ve seen similar ones in other unrelated surveys – but, really, does “Would you be willing to accept Jews as: a. Members of your family, b. Neighbors, c. Citizens of our country” even make sense to an average respondent??

It makes me ask far, far too many questions about whether this question, and the data from it, is even reliable. How has this question been translated into the many languages used in actually administering the survey? Does it make any sense in them? Does it make less/more sense in some languages? Or, worst, does it mean different things to different respondents in different languages? If I was at the table discussing this survey in its design phases I’d have been asking a lot of questions about all this.

But what bothers me the most here is how these findings – or, more accurately, findings from *one question* on a two to three-year old survey – are being exploited.

As I’ve written about before, there’s a healthy crew of people out there who are more than happy to use Jews (or at least imagined representations of Jews) as a means to score political points (e.g., ‘look how well we treat our minorities,’ not like [insert enemy country here or Muslims in general]). Worse, though, are the ones who use The Jews In The Way That We Imagine Them as a cudgel to bash other minorities in their own countries (read: Muslims and Roma).

This is hypocritical, creepy and, if you’re a member of a majority and you’re doing this, what you are doing is bad and you should feel bad.

The story behind that “terror attacks” map you keep seeing

The story behind that “terror attacks” map you keep seeing

You’ve seen this map somewhere on social media the last few weeks, haven’t you?

Here it is from Poland’s deputy justice minister, because somehow this is how you show solidarity with the citizens of a country millions of your own people live and work in.

2017-06-04 18.10.49

I’ve also seen it from Polish MEP and unsuccessful candidate for the presidency of the European Council Jacek Saryusz-Wolski, though my favourite version is the one tweeted out by that one guy who managed to get fired from The Rebel.

OOOOH this one has different colours!!

And last, a version I saw on Instagram this week.


So where the hell is this data even from? Turns out, as someone from the right-wing Polish Twitterati told me, it’s data from the reputable Global Terrorism Database (GTD) at the University of Maryland, which is an “open-source database including information on terrorist events around the world from 1970 through 2015.” (Notice right now that says “events,” not “attacks.” This will be important). I was further informed that, for some reason, the data on this map that’s been making the rounds is only from 2001 on. OK.

So I took a look through the GTD data on some of the countries (including Poland) on this map.  There are certainly no terror “incidents” (read, “incidents”) listed in Poland from 2001 on. OK, so that seems (seems) accurate.

But what about terror in other countries? I’m particularly interested in these apparent incidents in Iceland, which shows up in some versions of the map and, having been there, doesn’t exactly strike me as a terror hotbed.

Since 2001, there have apparently been two terror incidents in Iceland that explain the two Icelandic dots on the map:

  • In 2012, “An explosive device detonated near government offices in Reykjavik city, Reykjavik North Constituency, Iceland. The explosive device was partially detonated by a robot meant to deactivate it. No group claimed responsibility for the incident.” Property damage was listed as unknown.
  • In 2014, “Assailants attempted to set a Lutheran Church on fire in Akureyri city, Northeast constituency, Iceland. No one was injured in the attack; however, the building was damaged. No group claimed responsibility for the incident.” Property damage is listed as “minor.”

No one was killed or injured in these two incidents.

Again, “incidents” is the key word. These two big red Icelandic points, and many others on the map, don’t represent terror attacks at allMany of them, including these two in Iceland, represent vague criminal acts that may not actually have anything to do with terrorism (let alone jihadist terrorism), that have barely caused any property damage and, more importantly, haven’t killed or injured anyone.

Why no Polish incidents in the GTD since 2001? Surely there’s been at least one shitty attempt at something like a pipe bomb in a car that never went off (there was one in the Czech Republic database, as I discovered) that would merit a mention in this database, though presumably this will make it into 2017’s list for Poland, given the criteria for inclusion.

So the next time you see this map, you’ve got a few options. If it’s got no legend or title, you can always tell whoever shared it that the points represent vague definitions of criminal acts that don’t always seem to be reported consistently. If it says something about “terror attacks,” tell them they’re completely, 100% wrong, and tell them there’s more than enough data on the GTD website for them to make a proper map of actual terror attacks that isn’t just a cute meme for people who don’t like Muslims.

What do Ukrainians think about the EU?

What do Ukrainians think about the EU?

When people give me access to lovely nationally representative data sets, I am a happy man.

I got my hands on the data set from the most recent Omnibus Survey run by the Kiev International Institute of Sociology. Every three months KIIS surveys 2,000 Ukrainians about everything from who they’d vote for to what kind of presents they’d like to get for New Year’s. The most recent data is from February 5-16, and fieldwork for the next round is beginning in just a few days.

One of the first questions they asked was about attitudes towards the EU (KIIS’s translation from the Ukrainian and Russian versions of the questionnaire):

Please, imagine, that now is a referendum on whether Ukraine should join the European Union. You can vote for, against or abstain from voting. What would you choose?”

Just under half (49%) of Ukrainians said they’d vote in favour of EU accession, and 28% would vote no. A few (9%) said they’d abstain, but about one in seven (14%) said they didn’t know how they’d vote – a figure I’ll talk more about below.

UA and EU accession

Not surprisingly, there are some pretty stark regional differences, with more western Ukrainians (77%) inclined to support EU accession than anyone else, particularly eastern Ukrainians (25%).

EU UA region

I’ve tried to unpack those figures from southern and eastern Ukraine, to the extent that the sample size allows me to make confident comparisons – and for some of the larger oblasts, it does:

  • Donetsk oblast (including DNR): 57% of those surveyed said there weren’t in favour of EU accession; only 19% said yes
    • Just in the DNR, 74% said no, and only 6% said yes
  • Luhansk oblast (not including LNR): 56% said no; only 23% said yes
  • Odessa oblast: 38% said no; only 26% said yes and, interestingly, 21% said they didn’t know how they’d vote
  • Kharkiv oblast: 28% said no; 37% said yes

A few other differences of note:

  • Women were more likely than men to say they didn’t know how they’d vote (17% of women compared to 11% of men)
  • Younger people (between the ages of 18-29 and 30-39) were more likely to say they’d vote yes (57% and 58%, respectively) than those aged 60-69 (39%)
  • People with the lowest levels of education (less than ten years of education) were least likely to vote yes (23%), but at the same time most likely to say they didn’t know how they’d vote (31%)

Do these differences matter?

Well, some matter more than others.

I ran a few logistic regression analyses, because a) this is something I know how to do despite never really ever wanting to, and b) it’s a good way to parse out what matters most in yes/no survey questions.

What did I find?

Yes: Living in western Ukraine was by far the strongest predictor of voting Yes. This is probably the least surprising finding I’ve ever found running one of these things and I assume no one even remotely connected to the Ukraine-watching universe is shocked by this. The other differences I mentioned above (age, etc.) come up insignificant in the regression analysis.

No: Living in southern and eastern Ukraine were strong predictors of voting No. Interestingly, living in southern Ukraine was a stronger predictor than living in eastern Ukraine – I’m curious to see whether this holds in up in future surveys. Also, self-identification as Russian or ‘mostly Russian’, as well as being a member of the Moscow Patriarchate, predicted voting No. Not particularly shocking.

Don’t know: The only predictor of saying Don’t Know was being a woman.

What does this all mean?

Most of this is obvious and confirms what we all already know, but what I’m intrigued by are the relatively high numbers of Don’t Knows in the overall figures (14%).

I’m curious about the high Don’t Knows in Odessa oblast (21%), and whether this speaks to something below the surface there or whether it’s just something that’ll fall out in the wash in future surveys.

What I’m most curious about though is so many of the Don’t Knows being women – almost one in six women don’t know how they’d vote, compared to just over one in ten men.

Are they less willing to give a yes/no answer, (i.e., is there some sort of gender-related social desirability bias here?) or do they genuinely not know how they’d vote? If so, why?

And, most importantly, what would convince them to say Yes or No?

The dividing line at Paris’s square of unity

The dividing line at Paris’s square of unity

The statue of Marianne, the national symbol of the French Republic, towers more than a hundred feet above Paris’s sprawling Place de la République, an olive branch in her hand.

On the evening of the anniversary of the Charlie Hebdo attacks, the square is bustling. Locals and tourists circle around Marianne in an almost processional way while a Joe Strummer lookalike stands up on the plinth with a guitar and microphone.

Marianne herself is bathed in the bright lights of the news crews here to cover the sombre anniversary, but the tents of almost two hundred refugees right behind them remain barely lit. Continue reading “The dividing line at Paris’s square of unity”