Open Doors will review methodology for World Watch List
How do you measure persecution of Christians? Open Doors tries that but the method of that research is under constant questioning. On the photo: protest against persecution of Christians in Lyon, France, in July, 2014. Photo AFP, Romain Lafabregue
European Union
Open Doors International will review the methodology behind the international religious persecution ranking list.
The managing director of World Watch Research at Open Doors International says this in the Dutch newspaper Nederlands Dagblad. “The World Watch List is meant to meet scientific standards,” Frans Veerman says there. “We continue to learn and develop, which is why a process has been launched to re-examine the scientific value of the WWL methodology with new external experts.”
The WWL (World Watch List) was published on Wednesday for the 30th time. North Korea is back on top. Turkey and Belarus are the European countries on the list.
Nederlands Dagblad wrote no less than four articles about methodological questions and criticism of the research behind the annual ranking list. The paper concluded: the investigation is “thorough and transparent”, but the criteria and method are home-made and therefore vulnerable to criticism.
Putting the religious persecution of Christians in a qualitative ranking list is not easy. What weight should discrimination have, and is that the same as slander and insult? Does it matter whether a country has a large population or not? And can you prevent that “apples and pears” are together in one basket, the paper asks.
Jokes
The former diplomatic envoy for religion and world view of the Dutch Foreign Office, Jos Douma, sees it as a problem that “Open Doors measures everything, also the smallest things”, he says to ND. “But what is religious persecution? Does it include distrust from your neighbours, or jokes when you ask for silence before your meal? You have to be aware that you do not reach conclusions that do no justice to the matter.”
The basis of the list is the 84 points from the questionnaires. People on the grassroots level from the countries themselves fill these in. Around the questionnaires, there is a little reserve because the same question may be understood very differently in Afghanistan than in Colombia. “Ensuring that such differences do not exist (this is called measurement invariance) is one of the greatest challenges of this type of research”, statistics expert prof. Casper Albers (University of Groningen) says. ODI manager Veerman admits this, but he thinks the impact of it is at “a maximum of 2 points”.
The paper also quotes Joost Scheffers, a Western journalist living in Egypt. He watched the making WWL of 2016 and was “shocked” by the “inaccuracy and assumptions” during the research process. “It was really outrageous, how they put it together, what they based it on, how they mix data, experts who have never been to Egypt and are not involved in that country, citing examples from years earlier. Helping Christians is good, of course, but making up data and turning everything on is the other extreme.”
On the other hand, the Dutch Korea expert Remco Breuker (University of Leiden) thinks the ranking list is accurate. “We don’t know much about North Korea, but the persecution of Christians is an exception. We have more than enough testimonials from Christians to put North Korea high on these dire rankings.”
Samples
There is a simple mantra for Open Doors International: Research stands above ranking, the paper notes. The organisation has a more than 100 pages Methodology. But that is an internal document. “In a sense, it is judging your own meat”, the paper writes, “because both are your own baked goods.”
More value has the ‘audit’ from the IIRF (International Institute for Religious Freedom). That organisation checks the research and takes samples of countries. The IIRF is free in its comment, and ODI is also free to use the results from the audit. The IIRF analyst Christof Sauer thinks ODI’s method has improved since 2012.
Related Articles
