A more progressive Terrorism Studies

This I have resolved on … to run when I can, to go when I cannot run, and to creep when I cannot go.

Pilgrim’s Progress – Part 2 Chapter 6.

The role of the Feeble-mind character in Bunyan’s religious allegory Pilgrim’s Progress is to highlight the importance of continuing to make progress towards an identified goal.

A more progressive approach to Terrorism Studies could focus on extending the depth at which the Salafi-Jihadi movement is understood. This would be based on robust data science and human expertise, a focus on the primary language of the Salafi-Jihadi movement – Arabic – and the extensive archive of theologically inspired thought which the movement has produced.

It is the archive of theology as expressed and interpreted by the core of the movement which provides the best predictor of the actions of the movement. In contrast, some researchers and commentators within the orthodoxy of Terrorism Studies claim to see artefacts of their western habitus in Salafi-Jihadi material – the focus on crime, rap music, and naïve notions of a Jihadi ‘Utopia’.    

Since the 1980s research has shown that the study of terrorism has struggled with availability, handling and analysis of data. Despite the length of time and frequent observations about the problems with data, rather than making progress, these problems within orthodox Terrorism Studies have remained. In addition, “the dispersed nature of much of the more rigorous, ‘critical’ and conceptually innovative research on ‘terrorism’” means that level of rigor in research is conducted outside the orthodoxy.

Over a decade ago Magnus Ranstrop highlighted the ongoing problem, which Alex Schmid and Berto Jongman originally identified back in 1988; that ‘there are probably few areas in the social science literature in which so much is written on the basis of so little research’. As a result, much of the writing in Terrorism Studies is “impressionistic, superficial, and at the same time often also pretentious, venturing far-reaching generalisations on the basis of episodal evidence”.

Rüdiger Lohlker recently recently continued this theme when he highlighted, the tendency for orthodox Terrorism Studies to contain “an empty fog of words without inner content”. Quoting German philosopher G. W. F. Hegel. The section quoted by Rüdiger Lohlker continues:

This sort of chatter, though lacking the idea of philosophy, gains for itself a kind of authority through its very prolixity and arrogance. Partly this is because it seems almost incredible that such a big shell should be without a kernel…

 G. W. F. Hegel

Such is the coagulation of mediocrity within a section of orthodox Terrorism Studies that unscientific, methodologically flawed and statistically unsound commentary is talked up as ‘data science’ and ‘groundbreaking research’, while the evidentiary basis goes unquestioned within the mainstream scholarship and peer review. Over the last decade – rather than being addressed – these issues have become systemic.

Making progress

As noted in the development of Critical Terrorism Studies, “it is not enough to simply point out what is lacking in current research; a clear and realistic alternative must be provided”. Sections of the orthodox Terrorism Studies, and offshoots in the CVE industry, have an opportunity to break from the contemporary stagnation and develop a strong data culture and emphasis on evidence-based research. A progressive approach to analyzing the complex, theologically driven, Salafi-Jihadi movement, will move away from the contemporary obsession with finding so-called ‘gaps’ in the largely superficial and stagnant orthodox literature.

Many of the criticisms highlighted at the start of CTS remain within sections of the orthodoxy. CTS itself – with a focus on Western policy and Western academia – has struggled to break free from the Western frames of reference, among other challenges. The framing of Terrorism, with 9/11 as a moment of temporal rupture, still dominates CTS. This inhibits the deeper understanding of the Salafi-Jihadi movement which maintains different temporal reference points to those which dominate CTS and traditional approaches to Terrorism Studies, both in relation to time in the physical world and in relation to this world (Dunya) as an abode through which a soul passes.

To address the superficial, orientalist and neocolonialist tendencies of the orthodoxy, and the temporal framing of CTS, a progressive movement within Terrorism Studies would probe the intended meaning of Salafi-Jihadi content in their understanding of the world, rather than in a Western-centric English language dominated habitus. Critique of the orthodox Terrorism Studies has highlighted “its poor methods and theories, its state centricity, its problem-solving orientation and its institutional and intellectual links to state security projects”. Many of these problems have also concerned scholars within terrorism studies “who have long acknowledged the deficiencies and limitations of current research”.

In addition, a progressive Terrorism Studies approach would uphold standards for the appropriate use of statistical data to produce a clear break from the systemic malaise in data handling which have existed within orthodox Terrorism Studies. With a strong data culture and robust research design, a dynamic approach to Terrorism Studies could utilise the changes in the technological environment for research. This is not dissimilar from the way Salafi-Jihadi groups have adapted to the opportunities which evolving technology has created.

The way the object of study, such as the groups who make up the Salafi-Jihadi movement, choose to operate has evolved:

• The evolving concept of the electronic ribat. Since 2011, members of jihadist forums have issued media strategies that encourage the development of a media mujahideen. This encouragement has been accompanied by the release of guides to using social media platforms, which often include lists of recommended accounts to follow. 
• By 2013, Jihadists had aggressively expanded their use of Twitter, in addition to Facebook and YouTube, especially since the outbreak of violence in Syria. This propagation effort by the “media mujahideen” was approved and sanctioned by movement leaders and contributed to the interconnected jihadist zeitgeist. 
• Learning from each interaction on the electronic ribat, the media mujahidin rapidly developed to maintain a persistent presence based on the speed, agility, and resilience of the Swarmcast.
• In this struggle for survival, the media mujahidin have benefited from collective approaches and emergent behaviors, these have allowed a decentralised network to thrive in the face of increasingly aggressive content removal.
• The media mujahidin frequently use widely available software for media production – this software would also be easily accessible to researchers to provide a window into the production methods.

In addition the current technological environment provides many opportunities for research:

• Servers are cheap and easy to access – for example if you use Amazon for shopping, then that is enough to access cloud computing through AWS.
• Processing power and RAM are cheaper than they have ever been, allowing relatively complex calculations and data analysis to be produced rapidly.
• Most modern personal laptops and desktops have hardware sufficient to run the analysis required for many data science projects which would extend current research into the use of the internet by Salafi Jihadi groups, or ‘extremist’ groups more broadly. Of note, most contemporary material published by salafi Jihadi groups is produced on the same widely available hardware / software (more on that in a later post).
• While there are many commercially available and hugely powerful ‘data systems’ which integrate a range of data storage and analytical processes within a single platform, there are also many open source programs which can be used to conduct academic research. These open sources software options may not permit all the analysis to be conducted within one platform, requiring the researcher to use a range of approaches to achieve the desired analysis.   
• There are many ‘how to’ guides for those aspiring to become better at using python, java, or any of the other popular coding languages. Similarly, resources are freely available which researchers can use to learn more about data science or using specific open source software more generally in their work, whether in the form of walkthroughs or articles packed with quick tips and tricks.

To build a stronger data culture will mean;

• Acknowledging the problems with evidence and data which have to date beset orthodox Terrorism Studies, 
• Reviewers and editors robustly enforcing actual standards for statistical analysis, for example,
  • if you are going to claim something such as a correlation or a long-term trend – it will need to be backed up by a statistical calculation using data acquired through a scientifically appropriate method.  
  • if the analysis is based on subjective ‘coding’ of data – is there an appropriate intercoder reliability score. If there is not, there is little reason for readers to be confident that the research presented would be repeatable, that coding remains unchanged over time, and that other researchers would apply the same coding definitions in the same way. Without intercoder reliability there is little reason to have confidence in the resulting ‘analysis’ rendering it largely unpublishable.  
• Editors and publishers insisting on clear conflict of interest reporting,
• The Terrorism Studies community putting methodology above attention grabbing headlines and tweetable pseudo-metrics.
  • If the methodology is flawed or the statistical analysis unsound – no matter if the ‘findings’ are appealing or even intuitively correct – the study lacks the necessary basic elements to be considered publishable research.  
    • Sample:
      • If research is claiming to have analysed a sample, to what extent can the sample be considered representative of the whole?
      • Was that sample derived from a consistent methodology, or a hodgepodge of pieces cobbled together?
      • How was the sample identified and collected? In effect research design (architecture) and data collection (acquisition) to use two of Jeffrey Stanton’s four A’s of data science.   
  • If you cannot do the calculation to produce a statistical result, do not use the word related to that calculation e.g. correlation, trend etc.  
    • Correlation:
      • When an author claims correlation – a range of questions should spring to mind; do you mean a positive or negative correlation?
      • how strong a correlation?
      • Did you use Pearson, Spearman, or Kendall?
    • Trend:
      • If a trend / trendline is claimed, what is the R-squared value? Is your line a good representation of the data?
      • A trend requires more than two or three data points.
      • Do you mean one point in time has fewer of ‘x’ than another? This is not a trend, upward or downward, one point just has fewer than the other.

Based on what is currently being published within the orthodoxy of Terrorism Studies there are a range of issues, including;

• Journals specifically focused on terrorism research, a range of journals in related disciplines which have hurried to do ‘special issues’ on ISIS, and ‘research centers’ self-publishing special reports, which have published articles as if they are either unaware of the basic scientific and statistical standards or are content to publish material that they know falls short of the minimum acceptable scientific standards.
• Senior researchers, including Professors, who will cite work that falls short of scientific or statistical standards without commentary or critique, some even talking it up as ‘ground breaking’ or ‘data science’.  

The current state of orthodox Terrorism Studies must be judged on the behaviour of those in the discipline. Such is the coagulation of mediocrity in orthodox Terrorism Studies, senior researchers have not questioned unsound methodologies, and journals through their peer reviewers and editors, have not upheld standards. The previously observed problems of data and data analysis within some sections of orthodox Terrorism Studies have now reached systemic levels.

In a scientific discipline,

• If the relevant scientific or methodological information is not present in an article submitted to a journal, then that paper is going in the bin because it does not reach minimum standards for undergraduate level work, let alone peer review.
• When individuals deliver presentations, which make statistical claims about trends or correlation without any calculations, or use substandard / misleading data visualisation to support their argument, they could expect to be laughed out of the building.  

If statistical and data analysis in Terrorism Studies do not adopt the standards adopted by other fields, it cannot take full advantage of the potential offered by increasing integration of data science or forms of statistical analysis into the study of Salafi-Jihadi groups.

The following series of posts examines specific tangible reasons why robust data science and evidence-based analysis is important and offers a critique of contemporary uses of data within orthodox Terrorism Studies.  

• A progressive focus on what events and material means to Salafi-Jihadi groups – Da’wa As Constant on the Path of Jihad:

The purpose of research is to develop deeper understand the object of study. While Western-centric interpretations of ‘utopia’ and claims the ‘West is Winning‘ or that Salafi-Jihadi groups are defeated may be easier to produce and more comfortable for policymakers to read, they do not capture what the Salafi-Jihadi movement means or believes. The message that ISIS is defeated, may be politically expedient when tweeted by Donald Trump and echoed by researchers including the authors of the ISIS Reader. Yet, while it is possible to produce a definition of ‘defeat’ to back such a claim, that definition is unlikely to be a useful indicator of the current state nor future behaviour of the group. Far from defeated al-Dawlat al-Islamiyya, as a fighting force, “is bigger now than it was nearly six years ago”, according to Kurdish forces – a claim supported by a CIA assessment. UN Under-Secretary General, Vladimir Voronkov, has suggested that the number is even higher, some 27,000 Daesh fighters in Syria and Iraq, with up to a 100,000 civilians having some level of dependency on the group.

In addition to expedient notions of ‘defeat’, the focus on concepts such  as ‘Utopia’ are artefacts which result from Western researchers’ tendency to view material through their Western-centric lens. A progressive approach would focus on understanding what the intended audience understands by the material such groups produce, and be able to quote prior Salafi-Jihadi material to back that interpretation. In effect, progressive Terrorism Studies would focus on reading the lips of the Salafi-Jihadi movement, as Reuven Paz suggested over a decade ago.

• Progressive commitment to scientific methods to escape the impressionistic and statistically unsound representations of data in orthodox Terrorism Studies:

The failure to uphold statistical methods in orthodox Terrorism Studies has become systemic. Methodologically flawed, statistically unsound, unpublishable garbage is now talked up within the orthodoxy of Terrorism studies as ‘data science’ by professors and published in journals or circulated as ‘special reports’. This section provides a critique of the unscientific approaches to data and statistics considered acceptable within orthodox Terrorism Studies. The adoption of a progressive approach to Terrorism Studies would demand a clear break from this flawed research, putting robust methodology above tweetable headlines. 

• Orthodoxy claims decline – time for a reality check:

While production of media content by IS has fluctuated, some commentators have sought to coerce the data into a linear direction – a so-called decline. This section examines how the narrative has been constructed and shows that committing to the decline narrative has meant overlooking some serious methodological flaws and fluctuation in content. The decline narrative was built by shifting the goalposts both in terms of definition and time-periods rather than robust statistical analysis. In fact, while some claimed consensus around the decline narrative – a robust statistical analysis reveals average weekly video output increased in both quantity and longevity of production between 2017 and 2019.

• Neo-Colonialist tendency to devalue ideas in Arabic:

This section unpacks some of the methods widely accepted within orthodox Terrorism Studies to show how they devalue material in Arabic in favour of English language sources and Western-Centric interpretations. A progressive Terrorism studies would focus on the primary language of the Salafi-Jihadi movement (Arabic).

• Decline narrative as strategic communication tool:

The so-called decline has been more than a narrative deployed in commentary; it has also been used as a strategic communication tool. This section highlights the need for genuine scientific methodologies, appropriate statistical analyses, and robust conflict of interest reporting to ensure the field can escape the current coagulation of mediocrity and rebuild confidence in the academic output.

• Progressive commitment to robust statistical analysis; the end to Mc_Data:

Scientific methodologies and robust statistical approaches can lift orthodox Terrorism Studies out of the current malaise of mediocrity, and enable the field to embrace the opportunity available through evidence-based research and a stronger data culture.  

A progressive Terrorism studies, using robust data science and evidence-based analysis, is important because contrary to the dominant narratives of IS having collapsed or been in terminal decline, al-Dawlat al-Islamiyya (IS) used the time on the open front in Iraq and Syria as an educational opportunity, to build a base of supporters running into the thousands. This is why al-Dawlat al-Islamiyya, as a fighting force, “is bigger now than it was nearly six years ago”.

As such, the theology which groups such as al-Dawlat al-Islamiyya sought to promote, will remain, will endure and, when given the opportunity of fertile soil, it will expand visibly when it suits their strategic cause.

Dawn is coming.