Monitor & Evaluate
Monitoring your campaign's impact indicators will help you track its progress. Any adjustments you make to the campaign content or tactics, based on the monitoring, should be recorded and explained. The knowledge gained from this iterative process can help to maintain the progression of your campaign and inform future work you or other campaigners may carry out. It will also be valuable for potential funders to see that you have a built-in process for feeding ongoing campaign insights back into the delivery.
Your objectives are the targets you will evaluate your campaign against once it has concluded. If you have successfully achieved your objectives, you should be able to illustrate this by explaining how the tactics you used were successful (based on the ongoing monitoring of impact indicators), and how these tactics then fed upwards, logically, to your objectives.
Avoid vanity metrics and keep your integrity! Some campaigns present metrics of success that are not true reflections of impact. This can either be due to a lack of understanding of what constitutes impact or a deliberate attempt to make the campaign look more successful than it was. There are many examples of organizations claiming to have reached millions of people with their campaigns, yet failing to prove whether or not these people were the target audience and failing to show any evidence of actual engagement or impact. It is so important to be honest and accurate in your impact evaluation and presentation.
Please Note:Many campaigns, both online and offline, fail to recognise the importance of these two stages. As such, there remains a gap in understanding as to the most successful ways of influencing the behaviour of certain audiences. The measurement of your campaign does not have to be as complex or arduous as might be assumed, and even simple monitoring and evaluation efforts can produce useful and meaningful insights
Why do we need to Monitor & Evaluate?
- While many organizations around the world are carrying out campaigns to respond to challenges created by terrorist and violent extremist groups, those applying effective monitoring and evaluation practices remains limited. Unfortunately, many campaigners do not evaluate their campaigns or conduct a limited and surface-level evaluation after their campaign, as opposed to considering it throughout the design and delivery phases
- There are many factors that can deter campaigners from undertaking effective evaluations, from tight delivery timeframes and a lack of evaluation expertise or confidence, to insufficient public or private sector support or funding.
- The lack of effective monitoring and evaluation means there is limited knowledge about the effectiveness of many campaigns, especially within those that counter violent extremist messaging and narratives. It also means that many powerful campaigns do not always receive the necessary and deserved long-term funding or support.
- It is important to plan how you will monitor and evaluate your campaign at the outset, and not wait until after the campaign is finished. This will ensure that your data collection and reporting remain measurable, adaptive, and tied to campaign objectives. Before launching the campaign, identify what monitoring and evaluation data will need to be collected, how often, who will collect it, what tools will be used, and in what format insights will be shared and reported. This will also put you in the strongest of positions when applying for funding from governments, foundations, or the private sector.
Activists engaging in online campaigns cannot disregard the possible privacy and security implications of online monitoring and evaluation. Although it is important to assess a campaign's effectiveness, monitoring and evaluation tools might pose risks to personal information.
Privacy concerns arise when personal information - such as names, contact details, or online activities - is collected, stored, or analyzed without appropriate consent or safeguards. Activists may face risks of exposure, harassment, or surveillance by unauthorized actors or authorities who gain access to monitored data. Additionally, the aggregation and analysis of data collected through monitoring can potentially lead to profiling, discrimination, or the infringement of individual rights. Beyond the different legal frameworks on data protection applicable to each jurisdiction, it is important to note that activists should consider adopting a privacy-preserving approach whenever possible.
- Handle personal information carefully . Personally Identifiable Information (PII) refers to any data that can be used to identify an individual, such as a name, email address, phone number, IP address, etc. Treat any information that could reveal someone's identity as highly sensitive.
- Collect only what you need. Gather the minimum amount of information required to measure your campaign's objectives, and avoid collecting unnecessary personal data.
- Be transparent and obtain consent. When gathering feedback or survey responses, clearly explain what data is being collected, how it will be used, and for how long it will be stored.
- Store and share data securely. Use password protection, encryption, and multi-factor authentication for internal systems that store sensitive campaign data. Limit access only to staff who need the data for monitoring or evaluation.
- Set a deletion plan. Establish clear timelines for deleting or archiving campaign data when it is no longer needed, i.e., a certain period of time after the campaign is completed and all reporting has been submitted.
- Use social media analytics to understand the audiences being reached by targeted advertising.
- A qualitative analysis of the comments and messages you receive.
- Pre- and post-event surveys for participants at an interactive, networking event.
As your campaign is being delivered, continued monitoring of the impact indicators will let you know whether or not you need to alter anything within your campaign. For example:
For Example:
Is your audience watching your videos for long enough, or do they need to be edited? Is your audience engaging with your message, or do you need to change or adapt the message? Are people responding positively to your events, or do you need to change the format?
- Where you aiming is a slick or response or reaction from the ordinary?
- Did you change attitude, knowledge and behavior?
- How detailed is the data and what does it express beyond reach?
- Have you captured sentiment, reflection, acceptance and attachment?
- Where you aiming is a slick or response or reaction from the ordinary?
- Did you change attitude, knowledge and behavior?
- How detailed is the data and what does it express beyond reach?
- Have you captured sentiment, reflection, acceptance and attachment?
- Measuring and evaluating the impact of online campaigns is crucial for their success. However, simply relying on vanity metrics (likes, shares, impressions, etc.) may not give campaigners a complete picture of the impact they are having. To run a successful campaign, activists need to go beyond these metrics and capture qualitative and quantitative aspects of behavior change efforts, such as sentiment, longitudinal impact, and the role of other interlocutors.
- In the context of hate and violent extremism, capturing meaningful data and results for highly subjective and open-ended issues such as radicalization, deradicalization, disengagement, and desistance is challenging. Therefore, activists must prioritize planning and insight phases to ensure audience targeting, project measurement, and performance results are aligned with their desired outcomes. To measure the efficacy of a campaign, clear evaluation needs must be established. Evaluations should not rely solely on subjective and contested ideas, such as values and definitions of violent extremism. Activists should strive to produce reliable data and outcomes that reflect the impact of their campaigns.
- Developing an effective strategy is key to managing questions and considerations regarding the impact and reach, ensuring alignment with campaigners' objectives and the needs of their target audience.
- To avoid over-reliance on generic data, activists and campaigners should use an effects framework to identify and measure the impact of key tactics, content, or messaging deployed in their campaigns. The outcomes of the effects framework should be tied to strategic intent, objectives, and impact, and include objectives such as "reducing impact" or "undermining the narrative" of a terrorist or violent extremist group or ideology.
- Activists should go beyond reach statistics alone and develop more nuanced impact and effects measurements. A well-planned campaign strategy can be a powerful tool for activists to achieve their goals and create meaningful change.
- A clear framework
- A proposition
- The rationale expressing:
- What needs to be done?
- Why does it need to be done?
- How will the goals be achieved?
Relying solely on online reach data can create skewed notions of audiences and their needs. Online engagement, such as likes and shares, may not accurately reflect an individual's values, preferences, or position on a subject matter. Therefore, it is important to go beyond reach and focus on active communication that elicits a response beyond pressing "like." In the terrorism and violent extremism space, prebunking-based interventions are becoming more and more popular.
- Prebunking takes an inoculation theory approach and focuses on prevention rather than cure. It aims to build mental armor by preparing individuals to identify, assess, and make informed choices when violent extremist content appears on their screen or social media feed.
- Debunking-based efforts can struggle to match the quantity and propensity of violent extremist content, which is why Prebunking is seen as a more effective method in preventative spaces. By showing individuals enough examples of violent extremist-based content through pre-bunking tactics, they become better equipped to identify and question it.
- Activists should focus on building resilience and promoting active communication rather than solely relying on reach metrics.
- What is the problem-set?
- What does the campaign/project need to achieve?
- How will it achieve these and through what tactics/approach?
- What types of intent, action and engagement will make these tactics work?
- How will these tactics and effects be measured in order to create outcomes?
- What is the problem-set?
- What does the campaign/project need to achieve?
- How will it achieve these and through what tactics/approach?
- What types of intent, action and engagement will make these tactics work?
- How will these tactics and effects be measured in order to create outcomes?
To improve outcome indicators in a campaign, an assessment framework should have two core pieces of information: the baseline understanding of the intended audience's sentiment towards the issue being addressed and the desired end-state.
- The baseline assessment is an initial assessment of the audience's level of support, sentiment, grievance, hope, fear, or other relevant factors related to the issue being addressed. It allows campaigners to establish a starting point to measure any changes or movements in audience attitudes or behaviors during the course of a campaign.
- The desired end-state refers to the specific outcome or goal a campaign aims to achieve. It is important to clearly define the desired end-state to measure progress and success effectively.
An important step towards creating a successful campaign is the testing or pilot phase aimed at identifying strengths, weaknesses, challenges, and nuances. This phase can involve focus group discussions and A/B testing, which render different versions of the same content to similar audiences to test reactions. An iterative approach that is open to feedback from the sample audience can also provide nuanced results.
Activists trying to counter violent extremism or promote positive change can measure the effectiveness of their communication efforts via sentiment analysis.
Sentiment analysis can help activists identify the positive or negative emotions and intentions conveyed in the text. It is important to recognize that sentiment and effectiveness are not always synonymous, meaning measuring how people feel about an intervention is not necessarily evidence that the campaign is working. Activists should focus on understanding the specific emotions that participants experience and their associated action tendencies.
Sentiments and emotions can be measured via thought-listing exercises or traditional surveys. When designing surveys, it is important to tailor them to the specific intervention and ask participants about the relevant beliefs, attitudes, and sentiments related to that intervention.
