Monitoring Perceptions of Crisis-Related Stress Using Social Media Data

Pulse Lab Jakarta

Project Description

This research identifies and quantifies discussion themes in Twitter data in order to investigate what indicators can help understand people’s perceptions and concerns around food, fuel, finance and housing in the US and Indonesia. The purpose of this research project is to determine which indicators might be present in social media data that could shed light on how populations cope with global crises, such as commodity price volatility or the continuing global economic crisis. In this investigation, the analysis was limited to publicly available data from Twitter for July 2010 through October 2011 in Javanese/Bahasa Indonesia and English. The topics of focus included the affordability/availability of food, fuel, housing and loans.

By classifying a populations’ tweets into several categories associated with relevant topics, it was possible to perform quantitative analysis to better understand populations’ concerns: detecting anomalies such as spikes or drops in the number of tweets about particular topics (e.g. comments about power outages in Indonesia or student loans in U.S.), observing weekly and monthly trends in Twitter conversations (e.g. discussions around debt in U.S.), finding patterns in the volume of particular topics over time (e.g. discussions around housing in U.S.), comparing the proportions of different sub-topics to understand shifts in trends over time (e.g. the ratio of tweets about formal loans vs. informal loans in Indonesia), or relating trends in Twitter conversations with external indicators (e.g. conversations around the price of rice in Indonesia mimicking the official inflation statistics).
Monitoring Perceptions of Crisis-Related Stress Using Social Media Data 1
The number of tweets discussing the price of rice in Indonesia over the last year follows a similar function as the official inflation statistics for the food basket.This research has confirmed that Twitter data can be useful for understanding the immediate worries, fears and concerns of populations, but at the same time, the research suggested that it is a poor source of data for gauging people’s long term aspirations. There are several remaining challenges, in particular that Twitter has a specific culture and demographic which needs to be better understood. Overall, this exploratory research shows some of the potential of Twitter data for exploring people’s perceptions of crisis-related stress and suggests research lines and methodologies for further investigations.
Monitoring Perceptions of Crisis-Related Stress Using Social Media Data 2

Conversations around finance in the US, modulated by a baseline weekly pattern of fewer discussions on the weekends, show an increase of conversations from the 15th July to the 15th August motivated by the US debt ceiling debate.

Learn More about the Project Methodology:

To delve deeper into the Twitter data analyzed in Bahasa and Javanese, download this background annex (PDF), which provides example Tweets and the taxonomies which were developed about food, fuel, finance, and housing. The sample Tweets represent the type of material used to train machine learning algorithms to classify the Tweets into the different categories.

Watch an Overview Presentation of the Project:

Did you find this project interesting? Share it with your networks!

Share on facebook
Share on twitter
Share on linkedin
Share on email

Below you can find our latest examples of our collaborative research, prototypes and experiments, where we analyse digital data to advance global development, support humanitarian action, and promote peace. For more, go to the research projects page.

Pulse Lab Jakarta

Nowcasting Air Quality Using Twitter

In some communities in Indonesia, real-time air quality information is not available, which is crucial for improving response efforts in regions affected by air pollution.

Scroll to Top