Wednesday, October 2, 2013

First day of State of Food Insecurity (SOFI) 2013

The FAO flagship publication SOFI 2013 was release yesterday on the 1st of October, the publication is the most important report in monitoring the progress towards the 2015 Millenium Development Goal and ultimately eliminate hunger.

I was interest in how the people responded, so I scrapped some data from Twitter and previous work to carry out some basic analysis.

In total, there were 1,284 Tweets over the course of 24 hours. However, 86% were retweets which does not add much information for text mining and association analysis.

Nevertheless, lets have a look at what people are saying using some basic frequency and sentimental analysis. Clearly, hunger is the most mentioned words however there are words which are interesting and they have been colored according to theopinion lexicon of Hu and Liu. Negative feelings are red, positive feelings are blue while neutral words has remained grey.

The negative words like miss, bad and slow has pointed out that more efforts are required to achieve the MDG; furthermore people are suffer from chornic hunger. Nevertheless, the work of FAO has been recognised by good work and progress.

From the word cloud we can observe that it is impossible to use a single lexicon to generalize emotions. For example, fallen may usually used in a negative way but in this context it represents the positive movement of rate of undernourishment falling. A special lexicon may be required to examine the social feedback about SOFI.

Unfortunately I had problem obtaining the geocode from the API, otherwise it would be interesting to examine where these tweets originate.

Given that majority of the tweets are retweets of the official release message, little information can be obtained from these tweets. We hope a revision in a few weeks/months time will provide us with much more insight about the status of the publication.

The codes can be found at my Github repository .

3 comments:

  1. As I was reading this, I noticed "progress" showed up alot (which we would assume is a good thing) but then "lack" also appears a fair bit. It's hard to say if "progress" is truly a positive word because it could be coming primarily from a phrase such as "lack of progress".

    Very interesting dataset and analysis, though! I'll be interested to hear more as time progresses.

    ReplyDelete
  2. Nice Michael. I'm looking at your code and don't see where you passed the colors to the word cloud (red for bad, green for good). How'd you do that sorcery?

    ReplyDelete
  3. Josh and Maria, you are right I think it would be important to account not only for single words but also joint meanings.

    Amit, just pushed the final version to Github.

    ReplyDelete