Research review - 'Jettisoning Junk Messaging in the Era of End-to-End Encryption: A Case Study of WhatsApp'

Review of the research paper "Jettisoning Junk Messaging in the Era of End-to-End Encryption: A Case Study of WhatsApp"

Overview

This paper does a case study of spam identification in WhatsApp. WhatsApp is a particularly interesting case because it offers end-to-end encryption, which means the message content is not accessible to WhatsApp servers. At a high level, the paper describes the spam data that they have collected, makes some observations on the data, then goes on to propose techniques for spam identification.

Data

I was curious to see how the authors obtained data for the study. The paper claims that the data was obtained from 'public' WhatsApp groups. It defines Public' WhatsApp groups as openly accessible groups, frequently publicised on well known websites, and typically themed around particular topics, like politics, football, music, etc. WhatsApp FAQs do not explain it using the exact terminology but I found the steps to create one described here https://faq.whatsapp.com/3242937609289432/?cms_platform=web. Basically, someone creates a WhatsApp group invite link and posts it on the web and anyone with the link can join the group. So the authors scanned the web, collected a lot of such public WhatsApp group links, then used a Selenium(UX tool) script to join and download all the messages from the groups, described in another paper[1]. They the authors perform filtering and clustering and labeling of data by using a combination of manual and automated methods.

Junk Mitigation Strategies

The paper proposes 3 junk mitigation strategies. 

1. Content based junk detection
2. Metadata based junk detection by platform
3. Content and metadata based detection by device

Since WhatsApp is end to end encrypted, we can discard the first strategy which requires the central server to have access to content. The second strategy is metadata based and seems feasible. The third one unfortunately due to the lower number of samples for training yields low accuracy(60 - 75%), which means that it can ultimately only assist admins in filtering spam and cannot be relied on for automated spam enforcement.

Strengths

Overall this paper was a good read. I found the section 'Characteristics of Junk content' fascinating because of the effort the authors have put into manual classification of their spam dataset. The observations on sender behavior like leaving and rejoining groups multiple times, to avoid being removed by admins are interesting to know. The proposed techniques that can be used by the end device or the platform (centrally) are also seem feasible.

Weaknesses

While this paper analyses a specific type of spam (spam in public WhatsApp groups), there are many other scenarios where spam is present - one to one, private group, semi-private group. It would be interesting to see how the techniques proposed here would fare in those scenarios. 

The paper claims that 1 in 10 messages in their dataset is spam. But to assess the scale of spam with respect to user impact, we need to get data on what proportion of WhatsApp groups are public and also what proportion of overall messages in WhatsApp are sent through such groups.

References

[1] "Kiran Garimella and Gareth Tyson. 2018. Whatapp doc? A first look at WhatsApp public group data. In ICWSM." https://arxiv.org/pdf/1804.01473.pdf