IMDEA Networks Institute Publications Repository

Mis-shapes, Mistakes, Misfits: An Analysis of Domain Classification Services

Vallina, Pelayo and Le Pochat, Victor and Feal, Álvaro and Marius, Paraschiv and Gamba, Julien and Burke, Time and Oliver, Hohlfeld and Juan, Tapiador and Vallina-Rodriguez, Narseo (2020) Mis-shapes, Mistakes, Misfits: An Analysis of Domain Classification Services. In: The 20th ACM Internet Measurement Conference (ACM IMC 2020), 27-29 October 2020, Virtual event.

[img] PDF
Download (3285Kb)


Domain classification services have applications in multiple areas, including cybersecurity, content blocking, and targeted advertising. Yet, these services are often a black box in terms of their methodology to classifying domains, which makes it difficult to assess their strengths, aptness for specific applications, and limitations. In this work, we perform a large-scale analysis of 13 popular domain classification services on more than 4.4M hostnames. Our study empirically explores their methodologies, scalability limitations, label constellations, and their suitability to academic research as well as other practical applications such as content filtering. We find that the coverage varies enormously across providers, ranging from over 90% to below 1%. All services deviate from their documented taxonomy, hampering sound usage for research. Further, labels are highly inconsistent across providers, who show little agreement over domains, making it difficult to compare or combine these services. We also show how the dynamics of crowd-sourced efforts may be obstructed by scalability and coverage aspects as well as subjective disagreements among human labelers. Finally, through case studies, we showcase that most services are not fit for detecting specialized content for research or content-blocking purposes. We conclude with actionable recommendations on their usage based on our empirical insights and experience. Particularly, we focus on how users should handle the significant disparities observed across services both in technical solutions and in research.

Item Type: Conference or Workshop Papers (Paper)
Depositing User: Alvaro Feal
Date Deposited: 01 Oct 2020 09:26
Last Modified: 10 Nov 2020 09:47

Actions (login required)

View Item View Item