Report title: Flaky Test Detection in Android via Event Order Exploration

Report time: December 1st, 14:00-15:00

Report host: Su Ting

Offline report location: B407 Science Building

Tencent Meeting Number: 503-653-610

Abstract:

Validation of Android apps via testing is difficult owing to the presence of flaky tests. Due to non-deterministic execution environments, a sequence of events (a test) may lead to success or failure in unpredictable ways. In this work, we present an approach and tool FlakeScanner for detecting flaky tests through exploration of event orders. Our key observation is that for a test in a mobile app, there is a testing framework thread which creates the test events, a main User-Interface (UI) thread processing these events, and there may be several other background threads running asynchronously. For any event e, whose execution involves potential non-determinism, we localize the earliest (latest) event after (before) which e must happen. We then efficiently explore the schedules between the upper/lower bound events while grouping events within a single statement, to find whether the test outcome is flaky. We also create a suite of subject programs called FlakyAppRepo (containing 33 widely-used Android projects) to study flaky tests in Android apps. Our experiments on the subject-suite FlakyAppRepo show FlakeScanner detected 45 out of 52 known flaky tests as well as 245 previously unknown flaky tests among 1444 tests.

Introduction of Lectuer:

Zhen Dong is a pre-tenure Associate Professor at Fudan University. Before joining Fudan University, Zhen was a senior PostDoc at the National University of Singapore. Zhen received his PhD from Heidelberg University in 2017. His research interest is development of techniques and tools for improving software reliability and security, particularly in the area of mobile computing. His work won the ACM Distinguished Paper Award at ICSE'20 and the Best Paper Award at AsiaCCS'21.