Obtaining Ground-Truth Software Architectures

Joshua Garcia, Ivo Krka, Chris Mattmann, and Nenad Medvidović

University of Southern California, USA; Jet Propulsion Laboratory, USA

Track: Software Engineering in Practice
Session: Software Architecture
Undocumented evolution of a software system and its underlying architecture drives the need for the architectures recovery from the systems implementation-level artifacts. While a number of recovery techniques have been proposed, they suffer from known inaccuracies. Furthermore, these techniques are difficult to evaluate due to a lack of ground-truth architectures that are known to be accurate. To address this problem, we argue for establishing a suite of ground-truth architectures, using a recovery framework proposed in our recent work. This framework considers domain-, application-, and contextspecific information about a system, and addresses an inherent obstacle in establishing a ground-truth architecture the limited availability of engineers who are closely familiar with the system in question. In this paper, we present our experience in recovering the ground-truth architectures of four open-source systems. We discuss the primary insights gained in the process, analyze the characteristics of the obtained ground-truth architectures, and reflect on the involvement of the systems engineers in a limited but critical fashion. Our findings suggest the practical feasibility of obtaining ground-truth architectures for large systems and encourage future efforts directed at establishing a large scale repository of such architectures.