With the advancement of open source firmware projects, we need a reliable quality assurance process to automate the firmware level testing. In this talk we would like to talk about ConTest - a modular framework aimed at automating system testing workflows and how one can build board-specific testing infrastructure on top of it.

ConTest aims to provide maximum flexibility to the users and to validate as much as possible at the compile-time as well as at job submission time to avoid unnecessary operations. ConTest provides a modular and pluggable set of interfaces to define and handle operations like fetching information of the systems to run the tests on, testing the initial state of the test systems, implementing system-specific actions & measurements and logging the test progress as well as final results. On top of this, ConTest’s test system uses the same framework for testing the correctness and reliability of its own plugins. This talk will also cover the existing plugins provided by upstream ConTest for different parts of the framework while outlining the process of creating the pluggable modules.