Linaro Connect Bangkok

Continuing our regular Users Forum, for all your questions about LAVA.

LAVA is at the same time supporting a wide range of devices and maintained by a small team of people. Which mean that maintainers do not have access to most device types that LAVA is supporting.
LAVA Federation project is aiming at testing the LAVA software on community owned hardware. Every day, LAVA functional tests are spread across multiple labs, owned by the community members, with a variety of community hardware.

The goal of this presentation is to help community members to jump in and participate to this common effort.

Different test frameworks have very different approaches to tests, including when they run, how they are run, what data formats are used, and what the various fields are that control test operation and results analysis. Recently, Tim has conducted a survey of different "test definitions", in an attempt to cull best practices and search for commonality that will lead to enhanced interoperability between test systems. In this session, Tim will present the results of his survey, and make suggestions for areas where tests could be harmonized, and used in common between Fuego and Lava - two test systems with very different approaches.

KernelCI generates daily an average of two thousand Linux Kernel builds, submitted to several labs across planet to mainly check if system booting works correctly. Therefore tons of data are created and displayed on a frontend interface for further use. Current KernelCI requires significant technical effort changing visualizations, including builds and jobs listings. The KernelCI New Generation seeks to apply state-of-the-art data collection and data visualization tools as an alternative frontend to KernelCI. Our focus is on an instance of ElasticSearch, Logstash, and Kibana collecting and displaying live KernelCI data. All the tooling used to this project will be available.

Fuego is yet another automated test framework for Linux. But it is different from other systems in a few key ways. This session will introduce Fuego and describe a few of its features that are different (to Tim's knowledge) from other test systems like LAVA, Buildbot, or LTP.

Some of the features are Fuego's focus on individual board testing (rather than test labs), results analysis separated from the test itself, and on building a suite of tests readily available to begin testing.

Tim will also share some of his vision for the future of open source testing, and what features of Fuego are being prototyped now to move towards that vision. (That is, Tim will share his Fuego roadmap.)

Test results triage is the most time-consuming step currently in Texas Instruments CI cycle. Embedded tests are more susceptible to failures caused by external factors due to test setup complexity. A typical embedded test contains multiple points of failure.
This work uses a neural network to classify test results automatically and to detect 'real' regressions. It helps scaling TI's validation cycle by automating the most time-consuming step.

Testing a bootloader in LAVA is more difficult than running tests on a typical Linux-based operating system such as Debian or Android. Robustly provisioning a new bootloader requires boards to be better designed for automation and to be deeply integrated into LAVA. For bootloaders, we often have to drive the tests externally since, with an OS or POSIX shell we cannot simply launch scripts to manage execution of the test suite.

During this presentation, we will discuss these challenges in greater detail and look at what solutions LAVA offers to achieve bootloader automate testing. We’ll close out the session with an example test description that allows a bootloader test suite to be fully automated using LAVA.