Sentry Autotest mode
About Autotest
Autotest is a test methodology using a dedicated task built in order to requires various Sentry kernel autotest methods execution.
The goal of the autotest mode is to validate various kernel subsystems, mostly managers, in order to validate runtime-related
To control the autotest mode from the autotest application, the Sentry kernel include, in autotest mode exclusively, a dedicated syscall denoted sys_autotest(), with the following API:
typedef enum autotest_request {
SENTRY_AUTOTEST_ENTER = 1, /**< enter autotest mode */
SENTRY_AUTOTEST_LEAVE, /**< leave autotest mode */
SENTRY_AUTOTEST_CALL_MGR_CLOCK, /**< ask for clock mgr autotest */
SENTRY_AUTOTEST_CALL_CHECK_MGR_DEBUG, /**< ask for debug mgr autotest */
SENTRY_AUTOTEST_CALL_MGR_DEVICE, /**< ask for device mgr autotest */
SENTRY_AUTOTEST_CALL_MGR_INTERRUPT, /**< ask for interrupt mgr autotest */
SENTRY_AUTOTEST_CALL_MGR_IO, /**< ask for IO manager autotest */
SENTRY_AUTOTEST_CALL_MGR_MEMORY, /**< ask for IO memory autotest */
SENTRY_AUTOTEST_CALL_MGR_SECURITY, /**< ask for security manager autotest */
SENTRY_AUTOTEST_CALL_MGR_TASK, /**< ask for task manager autotest */
SENTRY_AUTOTEST_CALL_SCHED, /**< ask for scheduler autotest */
} autotest_request_t;
sys_return_t sys_autotest(autotest_request_t command);
Runtime kernel testing
Autotest is built for two main usages:
testing manager behavior and performances in nominal usage. The goal is to get back test measurements of various Sentry kernel components directly on board (or on the targetting SoC), to get back some measurements on various testable values that need to be executed on target this is done through the usage of the call command set of the API, that ask a kernel module to execute its own autotest and return the result
testing invalid behavior kernel response, ensuring kernel hardening triggering to various userspace based invalid transmission. That later case verify that the various security triggers are being executed as expected. This is done using the enter/leave commands, that change the kernel behavior when executing all sanitation and security related functions starting at the ENTER command, and upto the LEAVE command.
To support both these usage, the Sentry kernel autotest API has been designed, in association with the autotest application, for a flexible autotest activation/deactivation model.
When being autotested, the kernel communicate with the autotest application using specially crafted signals that only exist in autotest mode. These signals are emitted by the kernel to the autotest application, which can receive them in the very same way it receives all signals: through the sys_waitevent syscall.
The only difference here is that the signal source is the kernel (source id being 0), and the signal value is not a standard signal but one of the autotest specific signals, as defined in the UAPI signal header:
Autotesting kernel components
Autotesting Sentry module is as easy as a syscall pair:
A typical usage of such an autotest is, for example, a performance measurement of the kernel random source:
1kstatus_t mgr_security_autotest(void)
2{
3 // [...]
4 pr_autotest("START execute 256 entropy generation from entropy source");
5 /* executing 256 random seed requests */
6 for (uint32_t i=0; i < 256; ++i) {
7 start = systime_get_cycle();
8 if (unlikely(mgr_security_entropy_generate(&seed) != K_STATUS_OKAY)) {
9 failures++;
10 }
11 stop = systime_get_cycle();
12 uint64_t duration = stop - start;
13 if (duration > max) {
14 max = duration;
15 }
16 if ((min == 0) || (duration < min)) {
17 min = duration;
18 }
19 average += duration;
20 }
21 /* average div 256 */
22 average >> 8;
23 pr_autotest("entropy_generate min time: %llu", min);
24 pr_autotest("entropy_generate max time: %llu", max);
25 pr_autotest("entropy_generate average time: %llu", average);
26 pr_autotest("entropy_generate failures: %llu", failures);
27 pr_autotest("END");
28
29 return status;
30}
Kernel autotests logging
As shown is the previous code sample, Sentry kernel autotest function use a specially crafted log output for test execution, in order to allow kernel logging post-processing.
The kernel autotest dediacated logging is using the pr_autotest() API, which work in a very similar way to other pr_ API of the debug manager:
1/** @def autotest log prefix */
2#define KERN_AUTOTEST "[T]"
3/**
4 * @def pr_ auto format string for pr_autotest only, adding TEST and current timestamping in u64
5 * format
6 */
7#define pr_autotest_fmt(fmt) "%lu%lu: %s: " fmt "\n", systime_get_cycleh(),systime_get_cyclel(), __func__
8/**
9 * @def autotest messages, for autotest functions only
10 */
11#define pr_autotest(fmt, ...) \
12 printk(KERN_AUTOTEST " " pr_autotest_fmt(fmt), ##__VA_ARGS__)
With such an API, test output formatting is automatically generated.
Warning
The usage of autotest logging is acceptable in handler mode for autotest use case only, when IRQ events are under control of the test suite. huge autotest logging highly increase uninterruptible kernel sequence and must be considered at test design time
Note
The Sentry kernel logging is voluntary IRQless and can be called in handler mode. Although, it should not be used in the middle of performances measurement loop without impacting it
Autotest application
Autotest application is a dedicated application that is made to check the overall kernel UAPI. Autotest app hold a dedicated, small footprint test suite mechanism using a test library based on multiple macros so that an efficient pretty printing is delivered to the kernel log in order to be CD-compliant
Note
A typical usage of the autotest mechanism is through a continuous delivey targetting a test farm representative of the supported SoC(s), in which all the kernel UAPI is checked and the check report retreived
Autotest application check successively each syscall, and get back multiple informations on it, including performances, effective security check tests (capability checking, etc.), and resulted value analysis.
Autotest testlib is built to support the notion of:
test suite: a set of tests that check the same subcomponent
test: one or multiple calls that correspond to a given test
All tests and tests suites are executed consecutively. autotest do not stop on error, but instead delivers all informational messages for both successful and failing tests.
Defining a test
A typical basic test looks like the following the following:
TEST_START();
micro_st = sys_get_cycle(PRECISION_MICROSECONDS);
copy_from_kernel((uint8_t*)µ, sizeof(uint64_t));;
ASSERT_EQ(micro_st, STATUS_OK);
ASSERT_GT((int)micro, 0);
TEST_END()
Multiple assertion macros and logging tooling has been defined as testbed helpers.
Testlib for autotest
Testlib is a small footprint testing library that aim to be executed on-target. A small set of usual test helper functions and macros are defined.
Boolean expressions assertion is supported:
/* check that numeric values are equal */
ASSERT_EQ(a,b);
/* check that numeric values are different */
ASSERT_NE(a,b);
/* check that a is bigger or equal to b */
ASSERT_GE(a,b);
/* check that a is smaller or equal to b */
ASSERT_LE(a,b);
/* check that a is bigger than b */
ASSERT_GT(a,b);
/* check that a is smaller than b */
ASSERT_LT(a,b);
/* check that a is in [b,B] range (b <= a <= B) */
ASSERT_IN_RANGE(a,b,B);
Generic helpers for test and test suites declaration are defined
/* starting a test. The current function name is used as identifier */
TEST_START()
/* stoping a test. The current function name is used as identifier */
TEST_END()
/* starting a test suite. The suite name is the given string argument */
TEST_SUITE_START("mysuite")
/* stoping a test suite */
TEST_SUITE_END("mysuite")
Some generic API for pretty printing and complementary information delivery is supported:
/*log the given printf fmt-formatted arguments (printf compatible) */
LOG("hello %lu", myvalue);
About tests and capabilities
By now, autotest capabilities is build-time fixed. The goal is to use the autotest-dedicated syscall in order to support dynamic capabilities in autotest mode only, in order to test capability checks with multiple capability set successively.
Note
The dynamic capability check is not yet supported