Manual Pages  — KYUAFILE

A Kyuafile is a Lua script whose purpose is to describe the structure of the test suite it belongs to. To do so, the script has access to a collection of special functions provided by kyua(1) as described in Helper functions.

The test suite name provided in the test_suite() call tells kyua(1) which set of configuration variables from kyua.conf(5) to pass to the test programs at run time.

ATF test programs are those that use the atf(7) libraries. They can be registered with the atf_test_program() table constructor. This function takes the name of the test program and a collection of optional metadata settings for all the test cases in the test program. Any metadata properties defined by the test cases themselves override the metadata values defined here.

Plain test programs are those that return 0 on success and non-0 on failure; in general, most test programs (even those that use fancy unit-testing libraries) behave this way and thus also qualify as plain test programs. They can be registered with the plain_test_program() table constructor. This function takes the name of the test program, an optional test_suite name that overrides the global test suite name, and a collection of optional metadata settings for the test program.

TAP test programs are those that implement the Test Anything Protocol. They can be registered with the tap_test_program() table constructor. This function takes the name of the test program and a collection of optional metadata settings for the test program.

The following metadata properties can be passed to any test program definition:

allowed_architectures
	Whitespace-separated list of machine architecture names allowed by the test. If empty or not defined, the test is allowed to run on any machine architecture.
allowed_platforms
	Whitespace-separated list of machine platform names allowed by the test. If empty or not defined, the test is allowed to run on any machine platform.
custom.NAME
	Custom variable defined by the test where 'NAME' denotes the name of the variable. These variables are useful to tag your tests with information specific to your project. The values of such variables are propagated all the way from the tests to the results files and later to any generated reports. Note that if the name happens to have dashes or any other special characters in it, you will have to use a special Lua syntax to define the property. Refer to the EXAMPLES section below for clarification.
description
	Textual description of the test.
execenv
	The name of the execution environment to be used for running the test. If empty or not defined, the 'host' execution environment is meant. The possible values are:

host	The default environment which runs the test as a usual child process.
jail	The FreeBSD jail(8) environment. It creates a temporary jail to run the test and its optional cleanup logic within. This feature requires kyua(1) to be running with superuser privileges. The difference between security.jail.children.max and security.jail.children.cur sysctl of the jail kyua(1) is running within must have a value high enough for the jail based tests planned to be run. For instance, the value 1 should be enough for a sequential run of simple tests. Otherwise, such aspects as parallel test execution and sub-jails spawned by specific test cases should be considered. The formula of a temporary jail name is 'kyua' + test program path + '_' + test case name. All non-alphanumeric characters are replaced with '_'. 'kyua_usr_tests_sys_netpfil_pf_pass_block_v4' is an example for /usr/tests/sys/netpfil/pf/pass_block:v4 test case.

execenv_jail_params
	Additional test-specific whitespace-separated parameters of FreeBSD jail(8) to create a temporary jail within which the test is run. It makes sense only if execenv is set to 'jail'. kyua(1) implicitly passes 'children.max' parameter to jail(8) for a temporary jail with the maximum possible value according to the jail kyua(1) itself is running within. It allows tests to easily spawn their own sub-jails without additional configuration. It can be overridden via execenv_jail_params if needed.
is_exclusive
	If true, indicates that this test program cannot be executed along any other programs at the same time. Test programs that affect global system state, such as those that modify the value of a sysctl(8) setting, must set themselves as exclusive to prevent failures due to race conditions. Defaults to false.
required_configs
	Whitespace-separated list of configuration variables that the test requires to be defined before it can run.
required_disk_space
	Amount of available disk space that the test needs to run successfully.
required_files
	Whitespace-separated list of paths that the test requires to exist before it can run.
required_memory
	Amount of physical memory that the test needs to run successfully.
required_programs
	Whitespace-separated list of basenames or absolute paths pointing to executable binaries that the test requires to exist before it can run.
required_user
	If empty, the test has no restrictions on the calling user for it to run. If set to 'unprivileged', the test needs to not run as root. If set to 'root', the test must run as root.
timeout
	Amount of seconds that the test is allowed to execute before being killed.

syntax(2)

test_suite('FreeBSD')

atf_test_program{name='network_test',
                 execenv='jail',
                 execenv_jail_params='vnet allow.raw_sockets',
                 required_user='root'}

A test case itself may have no requirements in superuser privileges, but required_user='root' metadata property reminds that the jail execution environment requires kyua(1) being running with root privileges, and the test is skipped otherwise with the respective message. The combination of execenv set to 'jail' and required_user set to 'unprivileged' does not work respectively.