It should be as easy as the following steps:
At the heart of it, this script does two things:
It acts like a CHROOT almost in the sense that everything is performed at the location of where this repository, "adtools_testing", is cloned. Nothing in your environment is molested outside of the location from where you have cloned this respoistory. It is meant to enable the rapid development and testing of ADTOOLS. Hopefully, a lot of the laborious work has been wrapped up in the scripts and makefiles contained herein but it is completely capable of using an already existing version of ADTOOLS on your machine versus building or using the in-place version if desired.
The usage of the "adt" script can be seen by merely invoking "./adt".
With regards to building the tests, the "adt" script will create an
artifact named "\
Each variant LHA will contain the source code and any makefiles. Be advised that the main makefile will not be usable outside of the testing framework environment which is generally a linux machine that is building the tests. In most cases, there is a standalone makefile generally named sa.makefile that can be used instead if it is desired to re-buld the program on the AmigaOne machine but, it should be noted that the sa.makefile is not gauranteed to be the same as the framework makefile since these may become unsynchronised.
It should be understood that some tests may fail to build for differently configured cross-compilers. This is not necessarily a failure. For example, there may be tests that can only be built using GCC 11. In the event of a failure, simply inspect the log file for that variant. See further below - "Integration into the test framework" - for ways to build dummy executables if build failures for particular environments should be avoided.
The script named "run_*.script" executes the test executable, records the STDOUT and STDERR, compares STDOUT to an expected set of results and reports the result as:
The usage complexity of this script can go from simple to complex. You can supply prefix options to the script. This allows for checking out and building multiple cross-compilers for modification and for testing. For example, you may want to build a cross compiler using GCC 10, SDK 53.30. The script allows you the option to do that by providing a prefix option. Basically, this prefixes the necessary files/folders with the prefix name. You may want to invoke something like "-p gcc10_sdk53_expClib". You can then build cross compiler and tests for that prefix so long as you supply the prefix option for each build and test action. You can then do the same for a different combination, such as GCC 11 with classic CLIB2.
In addition to prefixing is the ability to deviate from the principles of this script. You can install the cross compiler outside of the directory which was used to clone this repository. The checkout of ADTOOLS and all the source files for the cross compiler are still cloned inside the cloned folder, but the "make install" of the cross compiler is installed outside. This is similar to prefixing and not really recommended.
A number of options may be used to build a cross compiler. For example: "./adt -b -e beta10 -s 53.30 -g 10" and it can become difficult to remember those options. Indeed, they need to be remembered again if an action should be performed in the context of that particular cross compiler, such as building it again due to source file changes, or if wanting to build the tests for that compiler. This script will store the build sessions settings in a file. On the next invocation of the script
See example "10_x_div_by_float_zero_wchar" for a possible approach to building a test with SPE in mind. Currently, only GCC 6 can generate SPE instructions.
Alternatively, use the existing framework which attempts to make life easier. The only target required would be "$(PROG)". There is a makefile function, "LOG_CMD", which enables automatic logging to an appropriately named file. Using this approach will also archive up the test executable and necessary, dependent Shared Objects (in the case the Shared Objects are used) and log files and AmigaOS scripts into an LHA file which is ready to be transferred to the AmigaOne machine. The log files are useful to get verbose output from the compiler phase, linker phase, output of READELF, needed Shared Objects and suspected required Shared Libraries; suspected because the framework merely greps for a pattern of "*.library" in the binary and assumes that the executable will try to open that library.
The framework will always attempt to create 6 variants, as mentioned above, but in some test cases the test may not care about a particular variant. For instance, a test created using dlopen() may not not care about any of the 2 static variants. In such a case, a DUMMY test can be created for that situation. The framework builds a dummy binary adhering to the naming convention that just returns a particular exit code that the scripts know to mean a dummy test. Dummy tests always pass. An example can be seen in "12_dlopen_binutils_test".
The most basic example of an integrated test can be found in "0_rjd_simplest_example".
In case that the test or variant of a test is not building due to makefile errors, the "-h" option can be used when building the tests in the hope of generating some error output which can solve the problem.
Inspections can be added to tests by simply providing commented lines in the makefile of the test being added. See "1_rjd_test_example" for an example.
Since the testing framework runs all the building of the tests in parallel it is important to rename any test specific artifacts - such as relocatable object files, archives files or shared objects - with a unique name. This allows parallel builds where the same files are not being incorrectly linked with or added to the incorrect archive. See "2_capehill_adtools_issue_139_test_code" for an example of how to name any specific test artifacts using the already provided "FILE_INFIX" variable. Recall that for every test there are 6 dimensions as described above (newlib,clibs * dynamic,static) running in parallel which is also running in parallel with every other test case. The test framework handles contention everywhere else using this exact approach.
In the case you just want to add a test without needing to follow the test framework then you can add a standalone test. Just create a directory under tests with a makefile with an "all" target and a "clean" target. Variables are exported in the parent makefile (the one under tests, which include some useful variables if needed). A standalone test is shown in "99_rjd_standalone"; the test framework will still build it for you.
The test framework will log all output to a file named log.txt inside the test folder, but, this log file is primitive.
Standalone tests are also added to the final archive under the folder named "Standalone_Tests".
Standalone tests do not have all 6 variants built. The test framework invokes the "all" rule once. That will have to be handled manually, in the makefile, if desired.
In various tests there may be a file named "sa.makefile". This file can be used instead of the makefile in the case that there should also be an option to run the test without using the test framework. This file must be manually created. The file does not have to be called "sa.makefile", it can be called whatever is desired, but that name is a general guideline.
The test framework will never do anything with this file. It is just useful for those that want to pull this repository down and run the tests immediately in their own way.
In the case you are integrating a test into the test framework and you want to forcefully request certain files to be deleted when using ./adt -c you can add a Makefile variable named CLEAN_ME with a list of file that you want to delete. See test 13_constructor_destructor for an example. By default, the testing framework will delete commonly anticipated files, such as those that have an extension of .so or .o, but there are always exceptions.
Add any files to this variable and they will be included into the test variant. Look at the Makefile in "14_rjd_simpleSharedLib" for an example. Without adding the shared object that will be loaded at runtime to the EXTRA_FILES variable, it would not be added to the overall variant LHA file. By default the test framework tries to add all source files and makefiles to each variant.
The framework checks for the existence, finally, of "$(PROG)". As stated above, that is the only required rule. There can be situations where the test may require the creation of a file as well as "$(PROG)". For instance, a test that generates two separate executable files. Since the framework is purposefully permissive in order to log as much information as possible, its success check is merely the existence of "$(PROG)". By adding files to the NEED_DEP variable, the framework will ensure that "$(PROG)" and any of the files in "$(NEED_DEP)" exist, otherwise it will consider it a fail.
This variable is not really used internally, but it is worth mentioning. We know about the -athread option. This specifies which threading implementation to use. Normally, we use -athread=native to specify that we want to use the AmigaOS4 native approach to threading. See an example of its use in "1_rjd_test_example" and how it defaults to native, but can be overridden. For instance, to build with "pthread", i.e. -athread=pthread, you could issue the command "./adt -t THREAD_IMPL=pthread".