Update `scripts/run_commit_tests.sh` to Run Tests and Generate `/tmp/SHARED.UNITS`

[TeamSPoon]

The current scripts/run_commit_tests.sh script is not fully configured to run the tests and generate the /tmp/SHARED.UNITS file. To ensure that the tests are executed and the results are correctly captured in the expected file, we need to modify this script.

Task

• Install MettaLog: Ensure that MettaLog is properly installed in the environment where the script will run.

  • Admittedly this is one hell of a task by itself getting SWI-Prolog+Janus+MeTTaLog to install on Github's private VM!

• Modify the Script: Edit scripts/run_commit_tests.sh to:

  1. Run the MettaLog tests located in the tests/baseline_compat directory.
  2. Ensure the test output is captured and saved to /tmp/SHARED.UNITS.

• Example Command: The script should include a command similar to:

  ./mettalog tests/baseline_compat # this implicialy generates /tmp/SHARED.UNITS

• Verification: After making the changes, verify that the script successfully runs the tests and generates the /tmp/SHARED.UNITS file with the expected output.

Acceptance Criteria

• The scripts/run_commit_tests.sh script should execute the necessary tests and generate the /tmp/SHARED.UNITS file.
• The generated file should contain the test results that can be used in subsequent steps of the CI pipeline.

Notes

• Ensure that any required dependencies for MettaLog are installed as part of the setup in the script.
• If necessary, update any related documentation to reflect the changes made.

[AdrickTench]

It seems to me that mettalog tests/baseline_compat already generates the /tmp/SHARED.UNITS file. So it seems like the real work for this script then is actually to verify/install MettaLog?

I figure logging is good, and running the tests seems to generate a lot of output, so so far I wrote

timestamp=$(date +"%Y-%m-%dT%H:%M:%S")
logfile=/tmp/run_commit_tests_$timestamp.log

echo Running baseline_compat tests, logging to $logfile
mettalog tests/baseline_compat > $logfile 2>&1

The log is really huge though, and contains a lot of non-ASCII characters - my text editor is not happy when I look at it. Maybe we should generate less? Or I could try only logging the error output and pipe the rest to null

[TeamSPoon]

Yes the real meat of this issue is the first task (gettign the system installed)

The next part about runnig the tests is important to and actualyl you need to make up an output directory and do a ...

mettalog --output=reports/tests_output/ran-at-$timestamp/ tests/baseline_compat 2>&1 > /dev/null

Oh yeah this should be piped to (both stdin and stdout) to /dev/null .. no reason to capture it! The output of tests are being captured into individual *.ansi files that get converted to .metta.html files in a output directory.

the output directory you passed will get used in the scripts/into_junit.py and become a link to github pages instead of logicmoo.org

[AdrickTench]

So we're running tests on commit now, which means the meat of this issue is done:

mettalog --test --clean --output=$output  tests/baseline_compat/module-system/

But also copying a prior run so as to have a bigger report:

cat ./reports/SHARED.UNITS.PREV.md >> /tmp/SHARED.UNITS

So let's decide:

• which tests do we run on commit?
• which tests do we run nightly? (I believe we can set up a GH action to run tests nightly)
• which report do we publish as our GH pages site? The latest commit, or the latest nightly? I don't think we could really combine them into one report.

[AdrickTench]

We are able to publish both nightly and CI tests to pages (see https://trueagi-io.github.io/metta-wam/nightly/ and https://trueagi-io.github.io/metta-wam/ci/). run_commit_tests.sh runs the CI tests and run_nightly_tests.sh runs the nightlies.