Updates the process for generating the summary results

openwallet-foundation / owl-agent-test-harness

Aries agent test framework, with agent backchannel support

https://aries-interop.info

Apache License 2.0

60 stars 66 forks source link

Updates the process for generating the summary results #861

Closed swcurran closed 3 months ago

swcurran commented 3 months ago

This PR changes the process for publishing the gh-pages from using jekyll and the ReadTheDocs theme, to using mkdocs, as we have been using for the other sites -- aca-py, aca-py plugins, etc.

Changes in this PR:

Remove the generated results from the repo -- they will only be in the gh-pages branch after generation
Added Mkdocs configuration data, removed jekyll configuration data
Added new GHA, currently only manually invoked. Will define a trigger after using this for a bit.
Removed old GHA and removed reference to the action in the run tests job.
Updated the gen-interop.sh page to copy AATH guide to docs folder for publishing, including cleaning up links so they work on the published site.
Updates to Markdown pages for publishing, clean up and updating/removing out of date information.

Once this gets working, I'll add some documentation about how the process works.

swcurran commented 3 months ago

Everything worked — checkout https://aries-interop.info! I added an index.html file after to merge, and manually triggered the workflow.

The results are worse than before — I was not aware that the Credo-VCX tests had not even run since May, and a number of others runsets have not run since July 29. Not good. I’ll try to dig in more while you are away @nodlesh .

nodlesh commented 3 months ago

There is a thread with @gmulhearn in discord about the VCX failures. I hadn't noticed the Credo-VCX not running since May. I wan't paying much attention to the AFJ runs since I'm replacing them anyway. I took quick look at that runset and it seems like it is an issue in the uploading artifacts action.

swcurran commented 3 months ago

I’m running locally ./manage runset ariesvcx-javascript -r and some tests are passing.

I’ve kicked the same thing off from GHA to see what it says.

I’m wondering if the issue is pushing to Allure… we’ll see.

gmulhearn commented 3 months ago

On the surface, I'm surprised by those VCX results, I was expecting our latest main image to fix a lot (atleast with self and with acapy). I'll have a look at the vcx side on Monday, thanks for making me aware

swcurran commented 3 months ago

I’m running the ariesvcx-ariesvcx locally and seeing passes as well — and it’s reporting 0 passing. Something is wrong with the GHActions, I think...