search

p4lang / pna

Portable NIC Architecture
Apache License 2.0
54 stars 21 forks source link

Add CI workflow to test P4 examples #90

Closed rst0git closed 1 year ago

rst0git commented 1 year ago

This pull request adds a GitHub Action workflow that runs runp4test.sh on pull request. This workflow uses the p4lang/p4c container image, which is automatically updated when a pull request has been merged in the upstream p4c github repository. The workflow then mounts the pna.p4 file and examples directory inside the container to run the corresponding tests with p4test.

jafingerhut commented 1 year ago

@rst0git Right now I have only run the runp4test.sh script manually so far, and looked at the currently-short output to see if there were any errors. Unlike the p4c repo's test scripts, it neither checks output files against expected output files, and I believe currently it does not even check the p4c exit status to see if it is 0 or non-0.

If the intent is to get a test failure that is visible to those deciding whether to merge in changes, do you know whether in the state it is in with this proposed PR, whether it would do that? Or perhaps we need some small changes to runp4test.sh to remember non-0 exit status from p4c and "pass on" that exit status to whoever invoked runp4test.sh ?

rst0git commented 1 year ago

Hi Andy, yes, the test should fail if the exit code of p4test is not 0. Here is the output of the test before #88 was merged: https://github.com/rst0git/pna/actions/runs/3470684915/jobs/5799193267 I have also added set -e in runp4test.sh so that the script would exit immediately.

jfingerh commented 1 year ago

@rst0git Pardon my ignorance on docker here: how often is the version of p4c used by this PR updated? With every p4c commit? Less often? By whom or what process is it updated? If it is on every p4c commit, roughly what is the latency after a commit before this docker image updates?

The reason I ask: At some point, I expect some PNA example programs will be relying on features not yet in p4c, but hopefully to be added within the next few months.

rst0git commented 1 year ago

how often is the version of p4c used by this PR updated?

The container image for P4C is automatically updated (by ci-container-image.yml) when a pull request has been merged to the GitHub repository. In addition, there is a time schedule for that workflow to update the image every few hours.

By whom or what process is it updated?

The p4c repository has been configured (by Antonin Bas) with credentials that enable the updated image to be pushed to Docker Hub.

what is the latency after a commit before this docker image updates?

From my experience, it takes less than an hour, but it depends on the time it takes for the GitHub action to build the image and push it to the Docker Hub registry.

I expect some PNA example programs will be relying on features not yet in p4c, but hopefully to be added within the next few months.

The test for the PNA example programs currently uses only the front-end of p4c (p4test) with pna.p4 from this repository. Thus, it should pass as long as the syntax of the PNA example programs and pna.p4 is accepted by the p4c front-end.