search

bowtie-json-schema / bowtie

JSON Schema in every programming language
https://bowtie.report/
Other
50 stars 38 forks source link

Generating report for implementation versions workflow #1330

Closed adwait-godbole closed 1 week ago

adwait-godbole commented 1 week ago

Hi @Julian I have written this workflow and you can see a run of it over here on my fork (though there it shows workflow failed because of draft3 job failure which is happening because of no implementation (ruby and hyperjump in this case) supporting draft3 and hence none started successfully so it is exiting with some exit code and failing the entire job. I am not sure how to fix it yet. Can you help regarding this please.) and also possibly have a look at the uploaded artifacts.

Currently I have made it so that this versioned-report.yml workflow runs for all versions of all implementations which have "matrix-versions.json" file for them. However while implementing this I've made sure that once we run the suite against all versions for one implementation, then we delete whatever images we fetched for it so that helps us not running out of memory at any point of time.

What I am planning to do now is to modify our current report.yml workflow to download the artifacts that were published by this versioned-report.yml workflow's last run and every time upload it alongside other implementation reports which will be generated by report.yml itself. This way we have the traditionally generated reports for the latest versions of all implementations by report.yml and also alongside them we will have these versioned reports which were generated by versioned-report.yml workflow. This way report.yml does not have to do any heavy lifting of generating these versioned reports since we anyways will be manually triggering them once/whenever we need and storing them forever so report.yml just needs to download them and use them.

📚 Documentation preview 📚: https://bowtie-json-schema--1330.org.readthedocs.build/en/1330/

adwait-godbole commented 1 week ago

@Julian I will address your comments on the other PR with the appropriate changes but meanwhile can you have a look at this one first ? So I can fix any issues over here first before proceeding with the other one. Thanks!

Julian commented 1 week ago

(though there it shows workflow failed because of draft3 job failure which is happening because of no implementation (ruby and hyperjump in this case) supporting draft3 and hence none started successfully so it is exiting with some exit code and failing the entire job.

I'm not sure there's any point iterating over dialects here for this workflow, this is implementation-centric data, so I'd iterate over implementations, and for each implementation iterate over all its supported dialects, generating a per-implementation-per-dialect file across all versions.

As-is because you're doing this by dialect I think you end up redownloading the same images N times, where N is the number of supported dialects for the implementation.

adwait-godbole commented 1 week ago

I'm not sure there's any point iterating over dialects here for this workflow, this is implementation-centric data, so I'd iterate over implementations, and for each implementation iterate over all its supported dialects, generating a per-implementation-per-dialect file across all versions.

As-is because you're doing this by dialect I think you end up redownloading the same images N times, where N is the number of supported dialects for the implementation.

Thanks @Julian. This is definitely a better approach than this one.

adwait-godbole commented 1 week ago

What I am planning to do now is to modify our current report.yml workflow to download the artifacts that were published by this versioned-report.yml workflow's last run and every time upload it alongside other implementation reports which will be generated by report.yml itself. This way we have the traditionally generated reports for the latest versions of all implementations by report.yml and also alongside them we will have these versioned reports which were generated by versioned-report.yml workflow. This way report.yml does not have to do any heavy lifting of generating these versioned reports since we anyways will be manually triggering them once/whenever we need and storing them forever so report.yml just needs to download them and use them.

And what do you think about this @Julian ?

adwait-godbole commented 1 week ago

@Julian updated the PR. Here's the workflow run on my fork (dispatched manually).