UCL / openqcd-oneapi

GNU General Public License v2.0
0 stars 0 forks source link

Set up CI for SYCL code #11

Closed tkoskela closed 2 years ago

tkoskela commented 2 years ago

We could store the data sets on zenodo (like figshare). @JamieJQuinn and @ageorgou used this for larger test datasets for TROVE. This worked really well, allowed versioning of datasets. The datasets can be downloaded through the zenodo API (see https://github.com/Trovemaster/TROVE/blob/feature/unit-testing/test/regression/download_benchmarks.sh for an example). Didn't get round to caching in GHA but others have pointed to the cache action which is fab. Only problem we found with zenodo is it's really hard for multiple users to "own" a dataset so collaboration is tricky.

The size limit is 50Gb per dataset

krishnakumarg1984 commented 2 years ago

Ah. That reminds me of a few other options:

tkoskela commented 2 years ago

I've pushed the smaller *-16-16-16-16 and *-32-32-32-32 files to the repo via git lfs. I think those will suffice for some simple CI.