Closed kevinschaper closed 1 year ago
Can we figure out if the mapping tables are all <50MB in size? would be nice if we could use GitHub for versioning...
Ahh, GitHub file size limit! I'd been working with GitLab for so long I'd forgotten!
I'm not sure about all of them, but the version I just produced in testing is about 30MB, so for the time being this should be ok. What do you think @kevinschaper ?
If it gets too big we can definitely break into multiple files.
We could experiment with doing it all with github even and skipping the Jenkins completely?
I think having as much as possible done with GitHub and GitHub actions increases transparency a lot!
We should maybe use 'gene-mappings' as the directory in the buckets, with a dated directory and a latest in each one.
I'm not sure about upload locations, definitely gs://data-public-monarchinitiative and gs://monarch-archive. I'm not sure if it makes sense to upload to gs://monarch-ingest.
We could also consider writing to date-named GitHub releases... (and then we can bring that over to monarch-ingest)