search

data-liberation-project / aphis-inspection-reports

Inspection data and PDFs from the USDA's Animal and Plant Health Inspection Service.
13 stars 3 forks source link

Upload data folder to GitHub Pages to complete #7 #45

Closed palewire closed 1 year ago

palewire commented 1 year ago

This will publish the entire contents of our data folder to GitHub Pages, which is an expansive solution. If you'd like to more narrowly tailor what we send out let me know and I can revise the patch. I figured, "What the hell?"

jsvine commented 1 year ago

Hmm, interesting! My instinct says that adding all of data/ is the simplest approach, with the least future-fiddling required. But worth thinking through the potential downsides. The main one that occurs to me: Is this going to make updating GitHub Pages frustratingly slow? (I think we're talking 100k+ files ... most of which won't change between commits, but I don't know enough about the Pages infrastructure to know whether that matters.) Anything else we should consider? And what about just pushing data/combined?

palewire commented 1 year ago

It will be slow because it has to upload everything to the artifact. Limiting to combined would be the compromise, yes. Not clear me how big the difference would be. One way to find out!

jsvine commented 1 year ago

Merging (and thanks again for this), but I think will make that tweak to data/combined. Just feels a little cleaner and more intentional. But open to flipping it back!