search

edgi-govdata-archiving / archivers-harvesting-tools

ARCHIVED--Collection of scripts and code snippets for data harvesting after generating the zip starter
GNU General Public License v3.0
32 stars 28 forks source link

Add a location for very short single-purpose scripts #6

Closed mhucka closed 7 years ago

mhucka commented 7 years ago

I have someone who wrote a short script to harvest data from a specific page (at DOE, I think). The script is pretty specific to the particular page & purpose, so it (probably) doesn't merit a whole subdirectory + readme file and so on. At the same time, for the purposes of good reproducibility and documentation, it seems like such things ought to be kept somewhere, and it might serve as an example for others.

Perhaps there could be a subdirectory for such things? Something like "single-purpose-scripts" or "site-specific-scripts" or something like that? Each script could be placed as a single file there, and there could be a single readme file that contains a paragraph about each file in the directory.

datarocks commented 7 years ago

I think a utils subfolder with scripts within that, and maybe an entry for each script in the readme.md for the folder?

titaniumbones commented 7 years ago

Yeah, that sounds great, and if you can also add brief text to the readme that would be great.

On February 3, 2017 8:31:47 PM EST, Mike Hucka notifications@github.com wrote:

I have someone who wrote a short script to harvest data from a specific page (at DOE, I think). The script is pretty specific to the particular page & purpose, so it (probably) doesn't merit a whole subdirectory + readme file and so on. At the same time, for the purposes of good reproducibility and documentation, it seems like such things ought to be kept somewhere, and it might serve as an example for others.

Perhaps there could be a subdirectory for such things? Something like "single-purpose-scripts" or "site-specific-scripts" or something like that? Each script could be placed as a single file there, and there could be a single readme file that contains a paragraph about each file in the directory.

-- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/edgi-govdata-archiving/harvesting-tools/issues/6

-- Sent from my Android device with K-9 Mail. Please excuse my brevity.

titaniumbones commented 7 years ago

resolved in 340768d. @mhucka if you want to put that script there... and also @datarocks if you have stuff?

datarocks commented 7 years ago

I don't yet, but I will!

dcwalk commented 7 years ago

Closing because now we have a utils folder!