Feed pipeline directly from chrome extension

edgi-govdata-archiving / archivers.space

🗄 Event data management app used at DataRescues

https://www.archivers.space/

GNU Affero General Public License v3.0

6 stars 3 forks source link

Feed pipeline directly from chrome extension #15

Open kmcculloch opened 7 years ago

kmcculloch commented 7 years ago

From @titaniumbones on February 5, 2017 17:34

If we could feed the pipeline directly from the chrome extension, seeders at events could provide dataset URL's for use on that day, even if @b5 and @danielballan are not physically present. So... that seems like a very substantial improvement. Here' are some todo items for that -- happy to add to README if feature seems attainable/worth pressing towards.

[ ] implement a GET interface that accepts an entry from the Chrome extension & creates a new mongodb record
[ ] implement a security model for dealing w/ extension users -- either by marking URL's as chrome-seeded, or by adding authentication to the extension
[ ] decide whether we want the app to handle all nominations, even URLS going only to the IA. If so, figure out how to manage the IA seeds. If not, decide on criteria for sending URL to one storage location, the other, or both.

I'm sleepy! hope y'all are getting some rest!

Copied from original issue: b5/pipeline#40

kmcculloch commented 7 years ago

From @titaniumbones on February 6, 2017 3:43

just fyi @danielballan I am rewriting docs as though this feature had already been implemented... no pressure 😸

kmcculloch commented 7 years ago

From @danielballan on February 6, 2017 3:49

Heh heh. Aspirational documentation!

I agree that this is probably the most important feature to get right next. Can aim to make it happen this week.

kmcculloch commented 7 years ago

From @dcwalk on February 6, 2017 4:5

RE documentation, how should the revision proceed? Did you mention something about building some into the app?

I'm excited for @ebarry to do a pass on the event docs, maybe the updating should happen in sync?

kmcculloch commented 7 years ago

From @titaniumbones on February 6, 2017 5:10

(sorry, I was behind on this). I did a rewrite of the workflow, but will not push or house in the DR repo so as not to cause an explosion. However, you can see it here: https://github.com/edgi-govdata-archiving/workflow/tree/WIP-pipeline-refactor

kmcculloch commented 7 years ago

From @atesgoral on February 7, 2017 14:35

Ideally: GET -> POST since a new record is being created.

kmcculloch commented 7 years ago

From @danielballan on February 7, 2017 15:27

Did you mention something about building some into the app?

I think the app should include:

a little more explanatory text inline and/or more careful wording in reaction to points of confusion we've seen in users
a link to workflow text that is app-specific and not event-specific; this could live in the app codebase once the wording is stable enough

kmcculloch commented 7 years ago

From @titaniumbones on February 7, 2017 16:16

Yes. As I was rewriting the workflow doc, it occurs to me that there are really two differnt things:

a technical spec for successful chain-of-custody archiving
a guideline for event participants

We've been conflating the two.

Dan Allan notifications@github.com writes:

Did you mention something about building some into the app?

I think the app should include:

a little more explanatory text inline and/or more careful wording in reaction to points of confusion we've seen in users

a link to workflow text that is app-specific and not event-specific; this could live in the app codebase once the wording is stable enough

sonalranjit commented 7 years ago

Just continuing the conversation started by mattprice recently, @danielballan suggested have a url staging area in the app, i will try and get that setup first if that is okay with you guys?