search

edgi-govdata-archiving / web-monitoring

Documentation and project-wide issues for the Website Monitoring project (a.k.a. "Scanner")
Creative Commons Attribution Share Alike 4.0 International
105 stars 17 forks source link

abandon PageFreezer Diff API for now? #11

Closed dcwalk closed 7 years ago

dcwalk commented 7 years ago

From @titaniumbones on February 10, 2017 18:20

Given the limitations (10k hits/day, 5s/response, no batch processing) should we stop trying to use the PF diffapi for the time being, and maybe revisit this question later on? We can still use their api''s JSON format for our own outputs if we want, or we can add the additional diff streams that @lh00000000 has been talked about. Then if we decide to go back to their format later on, we can drop their differ in as a replacement for ours.

Copied from original issue: edgi-govdata-archiving/web-monitoring-ui#20

dcwalk commented 7 years ago

From @danielballan on February 14, 2017 20:50

I think a rag-tag team of volunteers will have trouble beating their performance when all things are considered, particularly given this comment. But it definitely doesn't hurt to design this in a way that's pluggable.

dcwalk commented 7 years ago

My understanding is this is a stale issue: we are currently making use of PageFreezer's API in our design.

Closing for now, feel free to reopen if this needs to be revisited now

titaniumbones commented 7 years ago

yeah, agreed, thx @dcwalk