Essential Diffing Tools for v0

danielballan commented 7 years ago

The minimum set of diffs views that need to be available:

Source diffs (i.e. the easiest possible thing)
Diffs of visible text content only (subtle, but a simple approach is enough to start)
Side-by-side comparisons of the rendered pages

This is according to @ambergman and generally fits in with the consensus view I've heard.

Mr0grog commented 7 years ago

Some implementation notes:

Source diffs: We have the diff we need with go-calc-diff (proxied through DB as the source type)
Diffs of visible/readable/semantic text: We don’t have a differ that does this yet (though @b5 noted he might have some code that does an OK job he could add to go-calc-diff)
Side-by-side: To do as good as Versionista, we have all we need, which is just the raw version response bodies (we just need to stuff a <base> element in the <head> so images/scripts/etc. work). We should aim for something better or more complex with PF, though—the Versionista approach breaks down as a page gets older and the associated resources (images/scripts/etc) disappear from the web.

(We could do a lot more work with the Versionista scraper to record versions of those resources ourselves, but that doesn’t strike me as worthwhile right now).

Mr0grog commented 7 years ago

For all of these, we have no implementation in -ui yet. See: https://github.com/edgi-govdata-archiving/web-monitoring-ui/issues/60

Mr0grog commented 7 years ago

Also: we need to coordinate with @b5 on how go-calc-diff is deployed: it lives as a randomly named obscure Heroku app right now; do we keep it that way? Beef it up? Re-deploy on some other infrastructure?

(None of these are major issues for the current level of use; but are important to consider for post-v1.)

Mr0grog commented 7 years ago

@danielballan Obviously we have problems with side-by-side, but can we close this? I feel like that’s better covered in more specific issues and this floating around is more noise than help now.

danielballan commented 7 years ago

Yes.

edgi-govdata-archiving / web-monitoring

Essential Diffing Tools for v0 #43