search

arquivo / pwa-technologies

Arquivo.pt main goal is the preservation and access of web contents that are no longer available online. During the developing of the PWA IR (information retrieval) system we faced limitations in searching speed, quality of results, scalability and usability. To cope with this, we modified the archive-access project (http://archive-access.sourceforge.net/) to support our web archive IR requirements. Nutchwax, Nutch and Wayback’s code were adapted to meet the requirements. Several optimizations were added, such as simplifications in the way document versions are searched and several bottlenecks were resolved. The PWA search engine is a public service at http://archive.pt and a research platform for web archiving. As it predecessor Nutch, it runs over Hadoop clusters for distributed computing following the map-reduce paradigm. Its major features include fast full-text search, URL search, phrase search, faceted search (date, format, site), and sorting by relevance and date. The PWA search engine is highly scalable and its architecture is flexible enough to enable the deployment of different configurations to respond to the different needs. Currently, it serves an archive collection searchable by full-text with 180 million documents ranging between 1996 and 2010.
http://www.arquivo.pt
GNU General Public License v3.0
38 stars 7 forks source link

Integration revisionista on Arquivo.pt user interface #981

Open igobranco opened 3 years ago

igobranco commented 3 years ago

What is the URL that originated the issue? Any wayback URL. https://arquivo.pt/wayback/20170927135355/https://www.jn.pt/nacional/interior/motas-inspecao-obrigatoria-a-partir-de-outubro-5167876.html

What happened? On replay options, next to Replay with old browser, add a link to revisionista.pt. But only add that link if the archived page exists on this edited list of revisionita. https://revisionista.pt/en/articles-url/?edit=True Redirect to https://revisionista.pt/pt/articles/reader/

For performance reasons and not to stress the revisionista servers, it's needed to create a simple webapp that periodically downloads the edited revisionista list and put available a path that returns true or false if the archived URL is on the revisionista list.

dcgomes commented 3 years ago

List of articles with editions identified by Revisionista.pt in real time: https://revisionista.pt/en/articles-url/?edit=True

amourao commented 2 years ago

Maybe instead of going directly to Revisionista's article view, show link to a page with all the edits for that URL (to be created in Revisionista) and have the user learn how to use Revisionista from there. image