Redirects to web archives should be indexed appropriately. - Githubissues

ukwa / ukwa-manage

Shepherding our web archives from crawl to access.

Apache License 2.0

10 stars 5 forks source link

Redirects to web archives should be indexed appropriately. #103

Open anjackson opened 1 year ago

anjackson commented 1 year ago

In ukwa/ukwa-pywb#95 a (broken) redirect to the web archive was causing problems. But self redirects cause other problems e.g. see ukwa/ukwa-pywb#72

Ideally, redirects into UKWA should be filtered out before being added to the CDX index.
Redirects to other web archives should ideally direct the user to those web archives.

It's not clear if either of these are particularly straightforward.