search

clarin-eric / resource-availability-status-api

0 stars 1 forks source link

set host for links to be checked #19

Closed wowasa closed 3 years ago

wowasa commented 3 years ago

the current API doesn't set hosts. This field is necessary since the stormcrawler (linkchecker) is parallelizing the url calls by host.

Therefore a host must be set for every link to check

wowasa commented 3 years ago

host field removed from database. The key for grouping (in Clarin case grouping is done by host) is identified from the URL at processing time