search

research-software-directory / RSD-as-a-service

This repo contains the new RSD-as-a-service implementation
https://research.software
22 stars 15 forks source link

Allow admins to disable scraping #1117

Open ewan-escience opened 5 months ago

ewan-escience commented 5 months ago

Some things being scraped will always fail, e.g. a non-public GitLab repo. These will, currently, keep on being scraped forever, generating errors and delaying other things to be scraped. It would be nice if an admin can disable certain things to be scraped (based on the error logs).