Cleanup scrapers - Githubissues

research-software-directory / RSD-as-a-service

This repo contains the new RSD-as-a-service implementation

https://research.software

23 stars 15 forks source link

Cleanup scrapers #1094

Closed ewan-escience closed 7 months ago

ewan-escience commented 7 months ago

Scraper cleanup

Changes proposed in this pull request:

Move (unused) tests that are actually integration tests to dedicated classes
Cleanup the Config class (typos, unused code, etc.)
Wrap forgotten HttpClient in try-with-resources

How to test:

docker compose down --volumes && docker compose build --parallel && docker compose up --scale data-generation=1
Wait for some scrapers to run

PR Checklist:

[ ] Increase version numbers in docker-compose.yml
[ ] Link to a GitHub issue
[ ] Update documentation
[ ] Tests

sonarcloud[bot] commented 7 months ago

Quality Gate failed for 'scrapers'

Failed conditions

C Maintainability Rating on New Code (required ≥ A)

See analysis details on SonarCloud

idea Catch issues before they fail your Quality Gate with our IDE extension SonarLint