Integration tests - Githubissues

q-m / scrapyd-k8s

Scrapyd on container infrastructure

MIT License

12 stars 7 forks source link

Integration tests #21

Closed wvengen closed 8 months ago

wvengen commented 8 months ago

[x] Add initial tests
[x] CI for Docker (running locally)
[x] CI for Kubernetes (running locally)
[x] Fix jobs on Kubernetes CI being stuck in scheduled
[x] CI for running scrapyd-k8s in Kubernetes (testing kubernetes.yaml)
[x] Add spider to example spider project that returns static data (or run from cache)
[x] ~~Finish tests~~ --> as part of other issues

Possible next steps:

Test job cancel while job is pending (needs something to create a job without starting it)
Test http auth (needs config file change and daemon restart, so perhaps different method)
Test if values in secret and configmap are passed through to the spider (*)
Test if pull secret is picked up
Test if schedule spider arguments are passed through to the spider (*)
Test if schedule spider settings are passed through to the spider (*)

(*) May need changes to the example spider, e.g. returning ENV, spider settings and spider arguments in static spider or a new debug spider or so.