algolia / docsearch-scraper

DocSearch - Scraper
https://docsearch.algolia.com/
Other
308 stars 107 forks source link

Add check for "unknown" parameters in DocSearch config #326

Open JanPetr opened 7 years ago

JanPetr commented 7 years ago

I had a min_indexed_lvl parameter in a config, but the parameter is actually called min_indexed_level. But DocSearch scraper crawled the site as usual and I had no idea there is something wrong.

Is it possible to add a check which would raise a warning message when there are “unknown” parameters in the config?

ElPicador commented 7 years ago

Yeah you are right, I had the same issue :/

We have some validations: https://github.com/algolia/docsearch-scraper/blob/master/scraper/src/config/config_validator.py

It should be pretty easy to had more validations.