We should move the validation and deduplication options into the settings file to allow us to tweak them easier or on the fly (maybe also building in dotenv support).
Among these would be something like site size as well as the min/max distance between the sites to be considered a duplicate.
We should move the validation and deduplication options into the settings file to allow us to tweak them easier or on the fly (maybe also building in dotenv support).
Among these would be something like site size as well as the min/max distance between the sites to be considered a duplicate.