mzgubin
commented
6 years ago - Changes that improve resiliency like catching some of the errors encountered to keep the jobs running. It would be great if that was perhaps put into a configuration whether to fail-fast or just log those errors as warnings and go on. I don't know enough about the best approach for something like that would be.
- Converted the servlet on the source being pulled from to handle post requests instead of get. That way the requests won't be limited by the GET URL length. This can be useful if there are a ton of exclusions as we've encountered.
- Also I made a change to how the exclusion filters are matching the paths. Currently the paths will be searched to find the exclusion string in any part of the path, instead I've made it so that the string is matched against just the startsWith instead. Not sure if that's how it's intended to be used, but perhaps that can also be configured somehow in the exclusion filter section itself.
- Added more debug logging