search

18F / domain-scan

A lightweight pipeline, locally or in Lambda, for scanning things like HTTPS, third party service use, and web accessibility.
Other
371 stars 137 forks source link

Refactoring scan #218

Open JasonMWhite opened 6 years ago

JasonMWhite commented 6 years ago

I'd like to see domain-scan be able to be packaged up and installed via pip. While this is certainly doable, to do it right would (IMO) require some fairly risky surgery to the internals. Since 18F is relying on this in prod, I wouldn't be able to tackle this myself in its current state.

Instead, I propose a series of smaller refactors, specifically aimed initially at breaking up scan and adding tests. Once a suitable test suite starts to form, it will slowly become easier to tackle the larger obstacles.

I'd also like to suggest introducing type annotations as we go. I've found it increases code readability and quality significantly, especially when combined with mypy to make sure they stay up to date.

@konklone thoughts?

konklone commented 6 years ago

@JasonMWhite I am massively supportive of this. We are using it in prod, but we'd get a lot of benefit out of making this a more testable and modular project. (scan, in particular, has evolved from a relatively small shim to a very interesting task management system, and is in need of a breakup.)

I'll cc @tadhg-ohiggins as well, who has been spending the last week getting up to speed on the project, with an initial focus on adding tests and paying down technical debt. In general, we are willing to prioritize our work around seeing domain-scan made more reusable and maintainable across organizational boundaries.

JasonMWhite commented 6 years ago

Great! Sounds like we're on the same page then. It's a project of great interest to us at CDS. I'll proceed with some initial PRs to get started.

/cc @obrien-j @sastels