When testing the CNN scraper utilities we wrote a little script that would generate a slew of tests from a URL and package the inputs / outputs of various steps which could then be invoked as part of a test suite. I'm realizing there might actually be some value in using a similar methodology to test the actual CNN scraper, which means I'm revisiting those scripts and starting to make them more generic.
The functionality here is getting to the point that it might actually be something that could exist as its own developer tool / npm module. The value of this approach is:
It would be usable across other projects.
It would be explicitly isolated from our code / the functionality would be documented on its own without complicating our documentation.
It could be managed and maintained outside of just this project.
When testing the CNN scraper utilities we wrote a little script that would generate a slew of tests from a URL and package the inputs / outputs of various steps which could then be invoked as part of a test suite. I'm realizing there might actually be some value in using a similar methodology to test the actual CNN scraper, which means I'm revisiting those scripts and starting to make them more generic.
The functionality here is getting to the point that it might actually be something that could exist as its own developer tool / npm module. The value of this approach is:
Curious what others think.