Write tests for docs-crawler/docs/docs/spiders/numpy_spider.py

[tcp]

Description

docs-crawler/docs/docs/spiders/numpy_spider.py

Type of Test

• [x] Unit test (e.g. checking a loop, method, or function is working as intended)
• [ ] Integration test (e.g. checking if a certain group or set of functionality is working as intended)
• [ ] Regression test (e.g. checking if by adding or removing a module of code allows other systems to continue to function as intended)
• [ ] Stress test (e.g. checking to see how well a system performs under various situations, including heavy usage)
• [ ] Performance test (e.g. checking to see how efficient a system is as performing the intended task)
• [ ] Other

Expected Behavior

We expect this file to be covered completely by tests. It's not intended to test feature-related functionality but is instead intended to test the file in isolation.

Additional Context

None

[ateniolatobi]

Hello,

I ran through and tested numpy_spider.py and it looks like a rewrite would be needed.

The targeted link for crawling - https://docs.scipy.org/doc/numpy-1.17.0/reference/ has been removed by the Scipy team and the documentation of numpy has also been ported to a standalone site.

I also tried using the new documentation link with the crawler but the page has been restructured so the selectors used in the crawler don't work.

I'll rewrite the file and send a pull request today