TeamHG-Memex / autologin-middleware

Scrapy middleware for the autologin
37 stars 15 forks source link

Add integration tests, support scrapy #2

Closed lopuhin closed 8 years ago

lopuhin commented 8 years ago

The only tricky bit here is 47750d4: I discovered that the logic for logout detection was slightly wrong. Before, when a logged-out request arrived, we thought that this was the request that caused logout, and did not retry it, but retried all next requests. But the actual logout request could arrive a little later in fact, and we could incorrectly mark a different request as logout. To fight this I allow each request be retried a couple of times, and if it is logged out all the time, then we don't repeat it. Middleware is tested without splash here. Splash tests are still done in the undercrawler, not sure if it makes sense to add them here too. It's not a lot of new code, just much slower build time.

codecov-io commented 8 years ago

Current coverage is 84.27%

Branch #2 has no coverage reports uploaded yet.

No diff could be generated. No reports for master found.

Powered by Codecov. Updated on successful CI builds.