Added vcr support for consistent tests - Githubissues

johnwmillr / LyricsGenius

Download song lyrics and metadata from Genius.com 🎶🎤

http://www.johnwmillr.com/scraping-genius-lyrics/

MIT License

898 stars 159 forks source link

Added vcr support for consistent tests #109

Open ludehon opened 5 years ago

ludehon commented 5 years ago

Only tests functions that use HTTP will have the vcrpy decorator

johnwmillr commented 4 years ago

Hi @ludehon, sorry for the delay. Do you mind adding a demonstration of how to run these VCR tests?

allerter commented 3 years ago

Reading the Rationale on VCR's docs, I think this might be a good idea. Currently, the tests take about 3:20 minutes on Travis CI and even longer on local machines.

allerter commented 3 years ago

Using VCR without the libyaml extension the tests took 105 seconds on my PC. It's also nice to be able to run the tests offline.

johnwmillr commented 3 years ago

Do you mind addressing these merge conflicts, @Allerter?

allerter commented 3 years ago

@johnwmillr, not at all. I had actually tried this out a while ago. So this won't take much time.

allerter commented 3 years ago

A few notes:

The format of the cassettes can be changed to JSON when initiating test_vcr by changing serializer='yaml'. But for some of the tests, I explicitly set the serializer to YAML because those tests directly or indirectly call genius.lyrics or genius.tag which the response of its request is binary and JSON can't normally encode that (unless we make our own serializer for vcrpy.)
These cassettes only work from the second time tests are run, since at the first run vcrpy records the cassettes. So Travis CI won't be using recorded cassettes. This won't reduce test time on Travis CI but has two advantages: 1. It's best that Travis CI test all endpoints live to check if an endpoint has become unavailable. 2. Putting the recorded cassettes will increase the size of the repository, especially more since we're downloading whole HTML pages when using genius.lyrics and genius.tag.
Although the recorded cassettes won't be in the repo, I still added filtering the authorization headers to avoid revealing them even in local machines.

allerter commented 3 years ago

[x] Ready to be merged

allerter commented 3 years ago

@johnwmillr if you think using vcrpy isn't worth the trouble, you can close this pull request. It's not necessary and it's only useful when testing locally.

johnwmillr commented 3 years ago

You're right, @Allerter. For the sake of moving forward with 3.0, let's hold off on this PR for now. I won't close it just yet, but we can move forward with the 3.0 work without merging this in.