search

freelawproject / recap

This repository is for filing issues on any RECAP-related effort.
https://free.law/recap/
12 stars 4 forks source link

Create Integration Tests for the Extension #233

Open mlissner opened 6 years ago

mlissner commented 6 years ago

So far, we just have this page where we do QA tests manually, sometimes:

https://github.com/freelawproject/recap/wiki/QA-Testing

It'll be great to have something better.

jraller commented 5 years ago

Once https://github.com/GoogleChrome/puppeteer/issues/1215 is fixed it would be possible to use Puppeteer to set this up.

audiodude commented 5 years ago

The open question though was always how to handle the fact that you want to verify that data gets uploaded to the archive correctly without 1) paying money to PACER and 2) having to keep finding a new case to upload.

We had discussed previously using something like https://github.com/vcr/vcr to record PACER responses and "play them back" as a fake PACER server.

mlissner commented 5 years ago

FWIW, I've found using VCR for Juriscraper to be pretty bad. It means that the code you run during tests doesn't do what you expect it to some of the time because it gets played back from a "cassette" instead of actually hitting the network. That'd be fine if all the tests did that, but not knowing whether a particular test does has led to many frustrating debug sessions.

On Wed, May 1, 2019 at 8:45 PM Travis Briggs notifications@github.com wrote:

The open question though was always how to handle the fact that you want to verify that data gets uploaded to the archive correctly without 1) paying money to PACER and 2) having to keep finding a new case to upload.

We had discussed previously using something like https://github.com/vcr/vcr to record PACER responses and "play them back" as a fake PACER server.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/freelawproject/recap/issues/233#issuecomment-488544536, or mute the thread https://github.com/notifications/unsubscribe-auth/AABZ3KQYFUYHITX4B5FJS53PTJPUXANCNFSM4EMUNECQ .

-- Mike Lissner Executive Director Free Law Project https://free.law

audiodude commented 5 years ago

I don't quite understand, but I'll take your word for it.

mlissner commented 5 years ago

You wind up spending time trying to fix something you think is hitting the network when it's not. Seems like you'd catch that quickly, but in practice I've had a few rather frustrating hours lost to this.

johnhawkinson commented 5 years ago

I don't quite understand, but I'll take your word for it.

I ran into this when I had no idea about vcr or what it was and was trying to debug something in juriscraper after making a change. I was fooled into believe the tests were live when they weren't and wasted a lot of time.

This isn't something that would happen as easily if I'd known about vcr, and could probably be ameliorated with better developer documentation or printfs during the test runs..