github-education-resources / classroom

GitHub Classroom automates repository creation and access control, making it easy for teachers to distribute starter code and collect assignments on GitHub.
https://classroom.github.com
1.34k stars 564 forks source link

Moving off of VCR #2372

Open d12 opened 5 years ago

d12 commented 5 years ago

VCR is the single biggest development pain point when working on GitHub Classroom. It took me forever to get working, and most developers who start working on Classroom run into issues with it. Creating tests means creating actual records on GitHub.com, and you can't make tests if you're offline, your credentials are incorrect, or your plan isn't sufficient (You can't make private repos in an org unless it's upgraded!).

What else can we use, and what are the pros/cons?

Possible alternatives:

cc @education/classroom-reviewers

spinecone commented 5 years ago

In addition to the concerns that @d12 has mentioned, as someone new to this codebase, VCR has made it difficult to understand and update existing tests (what API requests is this behavior making? is my change doing something bad to the existing requests? I don't know because the answer lies is an unintelligible 7000 character file).

My personal strategy in avoiding VCR has been:

Whenever a spec attempts to make an API request, either stub the request itself using stub_request or if possible stub an entire method to return an expected value (assuming the method itself is fairly simple and safe to stub out).

Me attempting to use stub_request recently: https://github.com/education/classroom/pull/2370/files#diff-92efe40a73d0ab51b966bb0d40fc3e29

A PR in which I stubbed out entire view partials rather than allowing them to render, which would make additional API requests that the spec was not concerned with: https://github.com/education/classroom/pull/2289/files#diff-7bcb208f8378d8efd162baf8920fe51dR12

A long term solution I'd like would be for tests to never make actual API calls and to write helper methods that stub out specific requests we expect to see with expected results (the stubbed methods that education-web uses are a good example https://github.com/github/education-web/blob/master/spec/spec_helper.rb#L285)