search

andymatuschak / orbit

Experimental spaced repetition platform for exploring ideas in memory augmentation and programmable attention
https://withorbit.com
Other
1.71k stars 54 forks source link

Add scaffolding for API integration tests #230

Closed kirkbyo closed 3 years ago

kirkbyo commented 3 years ago

Added packages/api-integration. It holds the full-stack tests of the backend. For now, this package will only run locally since the tests require an instance of the backend (including Firestore) to be running. We could also put these tests in the backend package, but I figured a separate package would be easier to configure how we want the tests to run (or not to run in the case of CI).

I only created basic test cases for the /actionLogs route in this PR, but I will add more if you agree with this approach!

Note: I included a few API validation tests in this PR. Although this portion of the code is handled by Ajv, I see these tests as a contract to ensure no regressions down the line (instead of testing the internal Ajv). However, I could also see why we wouldn't want to pollute the code with validation tests, so let me know if you have any preferences!

kirkbyo commented 3 years ago

@andymatuschak Thank you for pointing me towards the firebaseSupport helpers, I had no idea those existed and that we were already spinning up the Firebase emulator in CI; neat!

Given that this infrastructure is already in the backend, I decided to move the tests to package/backend/integration and run them with the backend tests. This comes with a trade off that the tests need to be ran sequentially, --runInBand. Without this flag, the test become flaky due to running firestoreDatabaseBackend.test.ts and the integration tests due to firestoreDatabaseBackend.test.ts wiping the DB after each test (it was very flaky locally).

This works for now, but I suspect the test times might become too slow in the future, so at that point we could break out the integration tests into its own package and maybe store the firebaseSupport from within there? Not sure. Let me know if you have any issues with this approach!

kirkbyo commented 3 years ago

@andymatuschak I was a bit surprised as well, I expected some odd compilation errors, but it works locally on my end and it ran successfully in CI. I suspect it has something todo with the Babel config ignoring the included sources defined in the tsconfig. Not sure 🤔.

I will go ahead and merge and if we run into any issues, I will move the files into backend/src