neurobagel / query-tool

User interface for searching across Neurobagel graph
https://query.neurobagel.org/
MIT License
2 stars 1 forks source link

toast-alerts popping up due to unrelated error can break e2e tests #98

Open surchs opened 6 months ago

surchs commented 6 months ago

This is a bit of an odd one that points to a bigger problem with the tests:

If I run an e2e test like https://github.com/neurobagel/query-tool/blob/c890a4a5a2c221993b8240ff106b44847df4ba2c/cypress/e2e/Alert.cy.ts and don't mock all the required remote data (as we currently do), and some of the required remote data fails (e.g. is unavailable or in the wrong form), then I will get the pretty error toasts like so:

image

If they happen to cover up an element that I expect to be able to interact with, then cypress will cleverly error out.

That's the fyi part. The "what now" part is: we should not rely on any running remote API being available to ensure our e2e tests pass. We don't yet have any real cross-app integration e2e tests. And changing the response model like we do in #60 is a good example where this reliance on a production API really comes to bite us.

So the short answer is: no more requests to prod APIs in e2e tests. Short term: mock it all. Longer term: have a dedicated test stack of all the services our app requires to talk to

github-actions[bot] commented 3 months ago

We want to keep our issues up to date and active. This issue hasn't seen any activity in the last 75 days. We have applied the _flag:stale label to indicate that this issue should be reviewed again. When you review, please reread the spec and then apply one of these three options: