mathesar-foundation / mathesar

Web application providing an intuitive user experience to databases.
https://mathesar.org/
GNU General Public License v3.0
2.28k stars 316 forks source link

Refactor list of RPC methods to test #3597

Closed seancolsen closed 1 month ago

seancolsen commented 1 month ago

This PR cleans up the code in test_endpoints.py so that it's easier to add new RPC methods to that file.

This is its own PR because:

  1. It would be nice to get this merged soon so that we can all work on top of it.
  2. I ran into some weird test failures that I want help troubleshooting in isolation.

Checklist

Developer Certificate of Origin

Developer Certificate of Origin ``` Developer Certificate of Origin Version 1.1 Copyright (C) 2004, 2006 The Linux Foundation and its contributors. 1 Letterman Drive Suite D4700 San Francisco, CA, 94129 Everyone is permitted to copy and distribute verbatim copies of this license document, but changing it is not allowed. Developer's Certificate of Origin 1.1 By making a contribution to this project, I certify that: (a) The contribution was created in whole or in part by me and I have the right to submit it under the open source license indicated in the file; or (b) The contribution is based upon previous work that, to the best of my knowledge, is covered under an appropriate open source license and I have the right under that license to submit that work with modifications, whether created in whole or in part by me, under the same open source license (unless I am permitted to submit under a different license), as indicated in the file; or (c) The contribution was provided directly to me by some other person who certified (a), (b) or (c) and I have not modified it. (d) I understand and agree that this project and the contribution are public and that a record of the contribution (including all personal information I submit with it, including my sign-off) is maintained indefinitely and may be redistributed consistent with this project or the open source license(s) involved. ```
seancolsen commented 1 month ago

@mathemancer or @Anish9901 could you look at this and help me troubleshoot the test failure?

These both work:

docker exec mathesar_service_dev pytest -k test_rpc_endpoint_expected_methods
docker exec mathesar_service_dev pytest -k test_correctly_exposed

But when the tests are run together this fails:

docker exec mathesar_service_dev pytest mathesar/tests/rpc/test_endpoints.py

I don't understand what's going on. I can continue troubleshooting on my own if you're confused too. But I figured one of you might have a quick guess as to the cause.

seancolsen commented 1 month ago

Came back to check on this... Now I see that the tests are passing in CI. Weird. 🤷‍♂️ 😬

mathemancer commented 1 month ago

Came back to check on this... Now I see that the tests are passing in CI. Weird. 🤷‍♂️ 😬

Long story short, the Django test DB setup doesn't play nicely with test parallelization. So, in CI, we just run the failed tests until everything is passing, or until some test fails 5 times. This, along with the parallelization, is faster than running the tests serially.

As we progress to the portion of the refactor where we're removing vast swathes of code, and associated tests, I want to move these back to running serially, and eventually we should only need the Django test DB setup quite rarely (e.g., for E2E testing).

For now, just re-run with the --lf flag, and only believe failing tests if they fail multiple times.