Unflexible Server Testing Setup

[BlobbyBob]

The current server testing setup has some disadvantages:

1. Requires postgres to be installed not only on database system, but also on server system (for pg_restore)
2. Pretty slow (at least on my system)
3. Database dump is in custom postgresql binary format and impractical for versioning and manual editing
4. When altering the models, the tests will fail as the layout does not include the latest migrations
5. From the outside it is hard to see what is inside the DB. Essentially, you need to restore the dump once to really explore the available testing data

I tried to realize some quick improvements over in my fork, but I think it would be better to make a larger restructure:

• Get rid of pg_restore and instead write a setup script which adds all entities and their relations to a clean database which was set up using all latest migrations. Then you only need to exchange the SQLALCHEMY_DATABASE_URI to switch to the test setup. (fixes 1, 3, 4, 5)
• Focus the tests on units, not on integration. I guess the tests are slow, because it uses only the HTTP endpoints for everything. However, it does not provide any benefit to test the authentication system before every single test in my opinion. I would suggest test cases similar to this pseudo code (fixes 2 probably):

  
  def get_token(role):
  return sign_jwt({role: True), SECRET_KEY)

def test(): line_data = { ... } rv = client.post("/api/lines", json=line_data, auth=get_token("moderator")) assert rv.status_code == 42 line = Line.find_by_slug(line_data["slug"]) assert line is not None # ... and some properties

def test2(): rv = client.post("/api/archive", json={"line":"some-line"}, auth=get_token("moderator")) assert rv.status_code == 42 line = Line.find_by_slug("some-line") assert line.archived == True

So ideally the amount of HTTP requests required per test case would be reduced to 1.

[BlobbyBob]

Correction: The tests are slow because the client fixture completely restores the database on every test. Maybe there is a nice solution for that.

[dorthrithil]

Nice, tests running in 3s now on my machine

[dorthrithil]

Wondering why I can't link 1.7.2 here so this get's auto closed after merge, mhh. Can you select it @BlobbyBob?

[BlobbyBob]

No, it's also disabled for me

[dorthrithil]

Strange

[BlobbyBob]

A stack overflow post hints, that you might not be able to link more than 10 issues to a PR. And indeed, v1.7.2 has currently 10 linked issues

[dorthrithil]

Strange decision of the GitHub team, but good to know.