Cyclic dependencies in existing modules

[coryan]

Problem

When I moved the REST service implementation to the testbench directory I created a cyclic dependency. Basically testbench needs testbench/rest_server.py (for the run()) function, which needs gcs/bucket.py, which needs testbench (for all the helper functions).

Proposal

I think we need to redefine the layers for the testbench, they were already kind of fuzzy, and my breakage makes it worse. I think we should organize the code in the following layers.

Name	Description	Depends on
testbench	The top-level layer, exposes the run() function for gunicorn and main	testbench.servers
testbench.servers	The implementation of the REST and gRPC APIs. Flask routes and gRPC servicers.	testbench.database, testbench.utility, google.storage.v2
testbench.database	The database of resources (Bucket, Object, *AcccessControl, etc.)	testbench.utility
testbench.utility	Helper functions	google.storage.v2
google.storage.v2	Proto generated files

Where do things move?

The grpc_server.py and rest_server.py files move to testbench.servers. We will not split rest_server.py as part of this work.

The current gcs module would move to testbench.database.

For the most part, other testbench/*.py files move to testbench.utility.

Breakdown

Some of these are tentative, but the general idea would be to break this down into steps.

• [ ] Move testbench/error.py to testbench/utility/error.py
• [ ] Split testbench/common.py snake case functions to testbench/utility/*
• [ ] Move testbench/acl.py to testbench/utility/
• [ ] Split testbench/project.py, move the server-like things to testbench/servers/ and the rest to testbench/utility/.
• [ ] Move testbench/csek.py to testbench/utility/
• [ ] Move testbench/generation.py to testbench/utility/
• [ ] Move testbench/handle_gzip.py to testbench/servers/
• [ ] Move gcs/*.py to testbench/database/*.py
• [ ] Move testbench/database.py to testbench/database/
• [ ] Move testbench/grpc_server.py to testbench/servers/
• [ ] Move testbench/rest_server.py to testbench/servers/

[unforced]

Sweet, this feels much cleaner, thanks for proposing this and giving the clear breakdown. I'll look a bit deeper and see if there's any other feedback I can add, or possible suggestions, but this definitely feels good to me as you've outlined it.

[coryan]

Thanks. Realistically I won't be able to work on this for several weeks. If anybody wants to improve things, please go ahead.