elixir-cloud-aai / TESK

GA4GH Task Execution Service Root Project + Deployment scripts on Kubernetes
https://tesk.readthedocs.io
Apache License 2.0
39 stars 28 forks source link

Refactor repository structure #152

Closed uniqueg closed 1 month ago

uniqueg commented 1 month ago

Follow basic structure of other FOCA apps, e.g., proTES and Cloud Registry.

In particular:

JaeAeich commented 1 month ago

The current file str

.
├── charts
│   └── tesk/
├── cloudbuild_testing.yaml
├── cloudbuild.yaml
├── containers
│   ├── filer.Dockerfile
│   └── taskmaster.Dockerfile
├── coverage.xml
├── doc
│   └── taskmaster_architecture.png
├── dockerBuild -> scripts//dockerBuild
├── dockerRun -> scripts//dockerRun
├── documentation/
├── examples/
├── init
├── install
├── LICENSE
├── MANIFEST.in
├── pytest.ini
├── README.md
├── scripts
│   ├── dockerBuild
│   ├── dockerRun
│   └── run
├── setup.cfg
├── setup.py
├── src
│   └── tesk_core/
├── taskmaster
├── tests/
└── tox.ini

32 directories, 109 files

THOUGHTS?

PS: Please look at the current main branch (commit ffac3ef23eeb827574bc54a99f37aceda89374b6) for more info. PPS: Aah, I see some of my concerns are addressed in other issues, but the questions remain the same, any opinions will be welcome :).

uniqueg commented 1 month ago

How about we remove all the scripts and have one make file of the entire repo, if we have specific commands for api or core we can always have a naming convention of the command that makes sense.

I don't really know how these scripts are used. There's probably room for improvement here and your proposal sounds reasonable. However, I don't really know how and where these scripts are currently used. In any case, I don't think it's a high priority issue right now and how to best organize these scripts (or the commands in them) will probably become clear(er) later on, so I wouldn't worry about that now. But perhaps @lvarin, @jemaltahir and/or @trispera have more insights here.

Use [poetry](https://python-poetry.org/docs/) for package management, I've used poetry and its far better and easier. Remove setup.py and various other configs that can be pushed together in pyproject.toml.

Sounds good, please go ahead with that.

Rename src as TESK and house tesk_core and python based tesk_api.

I would not use capital letters for the package name, it's kinda against Python conventions. So rather use tesk. Also, I wouldn't use tesk_core and tesk_api subdirectories inside of tesk. Rather, I would reorganize tesk-core under services and otherwise keep the same structure that other FOCA apps (proTES, Cloud Registry, TRS-Filer etc.) are using.

tests/ contains tests for tesk_core house that in tests/tesk_core/.

Again, I would rather use the same structure that other FOCA apps are using, and they basically reproduce the package structure from the app. So instead of tesk/services, it would be tests/test_services/test_service_x/test_module_y.py (check pytest's conventions for test discovery). Or, if you want to separate unit from integration tests, use tests/test_unit/test_services/test_service_x/test_module_y.py.

Create a new directory deployment and push charts, documentation and examples in it.

Yes, just follow how this is organized in proTES etc. However, if you don't mind the clutter for now, you can also leave this for now and address this in #164 towards the end. This would have the advantage that it might actually still all be roughly functional without having to change anything. I don't have a strong opinion either way, as long as it's nice and clean in the end :)