elixir-cloud-aai / TESK

GA4GH Task Execution Service Root Project + Deployment scripts on Kubernetes
https://tesk.readthedocs.io
Apache License 2.0
39 stars 29 forks source link

refactor: redesign filers #179

Open uniqueg opened 4 months ago

uniqueg commented 4 months ago

Current object filer design, as inherited from TESK-core), is a bit messy. To make it easier to add support for different file protocols in the future, it would be nice to clean up the filer-related code and make it a bit more pluggable.

For example, we could have the following directory layout:

.
└── tesk
    └── services
        ├── abstract
        │   ├── filer.py
        │   └── __init__.py
        └── filers
            ├── ftp.py
            ├── http.py
            ├── __init__.py
            └── s3.py

with an abstract Filer class/interfact defined in tesk.services.abstract.filer that has some methods that individual file protocol plugins in tesk.services.filers then can inherit from and implement.

uniqueg commented 4 months ago

@JaeAeich: It may be good for you to keep in mind/consider this design principle during your GSoC project. However, I think this particular issue is mostly beyond the scope of the project.

@lvarin, @jemaltahir, @trispera: Is this something that you could perhaps look into? If so, please assign yourself :)

jemaltahir commented 4 months ago

I can have a look into this. What @uniqueg suggested file structure is great in terms of separation of concern and modularity.

JaeAeich commented 3 months ago

Whenever services and its tests are being refactor, please take in consideration that all those files have been ignored from linter (ruff), so change that in pyproject.toml.