feat: streamline remote options

[cmdoret]

Objective:

Allows users to always specify the same modos server endpoint. Underlying htsget/s3 endpoints are auto-detected internally by querying the modos server.

For testing / advanced usage, a services option is still exposed in the API to bypass this and explicitely provide service endpoints.

Notes

• As every command in the CLI was copy-pasting the s3_endpoint option, this PR removes it in favor of a top-level endpoint flag that is inherited by all commands (through typer.Context).
• endpoint option can also read from the environment variable MODOS_ENDPOINT.
• the docker compose config used a static network ip. This was only needed because we passed the S3_LOCAL_URL to htsget instead of S3_PUBLIC_URL (by mistake). Fixing the env var makes the static IP unnecessary, and this PR removes it.
• pydantic.HttpUrl was used in some places for more explicit type hints, this PR adds it throughout the package and decorates functions with @validate_call, to validate url params at runtime.
• expose the version from pyproject.toml through modos.__version__ and in the cli through modos --version using a typer callback.
• describe MODO arguments in docstring

Limitations

• If the environment variable is set, the user has to manually override it to create local objects which may not be intuitive.
• This PR implements runtime parameter validation on functions (via pydantic.validate_call) and dataclases (by replacing dataclasses.dataclass with pydantic.dataclasses.dataclass), but not on normal classes.

Questions should we use a more explicit mechanism for remote/local paths to avoid confusion? (e.g. --local flag or explicitely requiring s3:// paths for remote objects.). Should this be in this, or a separate PR?

Examples

CLI:

$ modos --version
modos 0.1.0

# create remote object on the s3 provided by the modos server at http://localhost
$ modos --endpoint http://localhost create bucket/object

# read endpoint from env variable to avoid repetition
$ export MODOS_ENDPOINT='http://localhost'
$ modos create bucket/object2
$ modos show bucket/object2
$ modos delete bucket/object2

API:

>>> from modos.api import MODO
>>> MODO('bucket/object2', endpoint='http://localhost')

# This is also possible, e.g. user has an s3 server, but no modos server
>>> nostream = MODO('bucket/ex-nostream', services={'s3': 'http://s3.example.org'})
>>> nostream.stream_genomics('demo1.cram')
ValueError: No htsget endpoint provided

[cmdoret]

TODO:

• make only use remote modos when s3:// scheme is specified (otherwise local
• add some info logs (with print) to stderr
• add a help message for object_path in cli help, mentioning the format of remote paths

[cmdoret]

Added this based on our discussion:

• rudimentary logs on MODO instantiation (tells you if it's using local or remote, and what bucket)
• require s3:// for remote paths
  • check done on MODO instantiation using a custom S3Path pydantic model
  • changes made in modos.storage.S3Storage to support this

With these changes, the examples above become:

CLI:

$ modos --version
modos 0.1.0

# create remote object on the s3 provided by the modos server at http://localhost
$ modos --endpoint http://localhost create s3://bucket/object

# read endpoint from env variable to avoid repetition
$ export MODOS_ENDPOINT='http://localhost'
$ modos create s3://bucket/object2
$ modos show   s3://bucket/object2
$ modos delete s3://bucket/object2

# we can operate on local objects when MODOS_ENDPOINT is set!
$ modos create data/object2

API:

>>> from modos.api import MODO
>>> MODO('s3://bucket/object2', endpoint='http://localhost')

# This is also possible, e.g. user has an s3 server, but no modos server
>>> nostream = MODO('s3://bucket/ex-nostream', services={'s3': 'http://s3.example.org'})
>>> nostream.stream_genomics('demo1.cram')
ValueError: No htsget endpoint provided