extra gcs and s3

[yokomotod]

Problem

• now we always have to install both s3 and gcs dependencies
• it causes
  • more lock problem, since python's dependency resolving(like Poetry) is fragile
  • unnecessary install time, disk usage, container image size

Goal

• from gokart USER
  • can separate [s3] and [gcs] extras to reduce unnecessary dependencies
  • pip install gokart: not install gcs, s3 related deps
  • pip install gokart[s3]: install with s3 related deps
  • pip install gokart[gcs]: install with gcs related deps
  • pip install gokart[s3][gcs]: install both (results same as with previous behavior)

Sub Goal

• from gokart DEVELOPER

  • can unit tests that
  • core logic works without any extra deps. no unintended access to extra libs
  • each extra logic works with its dependency
  • extra dependencies are installed correctly
  • ~usually, developers can run all unit tests without being conscious of extras~
  • ~extras are installed by default~
  • (update)
    • this goal makes confusion of pyproject.toml, so after discussion, I drop this goal
    • and developer will poetry install --extras=s3 --extras=gcs --with=dev-s3 explicitly
  • all test for extras will be executed

    Discussion

• how it's confusing to need gokart[s3] or gokart[gcs] to works

• do we have to MAJOR update because this is breaking change?

  • (update) discussed and maybe YES!

ToDO

test publishing on pypy dev

[yokomotod]

other option:

• separate pypi package?
  • gokart-gcs gokart-s3
    • (update)
      • NO. it will cause combinatorial explosion if we have more extras
      • and pip extra is straight forward for this purpose

[yokomotod]

@hirosassa Hi! How do your team feel, as another well know users? 😃

especially # discussion points on description

[KAZYPinkSaurus]

unnecessary install time, disk usage, container image size

How much does it actually affect disk size and install time?