exasol / bucketfs-client

Command line client for BucketFS
MIT License
2 stars 1 forks source link

Add missing functionality #13

Closed ckunki closed 1 year ago

ckunki commented 1 year ago

See https://github.com/exasol/bucketfs-python/issues/33

Status MVP Feature
:white_check_mark: :white_check_mark: List buckets in bucketfs see https://github.com/exasol/bucketfs-java/issues/54
:white_check_mark: :white_check_mark: List files in buckets or in "folders" (common path prefix)
:white_check_mark: :white_check_mark: Upload and download files to buckets (command cp)
:white_check_mark: :white_check_mark: Enhanced convenience for components in BFS path, e.g. host and port
:white_check_mark: :white_check_mark: Enhanced convenience for retrieving password
:white_check_mark: :white_check_mark: Delete files in bucketfs
:white_check_mark: :white_check_mark: read-password for non-public buckets
:white_check_mark: self contained executable, see #10
:white_check_mark: Recursive operations ls, cp, rm incl. upload and download of directories (potentially with multiple files)
:white_check_mark: cp supports to leave filename unchanged and omit from target path
:white_check_mark: Decode password from base64 encoding
Support wildcards like * and ? for source URIs
Consistent and complete support for TLS incl. certificates.
create and delete buckets in in contrast to directories and files inside a bucket
upload and download multiple files or folders, e.g. cp a.txt b.txt bfs:/
copy from one directory in bucketfs into another
:x: mv command: rejected as not supported by HTTP API
:x: copy from one bucket directory into another: rejected as multiple passwords could be involved which makes interface too complex
ckunki commented 1 year ago
  1. In my daily work the name of the default bucket is always "default".
  2. I propose to make this a default assumption of BFSC.
  3. Still I want to support use cases with different name of default bucket.
  4. Both use cases should be as convenient and generic as possible and avoid unexpected behavior as much as possible.

Proposal

  1. If Environment variable BUCKETFS_BUCKET is unset then BFSC prepends "default" to the path of the URI specified on command line.
  2. If Environment variable BUCKETFS_BUCKET is set to an non-empty string then BFSC prepends this name.
  3. If Environment variable BUCKETFS_BUCKET is set to an empty string then BFSC will not prepend any prefix.

See also https://github.com/exasol/bucketfs-client/blob/feature/13-4-environment/doc/user_guide/user_guide.md#name-of-the-root-bucket

ckunki commented 1 year ago

Decision: BFSC should use prefer "profiles" over environment variables, compare aws cli.

ckunki commented 1 year ago

How about non-public buckets with a read password? Does BFSC need to provide additional support here?

see L3-144, https://docs.exasol.com/db/latest/database_concepts/bucketfs/database_access.htm

ckunki commented 1 year ago

As passwords for access to BucketFS in file /exa/etc/EXAConf are base64-encoded, the initial list of planned featured for BucketFS client (BFSC) included to optionally decode these passwords to make life easier for users.

Besides the PRO-argument

Naive users should neither need to manually base64-decode the passwords nor to find out how to do this on the command line.

a short discussion has shown some counter arguments:

  1. Independent of the additional base64-encoding in file /exa/etc/EXAConf the user potentially might have remembered the original password without base64-encoding. In this case the user does not need a decoding feature in BFSC.
  2. A naive user might not even know that passwords are base64-encoded in file /exa/etc/EXAConf. Nor would this user then actively look for a decoding option of BFSC.
  3. A simple internet search will return the required linux command with minimal effort.
  4. We could (and will!) add an appropriate sentence to the user guide.
  5. The decoding feature adds additional complexity to BFSC, its configuration file, command line syntax, and documentation and results in either long or imprecise naming.
  6. For internal convenience the feature is not required as Exasol developers are familiar with base64-decoding on the linux command line. In particular when extracting passwords for an Exasol instance in a docker container then the user or a shell script can easily additionally already apply base64-decoding.
  7. In contrast to being able to add the feature to a later point in time with low effort removing it would be a breaking change for users already using the feature.