src-d / borges

borges collects and stores Git repositories.
https://docs.sourced.tech/borges/
GNU General Public License v3.0
52 stars 20 forks source link

rework cli options and environment variables #294

Closed smola closed 6 years ago

smola commented 6 years ago
smola commented 6 years ago

The juice is in the first commit.

Example of the result:

Usage:
  borges [OPTIONS] consumer [consumer-OPTIONS]

This consumer fetches, packs and stores repositories. It reads one job per repository. Jobs should be produced witht he producer command.

Help Options:
  -h, --help                            Show this help message

[consumer command options]
          --queue=                      queue name (default: borges) [$BORGES_QUEUE]
          --broker=                     broker service URI (default: amqp://localhost:5672) [$BORGES_BROKER]
          --locking=                    locking service configuration (default: local:) [$BORGES_LOCKING]
          --workers=                    number of workers (default: 8) [$BORGES_WORKERS]
          --timeout=                    deadline to process a job (default: 10h) [$BORGES_TIMEOUT]
          --root-repositories-dir=      path to the directory storing rooted repositories (can be local path or hdfs://) (default: /tmp/root-repositories)
                                        [$BORGES_ROOT_REPOSITORIES_DIR]
          --root-repositories-temp-dir=
          --bucket-size=                if higher than zero, repositories are stored in bucket directories with a prefix of the given amount of characters
                                        from its root hash (default: 0) [$BORGES_BUCKET_SIZE]
          --temp-dir=                   path of temporary directory to clone repositories into (default: /tmp/borges) [$BORGES_TEMP_DIR]
          --temp-dir-clean              clean temporary directory before starting [$BORGES_TEMP_DIR_CLEAN]
          --database-driver=            database driver (default: postgres) [$BORGES_DATABASE_DRIVER]
          --database=                   database connection string (default:
                                        postgres://testing:testing@0.0.0.0:5432/testing?application_name=borges&sslmode=disable&connect_timeout=30)
                                        [$BORGES_DATABASE]

    debug options:
          --metrics                     expose a metrics endpoint using an HTTP server [$BORGES_METRICS_ENABLE]
          --metrics-port=               port to bind metrics to (default: 6062) [$BORGES_METRICS_PORT]
          --profiler                    start CPU, memory and block profilers [$BORGES_PROFILER_ENABLE]
          --profiler-port=              port to bind profiler to (default: 6061) [$BORGES_PROFILER_PORT]
jfontan commented 6 years ago

I believe we don't use --root-repositories-temp-dir anymore. I'll take a look and do a PR to core-retrieval if this is the case.

Also I would change --root-repositories-dir to --output in both commands. It's shorter, easier to understand and can have a short option -o.

smola commented 6 years ago

@jfontan I think --root-repositories-temp-dir should be removed if possible, although that implies some behaviour change I was not sure if I should do though. Note that I did not include --hdfs, and rely on a hdfs:// URL being passed to --root-repositories.

The problem I see with --output is that it is not purely an output directory, since it can also be read on updates.

Also note that you can setup Bash completion for an improved user experience:

_completion_example() {
    # All arguments except the first one
    args=("${COMP_WORDS[@]:1:$COMP_CWORD}")

    # Only split on newlines
    local IFS=$'\n'

    # Call completion (note that the first element of COMP_WORDS is
    # the executable itself)
    COMPREPLY=($(GO_FLAGS_COMPLETION=1 ${COMP_WORDS[0]} "${args[@]}"))
    return 0
}

complete -F _completion_example completion-example

See https://godoc.org/github.com/jessevdk/go-flags#hdr-Completion

jfontan commented 6 years ago

--root-repositories-temp-dir is no longer used. See src-d/core-retrieval#65

smola commented 6 years ago

@mcuadros What do you think about this approach for both flags and environment variables?

smola commented 6 years ago

I thought this would expose every possible setting in --help, but turns out that logging (via go-log) and queue (via go-queue) are still hidden (thanks @jfontan for pointing this out). I would like to add these explicitly to --help too.

smola commented 6 years ago

Closing in favor of https://github.com/src-d/borges/pull/338#issuecomment-419116618