search

pyinvoke / invoke

Pythonic task management & command execution.
http://pyinvoke.org
BSD 2-Clause "Simplified" License
4.41k stars 370 forks source link

invoke won't load config file unless -c is used #696

Open alexei opened 4 years ago

alexei commented 4 years ago

Hello,

I'm facing some issues with regards to using a configuration file and I'm not sure whether it's a bug in the code or the docs are confusing.

What I want to do is to have my tasks in a different directory e.g. ops, instead of tasks. If I read the docs correctly, that should be possible by placing a invoke.yaml file in the project's root. It appears that doesn't work:

$ ls
invoke.yaml ops.py
$ cat invoke.yaml 
tasks:
  collection_name: ops
$ cat ops.py 
from invoke import task

@task()
def hello(c):
    print("Hello world!")

Calling the task results in an error:

$ inv hello
Can't find any collection named 'tasks'!
$ INVOKE_RUNTIME_CONFIG=/Users/alexei/Projects/playground/inv/invoke.yaml inv hello
Can't find any collection named 'tasks'!

Unless I explicitly pass the -c flag

$ inv -c ops hello
Hello world!
$ INVOKE_DEBUG=1 inv hello
invoke.config._load_file: Didn't see any /etc/invoke.yaml, skipping.
invoke.config._load_file: Didn't see any /etc/invoke.yml, skipping.
invoke.config._load_file: Didn't see any /etc/invoke.json, skipping.
invoke.config._load_file: Didn't see any /Users/alexei/.invoke.yaml, skipping.
invoke.config._load_file: Didn't see any /Users/alexei/.invoke.yml, skipping.
invoke.config._load_file: Didn't see any /Users/alexei/.invoke.json, skipping.
invoke.config.merge: Merging config sources in order onto new empty _config...
invoke.config.merge: Defaults: {'timeouts': {'command': None}, 'run': {'dry': False, 'shell': '/bin/bash', 'hide': None, 'env': {}, 'encoding': None, 'in_stream': None, 'replace_env': False, 'err_stream': None, 'echo': False, 'warn': False, 'echo_stdin': None, 'watchers': [], 'asynchronous': False, 'pty': False, 'out_stream': None, 'fallback': True, 'disown': False}, 'tasks': {'search_root': None, 'collection_name': 'tasks', 'dedupe': True, 'executor_class': None, 'auto_dash_names': True}, 'sudo': {'password': None, 'prompt': '[sudo] password: ', 'user': None}, 'runners': {'local': <class 'invoke.runners.Local'>}}
invoke.config.merge: Collection-driven: {}
invoke.config._merge_file: System-wide config file (/etc/invoke.py): {}
invoke.config._merge_file: Per-user config file (/Users/alexei/.invoke.py): {}
invoke.config._merge_file: Per-project config file has not been loaded yet, skipping
invoke.config.merge: Environment variable config: {}
invoke.config._merge_file: Runtime config file has not been loaded yet, skipping
invoke.config.merge: Overrides: {}
invoke.config.merge: Modifications: {}
invoke.config.merge: Deletions: {}
invoke.program.parse_core: argv given to Program.run: None
invoke.program.normalize_argv: argv was None; using sys.argv: ['/Users/alexei/.virtualenvs/invtest/bin/inv', 'hello']
invoke.program.parse_core_args: Parsing initial context (core args)
invoke.parser.__init__: Initialized with context: <parser/Context: {u'dry': <Argument: dry (R) [bool]>, u'warn-only': <Argument: warn-only (w) [bool]>, u'print-completion-script': <Argument: print-completion-script>, u'hide': <Argument: hide>, u'complete': <Argument: complete [bool]>, u'write-pyc': <Argument: write-pyc [bool]>, u'list': <Argument: list (l) ?>, u'command-timeout': <Argument: command-timeout (T) [int]>, u'collection': <Argument: collection (c)>, u'echo': <Argument: echo (e) [bool]>, u'debug': <Argument: debug (d) [bool]>, u'list-depth': <Argument: list-depth (D) [int]>, u'version': <Argument: version (V) [bool]>, u'search-root': <Argument: search-root (r)>, u'pty': <Argument: pty (p) [bool]>, u'prompt-for-sudo-password': <Argument: prompt-for-sudo-password [bool]>, u'config': <Argument: config (f)>, u'list-format': <Argument: list-format (F)>, u'no-dedupe': <Argument: no-dedupe [bool]>, u'help': <Argument: help (h) ?>}>
invoke.parser.__init__: Available contexts: {}
invoke.parser.complete_context: Wrapping up context None
invoke.parser.parse_argv: Starting argv: ['hello']
invoke.parser.handle: Handling token: 'hello'
invoke.parser.handle: Bottom-of-handle() see_unknown('hello')
invoke.parser.changing_state: ParseMachine: 'context' => 'unknown'
invoke.parser.complete_context: Wrapping up context None
invoke.parser.store_only: Storing unknown token 'hello'
invoke.parser.changing_state: ParseMachine: 'unknown' => 'end'
invoke.parser.complete_context: Wrapping up context None
invoke.program.parse_core_args: Core-args parse result: [<parser/Context: {u'dry': <Argument: dry (R) [bool]>, u'warn-only': <Argument: warn-only (w) [bool]>, u'print-completion-script': <Argument: print-completion-script>, u'hide': <Argument: hide>, u'complete': <Argument: complete [bool]>, u'write-pyc': <Argument: write-pyc [bool]>, u'list': <Argument: list (l) ?>, u'command-timeout': <Argument: command-timeout (T) [int]>, u'collection': <Argument: collection (c)>, u'echo': <Argument: echo (e) [bool]>, u'debug': <Argument: debug (d) [bool]>, u'list-depth': <Argument: list-depth (D) [int]>, u'version': <Argument: version (V) [bool]>, u'search-root': <Argument: search-root (r)>, u'pty': <Argument: pty (p) [bool]>, u'prompt-for-sudo-password': <Argument: prompt-for-sudo-password [bool]>, u'config': <Argument: config (f)>, u'list-format': <Argument: list-format (F)>, u'no-dedupe': <Argument: no-dedupe [bool]>, u'help': <Argument: help (h) ?>}>] & unparsed: ['hello']
invoke.program.parse_core: Finished parsing core args
invoke.program.parse_collection: No default namespace provided, trying to load one from disk
invoke.loader.find: FilesystemLoader find starting at '/Users/alexei/Projects/playground/inv'
invoke.loader.find: ImportError loading 'tasks', raising CollectionNotFound
invoke.program.run: Received a possibly-skippable exception: Exit()
Can't find any collection named 'tasks'!
$ INVOKE_DEBUG=1 inv -c ops hello
invoke.config._load_file: Didn't see any /etc/invoke.yaml, skipping.
invoke.config._load_file: Didn't see any /etc/invoke.yml, skipping.
invoke.config._load_file: Didn't see any /etc/invoke.json, skipping.
invoke.config._load_file: Didn't see any /Users/alexei/.invoke.yaml, skipping.
invoke.config._load_file: Didn't see any /Users/alexei/.invoke.yml, skipping.
invoke.config._load_file: Didn't see any /Users/alexei/.invoke.json, skipping.
invoke.config.merge: Merging config sources in order onto new empty _config...
invoke.config.merge: Defaults: {'timeouts': {'command': None}, 'run': {'dry': False, 'shell': '/bin/bash', 'hide': None, 'env': {}, 'encoding': None, 'in_stream': None, 'replace_env': False, 'err_stream': None, 'echo': False, 'warn': False, 'echo_stdin': None, 'watchers': [], 'asynchronous': False, 'pty': False, 'out_stream': None, 'fallback': True, 'disown': False}, 'tasks': {'search_root': None, 'collection_name': 'tasks', 'dedupe': True, 'executor_class': None, 'auto_dash_names': True}, 'sudo': {'password': None, 'prompt': '[sudo] password: ', 'user': None}, 'runners': {'local': <class 'invoke.runners.Local'>}}
invoke.config.merge: Collection-driven: {}
invoke.config._merge_file: System-wide config file (/etc/invoke.py): {}
invoke.config._merge_file: Per-user config file (/Users/alexei/.invoke.py): {}
invoke.config._merge_file: Per-project config file has not been loaded yet, skipping
invoke.config.merge: Environment variable config: {}
invoke.config._merge_file: Runtime config file has not been loaded yet, skipping
invoke.config.merge: Overrides: {}
invoke.config.merge: Modifications: {}
invoke.config.merge: Deletions: {}
invoke.program.parse_core: argv given to Program.run: None
invoke.program.normalize_argv: argv was None; using sys.argv: ['/Users/alexei/.virtualenvs/invtest/bin/inv', '-c', 'ops', 'hello']
invoke.program.parse_core_args: Parsing initial context (core args)
invoke.parser.__init__: Initialized with context: <parser/Context: {u'dry': <Argument: dry (R) [bool]>, u'warn-only': <Argument: warn-only (w) [bool]>, u'print-completion-script': <Argument: print-completion-script>, u'hide': <Argument: hide>, u'complete': <Argument: complete [bool]>, u'write-pyc': <Argument: write-pyc [bool]>, u'list': <Argument: list (l) ?>, u'command-timeout': <Argument: command-timeout (T) [int]>, u'collection': <Argument: collection (c)>, u'echo': <Argument: echo (e) [bool]>, u'debug': <Argument: debug (d) [bool]>, u'list-depth': <Argument: list-depth (D) [int]>, u'version': <Argument: version (V) [bool]>, u'search-root': <Argument: search-root (r)>, u'pty': <Argument: pty (p) [bool]>, u'prompt-for-sudo-password': <Argument: prompt-for-sudo-password [bool]>, u'config': <Argument: config (f)>, u'list-format': <Argument: list-format (F)>, u'no-dedupe': <Argument: no-dedupe [bool]>, u'help': <Argument: help (h) ?>}>
invoke.parser.__init__: Available contexts: {}
invoke.parser.complete_context: Wrapping up context None
invoke.parser.parse_argv: Starting argv: ['-c', 'ops', 'hello']
invoke.parser.handle: Handling token: '-c'
invoke.parser.handle: Saw flag '-c'
invoke.parser.switch_to_flag: Moving to flag <Argument: collection (c)>
invoke.parser.handle: Handling token: 'ops'
invoke.parser.handle: We're waiting for a flag value so 'ops' must be it?
invoke.parser.see_value: Setting flag <Argument: collection (c)> to value 'ops'
invoke.parser.handle: Handling token: 'hello'
invoke.parser.handle: Bottom-of-handle() see_unknown('hello')
invoke.parser.changing_state: ParseMachine: 'context' => 'unknown'
invoke.parser.complete_flag: Completing current flag <Argument: collection (c)> before moving on
invoke.parser.complete_context: Wrapping up context None
invoke.parser.store_only: Storing unknown token 'hello'
invoke.parser.changing_state: ParseMachine: 'unknown' => 'end'
invoke.parser.complete_flag: Completing current flag <Argument: collection (c)> before moving on
invoke.parser.complete_context: Wrapping up context None
invoke.program.parse_core_args: Core-args parse result: [<parser/Context: {u'dry': <Argument: dry (R) [bool]>, u'warn-only': <Argument: warn-only (w) [bool]>, u'print-completion-script': <Argument: print-completion-script>, u'hide': <Argument: hide>, u'complete': <Argument: complete [bool]>, u'write-pyc': <Argument: write-pyc [bool]>, u'list': <Argument: list (l) ?>, u'command-timeout': <Argument: command-timeout (T) [int]>, u'collection': <Argument: collection (c)>, u'echo': <Argument: echo (e) [bool]>, u'debug': <Argument: debug (d) [bool]>, u'list-depth': <Argument: list-depth (D) [int]>, u'version': <Argument: version (V) [bool]>, u'search-root': <Argument: search-root (r)>, u'pty': <Argument: pty (p) [bool]>, u'prompt-for-sudo-password': <Argument: prompt-for-sudo-password [bool]>, u'config': <Argument: config (f)>, u'list-format': <Argument: list-format (F)>, u'no-dedupe': <Argument: no-dedupe [bool]>, u'help': <Argument: help (h) ?>}>] & unparsed: ['hello']
invoke.program.parse_core: Finished parsing core args
invoke.program.parse_collection: No default namespace provided, trying to load one from disk
invoke.loader.find: FilesystemLoader find starting at '/Users/alexei/Projects/playground/inv'
invoke.loader.find: Found module: '/Users/alexei/Projects/playground/inv/ops.py'
invoke.config.merge: Merging config sources in order onto new empty _config...
invoke.config.merge: Defaults: {'timeouts': {'command': None}, 'run': {'dry': False, 'shell': '/bin/bash', 'hide': None, 'env': {}, 'encoding': None, 'in_stream': None, 'replace_env': False, 'err_stream': None, 'echo': False, 'warn': False, 'echo_stdin': None, 'watchers': [], 'asynchronous': False, 'pty': False, 'out_stream': None, 'fallback': True, 'disown': False}, 'tasks': {'search_root': None, 'collection_name': 'tasks', 'dedupe': True, 'executor_class': None, 'auto_dash_names': True}, 'sudo': {'password': None, 'prompt': '[sudo] password: ', 'user': None}, 'runners': {'local': <class 'invoke.runners.Local'>}}
invoke.config.merge: Collection-driven: {}
invoke.config._merge_file: System-wide config file (/etc/invoke.py): {}
invoke.config._merge_file: Per-user config file (/Users/alexei/.invoke.py): {}
invoke.config._merge_file: Per-project config file (/Users/alexei/Projects/playground/inv/invoke.yaml): {'tasks': {'collection_name': 'ops'}}
invoke.config.merge: Environment variable config: {}
invoke.config._merge_file: Runtime config file has not been loaded yet, skipping
invoke.config.merge: Overrides: {}
invoke.config.merge: Modifications: {}
invoke.config.merge: Deletions: {}
invoke.parser.__init__: Adding <parser/Context 'hello'>
invoke.program.parse_tasks: Parsing tasks against <Collection 'ops': hello>
invoke.parser.__init__: Initialized with context: <parser/Context: {u'dry': <Argument: dry (R) [bool]>, u'warn-only': <Argument: warn-only (w) [bool]>, u'print-completion-script': <Argument: print-completion-script>, u'hide': <Argument: hide>, u'complete': <Argument: complete [bool]>, u'write-pyc': <Argument: write-pyc [bool]>, u'list': <Argument: list (l) ?>, u'command-timeout': <Argument: command-timeout (T) [int]>, u'collection': <Argument: collection (c)>, u'echo': <Argument: echo (e) [bool]>, u'debug': <Argument: debug (d) [bool]>, u'list-depth': <Argument: list-depth (D) [int]>, u'version': <Argument: version (V) [bool]>, u'search-root': <Argument: search-root (r)>, u'pty': <Argument: pty (p) [bool]>, u'prompt-for-sudo-password': <Argument: prompt-for-sudo-password [bool]>, u'config': <Argument: config (f)>, u'list-format': <Argument: list-format (F)>, u'no-dedupe': <Argument: no-dedupe [bool]>, u'help': <Argument: help (h) ?>}>
invoke.parser.__init__: Available contexts: {'hello': <parser/Context 'hello'>}
invoke.parser.complete_context: Wrapping up context None
invoke.parser.parse_argv: Starting argv: ['hello']
invoke.parser.handle: Handling token: 'hello'
invoke.parser.changing_state: ParseMachine: 'context' => 'context'
invoke.parser.complete_context: Wrapping up context None
invoke.parser.switch_to_context: Moving to context 'hello'
invoke.parser.switch_to_context: Context args: {}
invoke.parser.switch_to_context: Context flags: {}
invoke.parser.switch_to_context: Context inverse_flags: {}
invoke.parser.changing_state: ParseMachine: 'context' => 'end'
invoke.parser.complete_context: Wrapping up context 'hello'
invoke.program.parse_tasks: Resulting task contexts: [<parser/Context 'hello'>]
invoke.config.merge: Merging config sources in order onto new empty _config...
invoke.config.merge: Defaults: {'timeouts': {'command': None}, 'run': {'dry': False, 'shell': '/bin/bash', 'hide': None, 'env': {}, 'encoding': None, 'in_stream': None, 'replace_env': False, 'err_stream': None, 'echo': False, 'warn': False, 'echo_stdin': None, 'watchers': [], 'asynchronous': False, 'pty': False, 'out_stream': None, 'fallback': True, 'disown': False}, 'tasks': {'search_root': None, 'collection_name': 'tasks', 'dedupe': True, 'executor_class': None, 'auto_dash_names': True}, 'sudo': {'password': None, 'prompt': '[sudo] password: ', 'user': None}, 'runners': {'local': <class 'invoke.runners.Local'>}}
invoke.config.merge: Collection-driven: {}
invoke.config._merge_file: System-wide config file (/etc/invoke.py): {}
invoke.config._merge_file: Per-user config file (/Users/alexei/.invoke.py): {}
invoke.config._merge_file: Per-project config file (/Users/alexei/Projects/playground/inv/invoke.yaml): {'tasks': {'collection_name': 'ops'}}
invoke.config.merge: Environment variable config: {}
invoke.config._merge_file: Runtime config file has not been loaded yet, skipping
invoke.config.merge: Overrides: {'timeouts': {}, 'sudo': {}, 'run': {}, 'tasks': {}}
invoke.config.merge: Modifications: {}
invoke.config.merge: Deletions: {}
invoke.executor.execute: Examining top level tasks [<parser/Context 'hello'>]
invoke.executor.execute: Tasks (now Calls) with kwargs: [<Call 'hello', args: (), kwargs: {}>]
invoke.executor.expand_calls: Expanding task-call <Call 'hello', args: (), kwargs: {}>
invoke.executor.dedupe: Deduplicating tasks...
invoke.executor.dedupe: <Call 'hello', args: (), kwargs: {}>: no duplicates found, ok
invoke.executor.execute: Executing <Call 'hello', args: (), kwargs: {}>
invoke.config.load_collection: Loading collection configuration
invoke.config.merge: Merging config sources in order onto new empty _config...
invoke.config.merge: Defaults: {'timeouts': {'command': None}, 'run': {'dry': False, 'shell': '/bin/bash', 'hide': None, 'env': {}, 'encoding': None, 'in_stream': None, 'replace_env': False, 'err_stream': None, 'echo': False, 'warn': False, 'echo_stdin': None, 'watchers': [], 'asynchronous': False, 'pty': False, 'out_stream': None, 'fallback': True, 'disown': False}, 'tasks': {'search_root': None, 'collection_name': 'tasks', 'dedupe': True, 'executor_class': None, 'auto_dash_names': True}, 'sudo': {'password': None, 'prompt': '[sudo] password: ', 'user': None}, 'runners': {'local': <class 'invoke.runners.Local'>}}
invoke.config.merge: Collection-driven: {}
invoke.config._merge_file: System-wide config file (/etc/invoke.py): {}
invoke.config._merge_file: Per-user config file (/Users/alexei/.invoke.py): {}
invoke.config._merge_file: Per-project config file (/Users/alexei/Projects/playground/inv/invoke.yaml): {'tasks': {'collection_name': 'ops'}}
invoke.config.merge: Environment variable config: {}
invoke.config._merge_file: Runtime config file has not been loaded yet, skipping
invoke.config.merge: Overrides: {'timeouts': {}, 'sudo': {}, 'run': {}, 'tasks': {}}
invoke.config.merge: Modifications: {}
invoke.config.merge: Deletions: {}
invoke.config.load_shell_env: Running pre-merge for shell env loading...
invoke.config.merge: Merging config sources in order onto new empty _config...
invoke.config.merge: Defaults: {'timeouts': {'command': None}, 'run': {'dry': False, 'shell': '/bin/bash', 'hide': None, 'env': {}, 'encoding': None, 'in_stream': None, 'replace_env': False, 'err_stream': None, 'echo': False, 'warn': False, 'echo_stdin': None, 'watchers': [], 'asynchronous': False, 'pty': False, 'out_stream': None, 'fallback': True, 'disown': False}, 'tasks': {'search_root': None, 'collection_name': 'tasks', 'dedupe': True, 'executor_class': None, 'auto_dash_names': True}, 'sudo': {'password': None, 'prompt': '[sudo] password: ', 'user': None}, 'runners': {'local': <class 'invoke.runners.Local'>}}
invoke.config.merge: Collection-driven: {}
invoke.config._merge_file: System-wide config file (/etc/invoke.py): {}
invoke.config._merge_file: Per-user config file (/Users/alexei/.invoke.py): {}
invoke.config._merge_file: Per-project config file (/Users/alexei/Projects/playground/inv/invoke.yaml): {'tasks': {'collection_name': 'ops'}}
invoke.config.merge: Environment variable config: {}
invoke.config._merge_file: Runtime config file has not been loaded yet, skipping
invoke.config.merge: Overrides: {'timeouts': {}, 'sudo': {}, 'run': {}, 'tasks': {}}
invoke.config.merge: Modifications: {}
invoke.config.merge: Deletions: {}
invoke.config.load_shell_env: Done with pre-merge.
invoke.env.load: Scanning for env vars according to prefix: 'INVOKE_', mapping: {'RUN_HIDE': ['run', 'hide'], 'SUDO_PASSWORD': ['sudo', 'password'], 'RUN_PTY': ['run', 'pty'], 'RUN_ASYNCHRONOUS': ['run', 'asynchronous'], 'TIMEOUTS_COMMAND': ['timeouts', 'command'], 'TASKS_EXECUTOR_CLASS': ['tasks', 'executor_class'], 'RUN_REPLACE_ENV': ['run', 'replace_env'], 'RUN_DISOWN': ['run', 'disown'], 'SUDO_PROMPT': ['sudo', 'prompt'], 'TASKS_COLLECTION_NAME': ['tasks', 'collection_name'], 'RUN_ECHO': ['run', 'echo'], 'TASKS_AUTO_DASH_NAMES': ['tasks', 'auto_dash_names'], 'RUN_ECHO_STDIN': ['run', 'echo_stdin'], 'RUN_WATCHERS': ['run', 'watchers'], 'RUN_SHELL': ['run', 'shell'], 'TASKS_DEDUPE': ['tasks', 'dedupe'], 'RUN_ERR_STREAM': ['run', 'err_stream'], 'RUN_DRY': ['run', 'dry'], 'RUNNERS_LOCAL': ['runners', 'local'], 'RUN_FALLBACK': ['run', 'fallback'], 'TASKS_SEARCH_ROOT': ['tasks', 'search_root'], 'RUN_WARN': ['run', 'warn'], 'RUN_OUT_STREAM': ['run', 'out_stream'], 'SUDO_USER': ['sudo', 'user'], 'RUN_ENCODING': ['run', 'encoding'], 'RUN_IN_STREAM': ['run', 'in_stream']}
invoke.env.load: Obtained env var config: {}
invoke.config.load_shell_env: Loaded shell environment, triggering final merge
invoke.config.merge: Merging config sources in order onto new empty _config...
invoke.config.merge: Defaults: {'timeouts': {'command': None}, 'run': {'dry': False, 'shell': '/bin/bash', 'hide': None, 'env': {}, 'encoding': None, 'in_stream': None, 'replace_env': False, 'err_stream': None, 'echo': False, 'warn': False, 'echo_stdin': None, 'watchers': [], 'asynchronous': False, 'pty': False, 'out_stream': None, 'fallback': True, 'disown': False}, 'tasks': {'search_root': None, 'collection_name': 'tasks', 'dedupe': True, 'executor_class': None, 'auto_dash_names': True}, 'sudo': {'password': None, 'prompt': '[sudo] password: ', 'user': None}, 'runners': {'local': <class 'invoke.runners.Local'>}}
invoke.config.merge: Collection-driven: {}
invoke.config._merge_file: System-wide config file (/etc/invoke.py): {}
invoke.config._merge_file: Per-user config file (/Users/alexei/.invoke.py): {}
invoke.config._merge_file: Per-project config file (/Users/alexei/Projects/playground/inv/invoke.yaml): {'tasks': {'collection_name': 'ops'}}
invoke.config.merge: Environment variable config: {}
invoke.config._merge_file: Runtime config file has not been loaded yet, skipping
invoke.config.merge: Overrides: {'timeouts': {}, 'sudo': {}, 'run': {}, 'tasks': {}}
invoke.config.merge: Modifications: {}
invoke.config.merge: Deletions: {}
invoke.executor.execute: Finished loading collection & shell env configs
Hello world!

The above shows the per-project configuration file is loaded late, perhaps after the tasks discovery. But then I wonder if the tasks.collection_name can actually be specified per-project.

FWIW:

$ pip freeze | grep invoke
invoke==1.4.0
florisla commented 4 years ago

This is related to #599.