[Feature] Run a function in an operations file, individually.

[gnat]

Is your feature request related to a problem? Please describe

Benefits

• Will provide a 1:1 migration path from Fabric tasks to Pyinfra operations.
• This could make Pyinfra an attractive option as a generic project task runner with 1 file at the root of a project.
• It's a simple pattern to combine highly related operations within 1 file without cluttering your /operations directory.
  • Use case 1: multiple branching steps for a highly related server setup operation, such as prod vs staging.
  • Use case 2: run specific diagnostic/audit steps instead of an entire diagnostic operation.

My /operations directory is becoming quite large with many highly related operations. It would be an amazing simplification to be able to group highly related / multi-step operations inside one file.

Describe the solution you'd like

Perhaps borrow the colon syntax : from gunicorn / uvicorn / sanic server. This will prevent conflicting with the existing syntax of calling built-in operations.

• pyinfra @local test:hello (this is how gunicorn / uvicorn / sanic all do it).
• or pyinfra @local test.py:hello (more explicit).

[Fizzadar]

Hi @gnat! So rather funny - I actually added partial support for this in v2.6: https://github.com/Fizzadar/pyinfra/commit/6b704b53a6ffb7cc63a1c9267ba3ee24d3c0638b

But it's not quite there - I like your example of passing a file by name with the colon selector as well, the current implementation only works for things that can be imported, ie that are in the PYTHONPATH.

[gnat]

Aha, nice additions! I like the direction pyinfra is headed here. Yeah it would be amazing to be able to call these without a module, and without explicitly setting PYTHONPATH before calling pyinfra.

It'll be great to be able to collapse all the various project-level task runners: bash, npm, fabric, gulp, make, etc. into a single pyinfra script at the root of a project- would make for a very cohesive experience with pyinfra also running the server ops stuff.

[gnat]

Can probably disregard this post if further development is in the works, but I'm continually running into Cannot assign to context base module after forcing PYTHONPATH to the correct search location (otherwise I just get No such module).

Not quite clear on usage yet but I get Cannot assign to context base module with either of these:

from pyinfra.operations import server
from pyinfra.api import operation

def test():
    server.shell("hello world")

@operation()
def test2():
    yield "echo 'hello world'"

Also related: https://github.com/Fizzadar/pyinfra/issues/833

Anyways, hoping for further development on this feature!

[Fizzadar]

@gnat how are you running the above?

$ cat testo.py
from pyinfra.operations import server
from pyinfra.api import operation

def test():
    server.shell("hello world")

@operation()
def test2():
    yield "echo 'hello world'"

Running both works fine:

$ PYTHONPATH=. pyinfra @local testo.test2

--> Loading config...

--> Loading inventory...

--> Connecting to hosts...
    [@local] Connected

--> Preparing operation...
    [@local] Ready: test2

--> Proposed changes:
    Groups: @local
    [@local]   Operations: 1   Change: 1   No change: 0

--> Beginning operation run...
--> Starting operation: Testo/Test2
    [@local] Success

--> Results:
    Groups: @local
    [@local]   Changed: 1   No change: 0   Errors: 0

[gnat]

Very cool trick with PYTHONPATH=.!

My bad, not sure what I was doing wrong all those weeks ago, works now with your example.

I wonder if we could eliminate needing to pass PYTHONPATH

[gnat]

Slightly off topic but I've re-discovered why I'm getting Cannot assign to context base module @Fizzadar

Minimum example:

$ cat testo.py
from pyinfra.operations import server
from pyinfra import state

# Print all output without -vvv. Problematic!
state.print_output = True
state.print_fact_output = True

def test():
    server.shell("echo 'hello world'")

PYTHONPATH=. pyinfra @local testo.test

Which seems to break pyinfra when used in this way.

Any alternatives for always getting output from echo?