chaining tasks

[scher200]

Hi @kcmerrill do you also have nice trick to chain tasks And so that the output of task.one becomes the input of task.two

I could amagine something like:

tasks:  task.one | task.two

[kcmerrill]

I've been thinking about a way to do it. Should be relatively easy and a feature I've wanted to add for a while. Would you mind sharing your specific use case?

On Thu, Dec 28, 2017 at 7:50 AM scher200 notifications@github.com wrote:

Hi @kcmerrill https://github.com/kcmerrill do you also have nice trick to chain tasks And so that the output of task.one becomes the input of task.two

I could amagine something like:

tasks: task.one | task.two

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/kcmerrill/alfred/issues/61, or mute the thread https://github.com/notifications/unsubscribe-auth/AAqMScvjO3UA4afpfoC5ZkmLgtNSrbFqks5tE6q8gaJpZM4ROYtq .

[scher200]

well I am trying to build an alfred module to wrap openfaas. I like to chain these created functions. And so that the function one creating an image can be converted to a PNG format by function two. right now I would need to do this:

echo "newimagename" | curl http://localhost:8080/function/random-image-by-name --data-binary @- | curl http://localhost:8080/function/convert-image-png --data-binary @-

[kcmerrill]

Oh I see. So literally piping via say stdin to another command/task command component?

[scher200]

yes, like that:

tasks: faas.create.image({{ index .Args 0 }}) | faas.convert.image

[kcmerrill]

I can't quite do that syntax, but this is what I was thinking. Feedback welcome.

https://github.com/kcmerrill/alfred/blob/master/TFM.md#stdin--stringtext-command

[scher200]

@kcmerrill this is awesome!

is there a way I can startoff with a task instead of a command, like 'combine.pipe.one':

task.pipe.one:
    stdin: echo "pipe1"
    ok: notify.ok
    fail: notify.fail({{ .TaskName }},{{ .Stdin }})

md5:
    command: |
        md5

combine.pipe.one:
    stdin: task.pipe.one
    ok: md5
    fail: notify.fail({{ .TaskName }},{{ .Stdin }})

Ah, yes .TaskName would be an nice one to have for custom type debugging

[scher200]

Now I am on fire: Same thing for register working with tasks:

task.one:
    command: echo "this looks really good!"

register:
    summary: Demonstrate the registration of variables
    register:
        taskone: task.one 
        user: whoami
        twitter: "@themayor"
    command: |
      echo "{{ index .Vars "user" }}"
      echo "{{ .Vars.twitter }}"
      echo "{{ .Vars.taskone }}"

[kcmerrill]

A few things ...

I am going to add a silent || raw flag to avoid all the output, but keep in mind a valid shell command is alfred my.task ;)

So in my next PR you can do something like this: alfred --no-formatting task.one and all you get is the raw output without any of the alfred stuff. If you need this right away I'd recommend just sending out put to a log, and then cat'ing out the log in the meantime.

I like the idea of the previous task name being somewhere. Something like {{ .TaskName }}

[scher200]

Awesome! I look forward to the next PR, --no-formatting would both boost functionality for my projects. Please consider to make it possible to log to a variable as well:

md5:
    command: |
        md5
    log: Var.Name

I am glad you liked the Idea of {{ .TaskName }} Well it almoast starts to smelling like a programming language :)

[kcmerrill]

It's there. On Sat, Dec 30, 2017 at 11:22 AM scher200 notifications@github.com wrote:

Awesome! I look forward to the next PR, --no-formatting would both boost functionality for my projects. Please consider to make it possible to log to a variable as well:

md5: command: | md5 log: Var.Name

I am glad you liked the Idea of {{ .TaskName }} Well it almoast starts to smelling like a programming language :)

— You are receiving this because you modified the open/close state.

Reply to this email directly, view it on GitHub https://github.com/kcmerrill/alfred/issues/61#issuecomment-354560769, or mute the thread https://github.com/notifications/unsubscribe-auth/AAqMSRcCzqUUWe0i9RjgaEx5ayCAtOceks5tFn9YgaJpZM4ROYtq .

[scher200]

You where right, I got what i was looking for:

register.pipe:
    summary: Demonstrate pipe of setup registered commands
    register:
        prefix: '{{randAlphaNum 7}}'
        who: '{{ with $prefix := .Vars.prefix | printf "%swho" }}{{ $prefix }}{{ end }}'
        base64encrypt: '{{ with $prefix := .Vars.prefix | printf "%sbase64encrypt" }}{{ $prefix }}{{ end }}'
        base64decrypt: '{{ with $prefix := .Vars.prefix | printf "%sbase64decrypt" }}{{ $prefix }}{{ end }}'
    setup: |
        register.who
        base64.encrypt({{ index .Vars .Vars.who }})
        base64.decrypt({{ index .Vars .Vars.base64encrypt }})
    command: |
        echo "RESULT: {{ index .Vars .Vars.who }} {{ index .Vars .Vars.base64encrypt }} {{ index .Vars .Vars.base64decrypt }}"

register.who:
    register:
        "{{ .Vars.prefix }}who": whoami

base64.encrypt:
    summary: Base64 encryption
    register:
        "{{ .Vars.prefix }}base64encrypt": echo "{{ index .Args 0 }}" | base64

base64.decrypt:
    summary: Base64 decryption
    register:
        "{{ .Vars.prefix }}base64decrypt": echo "{{ index .Args 0 }}" | base64 -d

(the whole prefixing is done for if running in multitask mode it would not confict or mix up variable values)

Please let me know if you think this could be done any better.

[kcmerrill]

You've actually, IMO, uncovered a bug.

The way you're prefixing the env shouldn't be required. I am(or so I thought) copying the context of the task and passing it along, but it's only doing a shallow copy, and not a deep copy. So things like the maps of vars and args are being shared throughout all of the tasks.

The idea was, the top level task would spawn it's own context, and each task thereafter.

So in your particular case, it would just be base64encrypt. Heh. I don't think this is a quick fix, but it's one I want to make so you don't have to prefix this. This was my original design idea from the getgo but I guess I didn't really stumble upon it until just now.

Take this alfred file as an example. It should, how I originally imagined it, sleep for 1 second, 2 seconds and three seconds, spitting out A,B,C .... except it spits out and sleep only for the last one registered. I can see both sides to it being "right" but I would prefer to see the below example working as I outlined above.

entrypoint:
    summary: Spawn a few commands
    multitask: |
        register(a, 1)
        register(b, 2)
        register(c, 3)

register:
    summary:
    register:
        letter: "{{ index .Args 0 }}"
        sleep: "{{ index .Args 1 }}"
    tasks: process

process:
    summary: I will process letter/sleep!
    commands: |
        sleep {{ .Vars.sleep }}
        echo "My letter is: {{ .Vars.letter }}"
        echo "Slept for: {{ .Vars.sleep }}"
    exit: 1

[scher200]

Game on! We catched Alfred in a mistake, I have never really see him do this in the movies.

[kcmerrill]

Alright, so this should now be fixed. Basically, each task will have it's own context, so you can register a variable, and all tasks that were spawned from that task should now have it's own registries, variables, etc ...

So in your example, like I was mentioning earlier, no need to register with a prefix. Just register the variable, and everything that uses uses it downstream should have access to it(anything upstream would not however).

[scher200]

Thank you @kcmerrill

[kcmerrill]

So just a heads up ...

I ended up reverting my code in this regard. Although I didn't want to, it broke 2 other main workflows that were really common. The the original idea of prefixing variables will be the preferred path going forward.

Sorry for the back and fourth. I had some good learnings this week in regards to this.

[scher200]

@kcmerrill no problem. Do you have a clever/quicker way of notating this?:

{{ with $prefix := .Vars.prefix | printf "%swho" }}{{ $prefix }}{{ end }}

[kcmerrill]

So this is a good thorough solution, but I'm lazy so I'd probably assume that an argument passed in would be the prefix/postfix. If it's not set appropriately then fail the task.

[scher200]

sorry, batman another time :)