walt node cp may have the ability to transfer the online file directly to an image

[sandikkaya]

The expected behavior of the cp command is to copy files from left side argument to right side argument. However, in the current implementation when one of the arguments is a node and the other one is an image, you need to call two commands in order. First, to copy a file from a node to your local storage with walt node cp. Then, copying the local file into the image with walt image cp.

These two steps could be joined into the single command of walt node cp maybe with a --transfer switch.

The opposite direction could also be considered. Adding the same ability to walt image cp to transfer files directly from images to nodes. However, I think this would be only necessary when someone requires a file from an image that is not used to boot the node temporarily on the node. I assume, the use case for this is limited.

[eduble]

I think I understand the use-case. This could be, for example:

• You have a node <N> booted from image <I>.
• While debugging you realize that a file must be modified. You modify on <N> directly in order to test in real conditions. It works.
• Then you would like to apply the same change on image <I>.

Is it the kind of scenario you were thinking of? We also have to keep in mind that, in this case, image <I> will be modified, thus it will have to be unmounted / remounted for changes to be applied, and nodes attached to <I> (including <N> obviously) will reboot. Is it acceptable?

[sandikkaya]

Yes, this is exactly the use case. The user may be notified before the copy operation that all connected nodes will be rebooted.

Actually, as I mentioned in the previous issue, when an image is at the right hand side of a copy command, the user must be notified beforehand.

[eduble]

OK I see.

[eduble]

For now we have:

• walt node cp for transfers between the client and a node, in one way or the other
• walt image cp for transfers between the client and an image, in one way or the other

Thus, if we want to make transfers between an image and a node, we have to clarify how the semantics of these commands evolve.

Currently, parsing arguments is quite simple:

1. we check which of the two arguments has no : character, and this is the path to the client file
2. we analyse the other argument, which must be <node>:<path> for the first command and <image>:<path> or <image>:<tag>:<path> for the other.

If one of those commands accepts, for instance, <node>:<path> for one argument and <image>:<path> for the other, we will have to look a little more (and handle possible ambiguities, for instance, if an image and a node have the same name). But that's still technically feasible I think.

What worries me a little more, is the clarity of such a command, for the user. If he reads this, for instance, in a tutorial:

$ walt node cp rpi-wifi:/root/script.sh rpi-disturb:/tmp

There is no way to know which is the node, which is the image.

An option would be to allow tags, to clarify:

$ walt node cp image:rpi-wifi:/root/script.sh node:rpi-disturb:/tmp

At the end, we have to make sure all this will really simplify the life of the user...

[eduble]

Allowing the - character for stdin & stdout could allow this:

$ walt node cp <node>:<path> - | walt image cp - <image>:<path>

But in this case, walt image cp could not implement a confirmation message (since stdin is used for the pipe). We might detect this case and just print a warning, since we cannot do better. What do you think?

[sandikkaya]

Considering two cases:

$ walt node cp : - | walt image cp - :

is a unix-style option. I can live with that. It is feasible. On the flip side, it requires a modification in standard streams and we rewrite walt commands twice.

The other option,

$ walt node cp rpi-wifi:/root/script.sh rpi-disturb:/tmp

is not clear at first and ambiguous for the user. However, we know that the only enabled possibility will be a copy operation from a node to an image. Therefore, it is clear that the left hand side is the node and the right hand side is the image. I opt to add an option here to emphasize this. Also, when cp is used with an option, it will be easier to parse the inputs as we will clearly know the perfect placement of the parameters. So, my tendency is to the modified second option, which is more walt command style:

$ walt node cp --node2image rpi-wifi:/root/script.sh rpi-disturb:/tmp

[sandikkaya]

One other possibility is to invent a new command, such as file-transfer. But, I assume this introduces more complexity and ambiguity. At the end, copy is synonymous with file transfer.

walt file-transfer node:file image:file

[eduble]

However, we know that the only enabled possibility will be a copy operation from a node to an image.

How do we know that?

is a unix-style option. I can live with that. It is feasible. On the flip side, it requires a modification in standard streams and we rewrite walt commands twice.

Actually this is easy to implement because both walt node cp and walt image cp commands already work internally like this: tar left-hand-side | untar to right-hand-side. I think I prefer that we go this way, because that allows us to keep walt commands simple and with easy-to-understand semantics. And anyway, we will have to add a topic in documentation about these commands because it is missing (e.g. walt help show transfers).

[sandikkaya]

We know as copying a file from a "persistent" image to "temporary" node does not make sense. In contrast, the reverse is useful to keep the running state of a node. Of course this is my opinion. Under some circumstances, one may want to copy a file from a third-party image into a running node.

I think documentation is the easiest part. After the decision, I can handle that.

[eduble]

OK. Last option I have been thinking of: If we just want to handle the specific use case you are mentioning, we could consider allowing a specific keyword, such as booted-image. And one would write:

$ walt node cp <node>:<path> booted-image

WalT would transfer the file or directory to the same <path> on the image the node has booted. (If you confirm that the image the node has booted is the only relevant image for this use case.)

[sandikkaya]

Yes, indeed. I believe this will be the most common case for users and it is apparently what I am continuously doing during debugging.