Alternatives for live editing on REANA cluster

[diegodelemos]

Since we have experienced many troubles with minikube lately we have investigated the alternatives and compared:

	Shared volume support	Live feedback	Stable
Telepresence	no	yes	yes
Rebuild/restart	yes	no	yes
Volume mounting	yes	yes	no

Since we depend on shared volume mounting for REANA, Telepresence is not yet an option (maybe when we support S3 storage backend). Regarding the two other options, the only virtualization backend for minikube which supports volume mounting is VirtualBox but it is giving some problems from time to time, so rebuild+kill pods technique is the alternative (optimization of Dockerfiles will come to facilitate the workflow).

@reanahub/developers

[diegodelemos]

For rebuild+kill pods it is interesting to plug some trigger after save in your editor, for example for VS Code:

{
    "saveAndRun": {
        "commands": [
            {
                "match": ".py",
                "isAsync": true,
                "cmd": "full_component_name=$(basename ${workspaceRoot}) && component_name=${full_component_name/reana-/} && eval `minikube docker-env` && docker build . -t reanahub/$full_component_name:latest && kubectl delete pods -l app=$component_name",
                "silent": true
            }
        ]
    }
}

[lukasheinrich]

I think there is some limited volume support, where it uses sshfs to proxy the state, I haven't tried this though:

https://www.telepresence.io/howto/volumes

also there is Draft from Azure

https://github.com/Azure/draft

[diegodelemos]

I think there is some limited volume support, where it uses sshfs to proxy the state, I haven't tried this though:

https://www.telepresence.io/howto/volumes

Yes, this is precisely the link where no points to in the table. It is easy to setup for let say Kubernetes secrets which is exactly what they show as an example. For our use case, sharing workflow workspaces, when a developer works on for example yadage engine, she/he needs to have the real volume mounted in her/his computer and set it under $TELEPRESENCE_ROOT which brings us again to square 0 (where we have problems with volume mounting). Thats why I point out that if we support S3 interfaces we can more easily develop locally using Telepresence, but this is yet to come.

also there is Draft from Azure

https://github.com/Azure/draft

Regarding Draft, from what I saw there is not much difference between it and rebuilding+killing pods without using any new tool, looks like the bottleneck of rebuilding is still there from what I understand.