a

[benoitf]

Is your enhancement related to a problem?

No matter how fast we get to bootstrap a Che workspace, no matter how many external resources we are able to pre-pull (images, extensions, examples source code), we will always need to wait 20+s for a PV to be attached and mounted on the workspace pod.

Describe the solution you'd like

New Workspace lifecycle:

1. Workspaces Startup phase
   • Pods (workspace and data sync) are started in parallel
2. Startup data sync phase:
   • Data flow goes from the persistent volume (rsync server) to the ephemeral volume (rsync client)
   • Containers in the Workspace Pod are started but are not allowed to write in the ephemeral volume
3. Normal workspace usage phase
   • Containers in the Workspace Pod have full R/W access to the persistent volume
   • Data flow goes from rsync client to rsync server
4. Workspace Shutdown phase
   • Containers in the Workspace Pod are stopped and data are flushed to the ephemeral volume
5. Shutdown data sync phase
   • Data are transferred from the ephemeral volume to the persistent volume
   • Pods are destroyed

Workspace components in Read-only mode

In the "Startup data sync phase" the user will already be able to use the editor and plugins but those should behave in a read-only mode until all the data has been synced to the ephemeral volume. That means that Che editors (for example theia) should be able to work on read only mode (initially this can be done by showing a progress bar that shows the data sync and not allowing the user to access theia).

rsync protocol

Rsync is mentioned as the remotes files synchronisation protocol but that’s just an example. If there is a better alternative, let's use it.

Ideas to improve performances (even more)

• The rsync server pod could be "always up" and its lifecycle and the workspace lifecycle would be distinct.
• The rsync server pod could serve every user workspace
• When a workspace is created for the first time the Startup data sync phase can be skipped

[benoitf]

Hello,

here are some notes:

about data sync pod:

• The service needs to be almost always on to be sure that we don't move the slow down to start a workspace to start this data sync pod.
  • based on some discussion, it may be a per user pod so the projects data are kept on the user namespace.
  • based on another discussion, it may allow to spawn need pod elsewhere that are requiring for example only read-access to the /projects folder and it could be client of the data sync pod as well.

optimization

• as it may require a cost to have this service always on:
  • idling after some hours
  • it can be started as soon as for example the user connects to the dashboard

One the goal is to be able to start the workspace as fast as possible. For that it means that we create a new workspace (no previous state):

• data-sync-pod service should reply as soon as possible that there are no existing data for this given workspace. (even before trying to mount the PVC). By mounting PVC ‘asynchronously’, it means that the workspace can boot quickly, projects can be cloned and persistence of the workspace will be done later, when the PVC is up. (IDE will notify like with a status bar that it starts with ephemeral storage and then backup/persistency is there)

If there is previous data, IDE needs to wait to have project restored before displaying full layout.

Storage synchronization:

• Prefer zip/tgz unpack over cp/rsync for workspace boot (like 5s to unpack a 300MB tgz file) vs 1mn30 to copy/sync files
• Issue of zip/tgz: will increase the size of PVC (need to have it both unpacked and packed archive). On che.openshift.io it seems it's easier to get other PVC with 1gb than increasing the PVC size ?
• have a way to exclude or not node_modules folder (or other folders) so zip/tgz file could be reduced but it implies that we download again the dependencies when workspace is restored so it may take time as well...

optimization: it could cleanup 'unpacked' folder and only keep zip files if files were not used since a lot of time.

theia enhancements :

• Wait that we have ‘ready’ event before booting extensions/plug-ins and displaying UI of the IDE (should be inside the IDE but with minimalist state and display status update)
  • On a fresh workspace start, if persistent volume is not yet mounted, IDE is still importing the sources but notify the user that it’s not yet persisted.
  • When event ‘sync finished’ is thrown, notify the user that now the persistent mode is enabled.

Another optimization: For now, import/clone of source code is performed when we're entering into the IDE. (it's useful if some 'private' repository is accessed as we may need the github token and have oAuth, etc) but in the case of a public repository, if the project is cloned as soon as possible, it means that we could enter into Theia by having already the project cloned previously or in parallel. So it might speed up again the process. --> needs another Epic just for this specific item.