default runtime images can not be removed

[thesuperzapper]

Currently when you attempt to remove one of the default runtime images, say Pandas 1.1.1, you get the error:

Removal of instance 'pandas' from the runtime-images schemaspace is not permitted!

The JupyterLab logs show:

[E 2022-02-14 07:18:24.837 ElyraApp] Removal of instance 'pandas' from the runtime-images schemaspace is not permitted!  Resource conflict at '/opt/conda/share/jupyter/metadata/runtime-images/pandas.json'                                                                                                                                                  
[W 2022-02-14 07:18:24.837 ServerApp] 403 DELETE /notebook/kf-development/elyra-3-6-0/elyra/metadata/runtime-images/pandas?1644823104409 (127.0.0.6): Removal of instance 'pandas' from the runtime-images schemaspace is not permitted!                                                                                                                      
[W 2022-02-14 07:18:24.837 ServerApp] 403 DELETE /notebook/kf-development/elyra-3-6-0/elyra/metadata/runtime-images/pandas?1644823104409 (127.0.0.6) 2.38ms referer=https://xxxxxxx/lab/workspaces/auto-b 

[thesuperzapper]

In a similar vein, are we considering adding a "runtime image catalog" feature as we have for pipeline components?

[kevin-bates]

The images that come out of the box are considered system-owned and cannot be deleted. They can be updated, in which a copy is made and persisted to the general user location. Those updated copies can be deleted, but the originals can't.

In a similar vein, are we considering adding a "runtime image catalog" feature as we have for pipeline components?

No, but that's an interesting thought. I'd love to figure out a way to NOT have "system-owned" behaviors.

Perhaps introducing the notion of "metadata packages" could be interesting, where a package includes pre-built instances. I think we'd need something like - if the instance already exists, it is not installed. This would allow for updates to override the packaged instance. The problem would then be how to not re-install a deleted pre-packaged instance. So you'd probably have to record installation at a package level, but then have a mechanism for introducing new instances when a new version is made, etc.

Hmm, we'd also have to figure out the expected behavior for when someone removes a metadata package. Do the instances remain? Or only those instances that have been updated (implying a checksumming/hash registry), etc.?

[ptitzler]

For reference: For source code Elyra builds its possible to customize which images are system-owned images by modifying the content of https://github.com/elyra-ai/elyra/tree/master/etc/config/metadata/runtime-images

[ptitzler]

Another possible alternative: Perhaps we could introduce a configuration setting that, if defined, causes the metadata service to ignore system-owned entries. This way these entries would not be listed in the UI or CLI. This would not change the default user experience and require an explicit action by the entity that installs Elyra, but does not require a custom-built Elyra. wdyt @kevin-bates @kiersten-stokes?

[kevin-bates]

This would be a matter of constraining the multiple paths to the preferred directory when gathering the paths to use. That said, I'd love to find a different (and optional) way of exposing these system-owned instances, but prior to that, a configurable for this makes sense.

Things get a little trickier if this needed to be done at a schemaspace level, but that doesn't appear to be the case since runtime-images are the only schemaspace we provide instances for out of the box.

[ptitzler]

That said, I'd love to find a different (and optional) way of exposing these system-owned instances, but prior to that, a configurable for this makes sense.

An approach similar to the one we are currently using for connectors should be able to address not just the symptom (there are metadata instances the user has no need for - which my earlier alternative would do) but would also support customization:

• create and publish a package, comprising primarily of the resources (system-owned metadata definitions => JSON code) and glue code that supports a new entrypoint
• the Elyra-provided package is declared as an optional extra that is included during pip install elyra[all], mirroring the current out of the box behavior ("there are preconfigured instances")
• at installation time one can decide wether or not to include the extra (by [not] installing the dependency)
• the metadata behavior would not change - these instances would be treated the same way they are today (can't be removed via the GUI or CLI)
• if desired, one could create a custom pip-installable package version

[kevin-bates]

yeah, this is probably better, although, today, we can edit system-owned instances where the "edited instances" are stored in the user's location - rather than sys.prefix. It's that behavior I'd really like to figure out how to address (i.e., remove). If we say these are read-only, then I think adding a meta-behavior that indicates a given instance is system-owned might be an improvement as the instances can then live next to user instances.

Another thought might be to introduce Bring-Your-Own-Instances where the Schemaspace class loads entrypoints relative to itself and those entrypoints provide instances. I think BYO instances would be used where common deployments are necessary. The behavior could be that if the BYO instances are only created. This would allow for modifications to the instance to survive server restarts. The issue there is that any deleted instances would get revived, so it would be nice to figure out a way to address that.

[ptitzler]

Perhaps we should reevaluate whether this system-owned RTI metadata is even necessary. If I recall correctly, system RTIs were provided to provide a smother up-and-running experience. My thoughts:

• For container based deployments, pre-seeded container images can be provided through other means:
  • publish -slim image (e.g. elyra-slim), which don't have any RTIs pre-defined. Users can extend this image FROM ..../elyra-slim and add elyra-metadata install runtime-images ..., specifying their own images.
  • publish elyra image, which extends above image using a small set of images (a subset of what we include today)
• Pre-seeded RTI metadata is useful for getting started scenarios, but in many cases it'd be just one additional step to add it to an Elyra instance that doesn't have it defined. (Additional tutorial artifacts can be provided to create the metadata on as as needed basis. For example, https://github.com/elyra-ai/examples/tree/master/pipelines/dax_noaa_weather_data only requires one RTI instance to be fully functional.)
• As is, system-owned RTIs don't really solve many problems: (1) users can overwrite them and (2) users can add/use their own, hence the mechanism does nor enforce any restrictions that an admin might want to impose.

[kevin-bates]

This would be great! It does seem like most of this could be handled via documentation and within images we provide - as you mention. System-owned metadata is also problematic for other storage solutions (which we don't have at the moment, but will be an immediate problem when that does occur).

We could also document how users could export/import the images they wish to retain. (Of course, we'd need the necessary import functionality in place, but I'm assuming this (removal of system-owned RTIs) is something slated for 4.0.)

[ptitzler]

We could also document how users could export/import the images they wish to retain. (Of course, we'd need the necessary import functionality in place, but I'm assuming this (removal of system-owned RTIs) is something slated for 4.0.)

Yep. Metadata import should be available no later than 3.8. We could also add the current files to the examples repository or document the corresponding CLI commands, which are straightforward for RTIs.