[Research][Spike] Backstage Golden Path Template (GPT) for `odo init`

[feloy]

/kind feature

Research to help estimate Q4 epics:

• Is it possible to call odo from Backstage template steps (including how to get odo present)?
• Do we need a custom input for listing Devfiles?

Which functionality do you think we should add?

Why is this needed?

Related Epic: https://github.com/redhat-developer/odo/issues/7091

[rm3l]

Proposal

@kadel @feloy Following some discussions with you, below is a proposal for a first version of the template. Let me know your thoughts.

The Backstage Golden Path Template for odo init could be made of the following:

• [ ] DevfileSelector custom field extension
  • Input would be a Devfile Registry URL (default value is the public registry)
  • And it would use the /v2index to populate a list of Devfiles available from the registry
  • Output is the devfile selected
• [ ] DevfileVersionSelector custom field extension
  • Input is the Devfile Registry URL + the selected devfile
  • Output is the devfile stack version selected
• [ ] DevfileStarterprojectSelector custom field extension
  • Input is the Devfile Registry URL + the selected devfile + the selected devfile version
  • Output is the starter project selected
• [ ] odo:component:init custom action running odo init
  • For now, it would require the odo binary to already be available in the environment running the Backstage instance (I think this is the simplest to start with).
  • Input would be all of the above + the component name

Once we have the items above implemented, we could provide a very simple template with the following:

• Parameters:
  • Devfile Registry URL (default value is the public registry)
  • DevfileSelector*
  • DevfileVersionSelector*
  • DevfileStarterProjectSelector*
  • Component Name*
• Steps:
  • odo:component:init
  • (?) Do we want to generate a catalog-info.yaml file in the workspace? This can be useful to register the new entity in the software catalog.
  • publish:github
  • (?) If we choose to register the new entity in the catalog, we will need to call catalog:register
• Outputs:
  • Repository remote URL
  • If we choose to register the new component in the catalog, link to the entity in the catalog

---

For more context, here is a more detailed summary of my investigations:

Approach 1: Cloning all Devfile starter projects along with their stacks

Given the structure of a typical Backstage Software Template (a template.yaml definition file and a skeleton directory containing all the necessary files and directories parameterized), I initially thought the UI could display a (manual) list of Devfiles to pick, and depending on what Devfile users pick, they would be shown dedicated sections to further customize the project.

Example


Pros

• Simple to understand as we have everything under the same GPT
• Allows for finer-grain customization (for example, we could expose a field allowing users to set the artifactId in the resulting pom.xml file for Maven projects), which IMO is more in line with how Backstage users create a component from a template
• No need for the odo binary, as everything can be handled in the template definition using built-in steps and actions

Cons

• Very high maintenance overhead: need to maintain everything under the skeleton directory as the starter projects and Devfile stacks evolve in the registry
• Huge template.yaml definition, as we would need to manually fill the Devfile list selector and the conditional fields displayed to the end user

Approach 2: Custom field extensions + custom actions + template

To alleviate the maintenance overhead, the idea here is to make the Template display fields that could be dynamically populated from the Devfile Registry Index. Such fields are the list of Devfiles, the Devfile versions, and the starter projects. This sounds feasible using Backstage custom field extensions. Once the UI is done, we need to perform actions like pulling the Devfile Stack version and starter project, publishing it to a repository service, and registering it in the Software Catalog.

Pulling the Devfile Stack can be done either with odo or by pulling it directly from the Devfile Registry.

Using the odo binary

We could provide a custom action, say odo:component:init, which would take care of executing odo init passing it the right arguments. From my investigations, it is possible to execute a shell command from a Backstage custom action. plugin-scaffolder-git-actions is a good example as it requires Git to be installed in the environment running the Backstage instance: https://github.com/arhill05/backstage-plugin-scaffolder-git-actions/blob/master/src/actions/git.ts#L37-L43

This has the benefit of opening up other usages relying on the odo binary in the future.

WARNINGS:

• Beware of potential security issues as we'll be passing user input to shell commands like odo. We need to check/sanitize those inputs.

However, comes the question: how do we expect odo to be present?

• i) either as a requirement (the user needs to install it in the environment running their Backstage instance),
• ii) or we publish an NPM Module wrapper that will automatically download the right odo binary at postinstall time if not already present in the environment. Then there is the problem of airgap installations, which we might work around by requesting users to manually download and install odo in the environment running their Backstage instance.
• iii) or we publish an NPM Module wrapper embedding all binaries of odo for all possible variants. And at postinstall time, we would install the right one for the current architecture and system. However, this will result in a big NPM module.

Pulling directly from the Devfile Registry

The idea here is to mimic the behavior of odo init without relying on the odo binary. Currently, odo uses the Registry Library to pull Stacks via the ORAS library (because Stacks are stored as OCI artifacts). So instead, we would need to:

• download Stacks using either raw HTTP calls or another JS library, like https://github.com/Pixeladed/oci-registry-js
• parse the Devfile to find the specified starter project. Depending on how it is defined in the Devfile, we would need to use for example Git or ZIP to handle it

[rm3l]

Closing this issue as complete. https://github.com/redhat-developer/odo/issues/7114 has been created to track everything needed to implement the Golden Path Template for odo init.

/close