Devfile Registry should provide the stack/sample associated files & artifacts

maysunfaisal commented 1 year ago

/kind user-story

Which area this user story is related to?

/area library /area registry

User Story

As a devfile registry user, I would like to access the files & artifacts associated with a Devfile stack/sample hosted on the Devfile Registry.

For example, let us consider this Quarkus Sample Devfile, it has the following Kubernetes component:

- name: kubernetes-deploy
    attributes:
      deployment/replicas: 1
      deployment/cpuLimit: '100m'
      deployment/cpuRequest: 10m
      deployment/memoryLimit: 250Mi
      deployment/memoryRequest: 100Mi
      deployment/container-port: 8081
    kubernetes:
      uri: deploy.yaml
      endpoints:
        - name: http-8081
          targetPort: 8081
          path: /

While the file deploy.yaml is available in the GitHub repository here as expected, the same file should be available on the Devfile Registry. To further clarify, the above Devfile is available on the Devfile Registry here but there is no deploy.yaml file at the expected location like https://registry.devfile.io/devfiles/deploy.yaml

This situation also applies to any other files or artifacts that are hosted on the repository and whose relative path is mentioned in the Devfile. Currently, application-service works around this issue by using the raw Github link like https://raw.githubusercontent.com/devfile-samples/devfile-sample-code-with-quarkus/main/deploy.yaml instead of a Devfile Registry URL.

Acceptance Criteria

[ ] All the files and artifacts hosted on a stack/sample Github repository and referenced relative in the Devfile should be hosted on the Devfile Registry
[ ] The hosted file path should be well thought out, because Devfile stacks have versions
[ ] The hosted file path should also account for the exact relative path as specified in the Devfile i.e.; if a file is available at /my/path/deploy.yaml in the Devfile then the file should be hosted at /my/path
[ ] Devfile is currently hosted at https://registry.devfile.io/devfiles/code-with-quarkus, so we need to think about how the files and artifacts would be available as an endpoint after also taking versioning into consideration

michael-valdron commented 1 year ago

While the file deploy.yaml is available in the GitHub repository here as expected, the same file should be available on the Devfile Registry. To further clarify, the above Devfile is available on the Devfile Registry here but there is no deploy.yaml file at the expected location like https://registry.devfile.io/devfiles/deploy.yaml

Since /devfiles/:name is a REST API endpoint for pulling devfiles I think we should expect something a bit different here, e.g. /devfiles/:name/resources/:resourceFilename or in this case /devfiles/code-with-quarkus/resources/deploy.yaml similar to starter projects.

github-actions[bot] commented 1 year ago

This issue is stale because it has been open for 90 days with no activity. Remove stale label or comment or this will be closed in 60 days.

michael-valdron commented 1 year ago

Discussing the acceptance criteria further before we size.

michael-valdron commented 1 year ago

My current proposal:

To avoid mixing with the REST endpoints, we can serve the raw stack content on a different route, this would work similar to the static route already there for stacks but work need function to receive the resources since they are stored remotely (as shown on GitHub) or under archive.tar on the server side. This behind the seens function would work similar to the starter projects but serve the route similar to static ones. Examples:

/raw/code-with-quarkus/devfile.yaml

/raw/code-with-quarkus/deploy.yaml

/raw/code-with-quarkus/<version>/devfile.yaml

/raw/code-with-quarkus/<version>/deploy.yaml

/raw/go/devfile.yaml

/raw/go/kubernetes/deploy.yaml

michael-valdron commented 1 year ago

Added a discussion topic on this issue under the community call.

Jdubrick commented 9 months ago

@michael-valdron was there any info from that discussion topic in the call that should be added here to help with sizing?

michael-valdron commented 8 months ago

@michael-valdron was there any info from that discussion topic in the call that should be added here to help with sizing?

Have reviewed the community call discussion topic on this, the approach that was the final decision for this issue was proposed by @feloy:

Everything past /devfiles/:name or /devfiles/:name/:version is captured as a path to stack resources, e.g. /devfiles/go/2.3.0/kubernetes/deploy.yaml should return the deploy.yaml content for the go stack. Also, /devfiles/go/devfile.yaml would return the same result as /devfiles/go since this is the REST endpoint that returns the devfile content.

This would allow the use of the relative URIs to be used with the base devfile endpoint slug to fetch the resources without the need to download them.

Note: It is worth mentioning that there needs to be an exception for /devfiles/:name/starter-projects/:spName as this is an existing REST endpoint that would conflict with this solution if it shares the same names.

michael-valdron commented 8 months ago

Note: It is worth mentioning that there needs to be an exception for /devfiles/:name/starter-projects/:spName as this is an existing REST endpoint that would conflict with this solution if it shares the same names.

We could expand this to improve the handling of offline starter projects by picking whichever is available, currently offline starter project require altering devfiles on build which could add complexity to dynamically changes registry content #1452.

michael-valdron commented 7 months ago

Update acceptance criteria to match accepted solution: https://github.com/devfile/api/issues/1073#issuecomment-1958161771

github-actions[bot] commented 1 month ago

This issue is stale because it has been open for 90 days with no activity. Remove stale label or comment or this will be closed in 60 days.

devfile / api