Language & Framework detection for Devfile Applications

elsony commented 3 years ago

(Edited, check history for original content)

Background

The devfile registry needs to contain tagging information to define the nature of a given stack stored in the registry. This tagging information allows tools like odo/Che to provide a filtering mechanism for users to find the stacks that are applicable to their user applications.

When generating the registry index from the devfile registry src, the registry builder goes through the devfiles to determine the stack or sample’s framework(projectType) and language. This can be seen in the generated registry index. These index fields depend on the devfile optional metadata field framework(projectType) and language.

However, this relies on the devfile mentioning the optional framework(projectType) and language fields.

Work

We would like to create a Go module that could be consumed by the devfile stakeholders so that we can identify the framework(projectType) and language in such cases.

The idea would be to parse the devfile and the application associated with it to determine these fields. We can start with the starterProject list to see what kind of application it is or browse the associated application directory if starterProject is missing to do the same. For instance, the presence of a build.grade is a gradle project and a pom.xml is a maven project, with the language being java.

We should design the module, so that anyone interested in leveraging the detection logic, can hook into it and integrate to their code.

Checklist:

[ ] Should take an application project dir or a path in general
[ ] Priority is devfile.yaml metadata > application > devfile.yaml project > devfile.yaml starterProject
[ ] Inspect devfile's metadata's framework(projectType) & language
[ ] Should be able to inspect an application project
[ ] Clone a devfile's project & starterProject (first one, if many are present) and inspect as a local application
[ ] Return the language and framework(projectType) for the input back
[ ] Reuse existing Language Detection library - https://github.com/go-enry/go-enry
[ ] Mapping logic should be implemented for framework detection, example- maven -> pom.xml, nodejs -> package.json, etc. and should be fairly simple (ideally contained within a single file)
[ ] If the application is a hybrid(both frontend and backend in the same repo) ie multiple languages - return a list of all the detected languages and framework

Stretch Goal

[ ] In the case of multiple language detection, we can return a probability % back with multiple entries.

Related Issue: This item is part of #82 #222

elsony commented 3 years ago

There is an existing framework https://github.com/redhat-developer/alizer for language and framework detection. It is currently written in java so it does not fit well into our devfile library. We should probably explore the possibility of sharing/reusing the detection rules (especially the framework detection) when working on this item.

elsony commented 3 years ago

Some example applications that can be used for testing application detection:

Each of the devfile under each stack folder (https://github.com/devfile/registry/tree/main/stacks) contains starter applications that can be used with a given devfile, e.g. for java-maven, you can find the link to the sample application under the starterProjects section:
```
starterProjects:
 - name: springbootproject
   git:
     remotes:
       origin: "https://github.com/odo-devfiles/springboot-ex.git"
```
Some other sample applications:

scottkurz commented 3 years ago

Some thoughts after listening to the 8/24 devfile cabal:

So..one can run any Maven Java EE / Jakarta / MicroProfile WAR (of the approporate spec/API versions etc.) on Open Liberty, and also run SpringBoot 2 applications on Open Liberty.

How does the framework decide to recommend between "Basic Spring Boot" vs "Open Liberty Maven" or between "Maven Java" and "Open Liberty Maven"? Should the Open Liberty devfile declare its support for SpringBoot somehow? Through a devfile tag?

Might we need to take into account the idea that certain devfile/stacks are maintained by the community for the specific purpose of being generic "targets" for projects without a devfile (while other ones aren't)? I.e. the community would bake opinions about preferred stacks into devfile tooling?

elsony commented 2 years ago

Framework requirements: https://docs.google.com/document/d/1HwKvQdXY8xTHY8o952fZPHV2qeJ0fauH0kGZGEq-zgc/edit#heading=h.wbt791jb16qi

API information: https://docs.google.com/document/d/1HwKvQdXY8xTHY8o952fZPHV2qeJ0fauH0kGZGEq-zgc/edit#bookmark=id.m773s63818vu

github-actions[bot] commented 1 month ago

This issue is stale because it has been open for 90 days with no activity. Remove stale label or comment or this will be closed in 60 days.

devfile / api