JHC - DevXP - Improve the generator-jhipster filetree

[Tcharl]

Overview of the feature request

This request would ideally be a topic of the 'Improve the generator developer experience' animated by @MathieuAA and me. But if you want to fix it now, feel free to do it for sure! We'll find some other ones :-p

The generator-jhipster/generator repository layout looks a bit messy right now, for example:

• In the generator folder, there are three kubernetes subfolder
• There's a cypress folder, while the protractor configuration is located in their respective client/src/test/e2e folders
• There are multiple folders for 'commands': spring-service, spring-controller, pages/vue
• There's a language folder containing i8n for language test, while the i8n is in the client folder
• The generators/export-jdl, generators/upgrade, generators/upgrade-config which doesn't seem to be at the right place (not really things that looks to be a template or a template helper)
• all the apis at the root (may deserve a good api vs impl split with adapters in order to be able to keep retrocompatibility) ...

Let's clean this up, WDYT?

I propose to have this kind of layout:

1) root: keep it like it :-): the skill aren't the same for metamodellers, cli implementers and generator contributors 2) the generators folder: let's reproduce the exact same layout as the 'meta' options presented on the first jhipster.tech page (client, server, deployment, ...) 3) the templates: I'm thinking of three subfolders: entities (like the actual entity-X folders), commands (in order to move the actual spring-controller - like folders) and a 'main' one (the actual -server) 4) the technology: vue, angular, react, ... (we can even imagine that the option in the site can be generated from this layout, which will enforce the layout :-p) 5) the usual maven layer (src/main/java|resources|javascript, ...)

We can switch 3 and 4, I'm not really sure about the order.

Other potential guidelines:

• no common folder: it's not microservice compliant and not really meaningful, prefer meta in case of meta or composer template, otherwise I would recommend to duplicate
• focusing on doing template technology specific with fewer code possible and meta template (so that experts can focus on framework they love): it will be the purpose of some other tickets, but may impact this one.
• focus on having a folder architecture which is similar to jdl syntax or similar to target hierachy: if a new folder is not relative to one or another, please engage a discussion (or at least think about it).

Motivation for or Use Case

Attracting new contributors is a question of life and death in OSS (at least key for success), and as @jdubois mentioned in his first JHipster day in Paris: Developer experience is essential and has always been something that Jhipster is targeting. Let's now improve the developper experience on the generator itself to grow our community, as well as for us to keep enjoying contributing.

Related issues or PR

I did not noticed any

• [x] Checking this box is mandatory (this is just to show you read everything)

[pascalgrimaud]

It's always nice to have a clean code, but it needs to be discussed in details. All our internal system is used in blueprints. If we change everything, we take the risk to break all existing blueprints:

• kotlin, nodejs, micronaut, quarkus, etc...

Our expert here is @mshima

[Tcharl]

Hi Pascal,

Totally agree for the need of discussion on how to do it. However, I think we have to clean this up the sooner otherwise it can only become worse. Do we have the list of all blueprints source code? We may also provide merge requests for them :-)

[pascalgrimaud]

We have:

• official blueprints, in our organization
• open source blueprints (it can be easy to search in our marketplace)
• private blueprints in different companies

So it's hard to say. Each changes must be deserved (for example common folder) and we need to write documentation to help our users for each changes.

[mshima]

I agree we could improve that. A few points:

• sub-sub generators like jhipster:client:angular are possible but not recommended.
• the usual way to implement generators are feature specific: client client-angular client-*

• In the generator folder, there are three kuberenetes subfolder

Agreed probably we should have only 1, but I don't know what each one does.

• There's a cypress folder, while the protractor configuration is located in their respective client/src/test/e2e folders

This is something to discuss:

• generators
  • client
  • templates/angular
  • templates/common
  • templates/react
  • templates/vue

Or

• generators
  • client
  • client-angular
  • client-cypress
  • client-react
  • client-vue

• There are multiple folders for 'commands': spring-service, spring-controller, pages/vue

They are not commands, they are feature specific generators. I recently did some changes to the page generator and it can be REALLY useful for blueprints.

• There's a language folder containing i8n for language test, while the i8n is in the client folder

IMO languages files should be moved to client/server generators. But we should keep languages generator to add new languages like it is now. It will become a command but it's much easier to keep as a generator than another kind of implementation.

• The generators/export-jdl, generators/upgrade, generators/upgrade-config which doesn't seem to be at the right place (not really things that looks to be a template or a template helper)

• export-jdl: can be easily implemented at /cli as a command.
• upgrade-config: should be dropped. It's already done it's purpose.
• upgrade: it's quite complex, the generator queue architecture makes it simpler and it uses generator-base utility methods. don't worth to change that.

1. root: keep it like it :-): the skill aren't the same for metamodellers, cli implementers and generator contributors
2. the generators folder: let's reproduce the exact same layout as the 'meta' options presented on the first jhipster.tech page (client, server, deployment, ...)
3. the templates: I'm thinking of three subfolders: entities (like the actual entity-X folders), commands (in order to move the actual spring-controller - like folders) and a 'main' one (the actual -server)
4. the technology: vue, angular, react, ... (we can even imagine that the option in the site can be generated from this layout, which will enforce the layout :-p)
5. the usual maven layer (src/main/java|resources|javascript, ...)

We can switch 3 and 4, I'm not really sure about the order.

Other potential guidelines:

• no common folder: it's not microservice compliant and not really meaningful, prefer meta in case of meta or composer template, otherwise I would recommend to duplicate
• focusing on doing template technology specific with fewer code possible and meta template (so that experts can focus on framework they love): it will be the purpose of some other tickets, but may impact this one.
• focus on having a folder architecture which is similar to jdl syntax or similar to target hierachy: if a new folder is not relative to one or another, please engage a discussion (or at least think about it).

Our current unofficial guidelines about features are:

• If you add a server feature that needs client changes, you will need to implement in every client.
• You can add client only feature like page (vue) and needles (angular).

I am in favor to rearranje the source like this:

• don't allow any file inside generators/
  • generators/generators-base.js -> lib/generator/base.js
  • generators/generators-private -> lib/generator/private.js
  • generators/utils -> lib/generator/utils.js
  • ...
• utils -> lib/utils
  • utils/entity.js -> lib/core/entity.js
  • utils/field.js -> lib/core/field.js
  • utils/relationship.js -> lib/core/relationship.js Move there and convert to classes. They are misplaced now, but our entity generator was too bloated, I had to move them somewhere.

Once npm 7/workspaces are done, we could consider switching to monorepo and package deployments separately.

A point you are missing is the usefulness of our feature specific generators: I suggest to merge spring-controller with spring-server into spring-component. Example of a page config:

const pageConfig = {
  name: `coolAction`,
  actions: ['do-it'],
  createServerController: true,
  createServerService: true,
};

Our app generators does:

composeWith(`page`, {...pageConfig, arguments: [pageConfig.name]});

Our page can do:

composeWith(`spring-component`, {arguments: [pageConfig.name], createService: pageConfig.createServerService, createController: pageConfig.createServerController});

Now we have a complete page with backend integration.

[MathieuAA]

Once npm 7/workspaces are done, we could consider switching to monorepo and package deployments separately.

Why? What would we gain by doing that?

I suggest to merge spring-controller with spring-server into spring-component.

That'd be a good idea! One of the possible benefits would be to support more than one framework, or better: if a framework can be used in more than one architectural design, then we could generate files from this one framework in another architectural design (for instance, controller-service-repo & hexagonal architecture).

BTW @mshima, I like your idea of rearranging the folders.

Move there and convert to classes.

Not so sure about that one.

[Tcharl]

Hi @mshima ,

Thank you for your answer, it provided many useful info for sure :-).

Concerning the first partI would really be in favor of choosing the more hierarchical structure (the less flatten): it's easier to transform to a module if we decide to.

For the addition of the 'lib' folder, totally agreed! I would prefer more meaningful name: I can be wrong, but the frontier between what is utils/core vs what is utils.js vs what is base.js is not very clear. We could for example make the distinction between:

• Metamodeling to generator helpers
• Technology specific (liquibase, maven, ...)
• Extensibility API (blueprint and module)
• Generator fragments (entity queries, ...)

Concerning the last point: I totally understand feature specific generator, and really enjoy the feature! In the end, we may even use it for the generator itself (ensuring the completness of the of the feature :-p). I was just mentioning that at first sight when crawling into the repository, the folders doesn't really look at their right location. I was just trying to identify a way to store it somewhere:

• generators
  • server
  • commands
    • services
    • spring => here the actual spring-services folder content
    • controller
    • spring => here the actual spring-controller folder content
    • entity
    • main
  • client
  • commands
    • vue => Actual content of page

I don't know at which point yeoman is permissive for command (feature specific) location, and I hope this layout is possible

[mshima]

For the addition of the 'lib' folder, totally agreed! I would prefer more meaningful name: I can be wrong, but the frontier between what is utils/core vs what is utils.js vs what is base.js is not very clear. We could for example make the distinction between:

• Metamodeling to generator helpers
• Technology specific (liquibase, maven, ...)
• Extensibility API (blueprint and module)
• Generator fragments (entity queries, ...)

Not sure how do you think we could implement this. Mixins? The correct solution would be to implement using generators composability. But this would be a major design change.

compose.with('liquibase').then(liquibaseGenerator -> liquibaseGenerator.addChangelog('my-custom-changelog');

An optional dependency:

compose.once('liquibase').then(liquibaseGenerator -> liquibaseGenerator.addChangelog('my-custom-changelog');

Concerning the last point: I totally understand feature specific generator, and really enjoy the feature! In the end, we may even use it for the generator itself (ensuring the completness of the of the feature :-p). I was just mentioning that at first sight when crawling into the repository, the folders doesn't really look at their right location. I was just trying to identify a way to store it somewhere:

• generators

  • server

  • commands

    • services

    • spring => here the actual spring-services folder content

    • controller

    • spring => here the actual spring-controller folder content

    • entity

    • main

  • client

  • commands

    • vue => Actual content of page

I don't know at which point yeoman is permissive for command (feature specific) location, and I hope this layout is possible

That's possible but:

• probably I'm the only person that knows how to do it.
• can complicate blueprint support.
• it will make it completely incompatible with yo.

[Tcharl]

That's possible but:

What about symlinks? Mosts IDEs have different icons when a folder or a file is symlinked: so tha we could keep being consistent and yeoman compatible :-p

[Tcharl]

Let's cleanup what we can though, thus the folders that don't break yeoman compatibility. I think that we can anyway do something in that area.

[github-actions[bot]]

This issue is stale because it has been open 30 days with no activity. Our core developers tend to be more verbose on denying. If there is no negative comment, possibly this feature will be accepted. We are accepting PRs :smiley:. Comment or this will be closed in 7 days