Support for multiple versions of generator

[derberg]

So generator reach out v2 dropping support for some node versions: https://github.com/asyncapi/generator/releases/tag/v2.0.0

Normally according to https://github.com/asyncapi/cli/issues/573 we should bump generator in cli and release new major of CLI. Basically generator drops support for node prior v18 and cli still supports older versions

but this is not a problem

problem is the generator functionality that we have, to prevent templates from failing with new generator version that may have breaking changes, we have such setting: https://github.com/asyncapi/java-template/blob/master/package.json#L102

so if we bump v2 generator in CLI, all the users of templates, will be blocked until templates developers will update config to support v2. Makes sense

so yeah:

• do we start supporting both generators here, and have a logic that basing on the config of the template, different generator is used
• or should we assume people use CLI properly, they use always a fixed version, and the same with templates - always specific version. So they will not notice issues, and basically upgrade only when they feel a need and other projects are ready?

Scope

• [ ] Discover how we can support both generators (v2 and < v2)
• [ ] Implementation
• [ ] Add new tests
• [ ] Communicate the change

[Amzani]

or should we assume people use CLI properly, they use always a fixed version, and the same with templates - always specific version. So they will not notice issues, and basically upgrade only when they feel a need and other projects are ready?

When you say they use always a fixed version you mean templates version right? this is something we could get from insights we are collecting (cc @peter-rr )

[derberg]

I mean, that in theory people in production should never install CLI as npm install @asyncapi/cli but npm install @asyncapi/cli@1.0.0 for example.

Same with templates, in production, in pipelines they should run generation with templates not pointing like here, to latest: asyncapi generate fromTemplate asyncapi.yaml @asyncapi/nodejs-template but for example asyncapi generate fromTemplate asyncapi.yaml @asyncapi/nodejs-template@1.0.1.

The problem is - can we assume people do it properly, can we "expect" them do it like this - and have it as defined assumption - so they always manually decide to use new version? the problem with this assumption is that it is not the best DX 🤔 So probably philosophy here should be to always assume people not always read docs, and we should somehow proactively protect them.

From my perspective, always better to enable support for new generator version (or optimizer or anything else) and set the older one to deprecate with some period to have it enabled, and then removed 🤔 really hard to decide

[derberg]

From my perspective, always better to enable support for new generator version (or optimizer or anything else) and set the older one to deprecate with some period to have it enabled, and then removed 🤔 really hard to decide

no other option really. We can educate through docs: https://www.asyncapi.com/docs/tools/generator/versioning but this is in my opinion just to make our lives easier, not users

[peter-rr]

When you say they use always a fixed version you mean templates version right? this is something we could get from insights we are collecting (cc @peter-rr )

Sure! Currently we are collecting the template used, and also the CLI version if specified by the user, according to the documentation. I guess we could collect the template version as well if needed 🤔 https://github.com/asyncapi/cli/blob/f9a6303997e9e8ea7fd8618aab436eca1a5e94de/src/commands/generate/fromTemplate.ts#L142

So the different use cases are:

• asyncapi generate fromTemplate asyncapi.yaml @asyncapi/html-template will produce:

  asyncapi_adoption.action.finished       COUNTER {
  user: '5db8f33f-320c-4d50-85a5-7e8135c33380',
  action: 'generate:fromTemplate',
  template: '@asyncapi/html-template',
  success: true,
  asyncapi_version: '3.0.0',
  asyncapi_servers: 1,
  asyncapi_channels: 1,
  asyncapi_messages: 1,
  asyncapi_operations_send: 0,
  asyncapi_operations_receive: 1,
  asyncapi_schemas: 4
  }       1

• asyncapi generate fromTemplate asyncapi.yaml @asyncapi/html-template@1.8.4 will produce:

  asyncapi_adoption.action.finished       COUNTER {
  user: '5db8f33f-320c-4d50-85a5-7e8135c33380',
  action: 'generate:fromTemplate',
  template: '@asyncapi/html-template@1.8.4',
  success: true,
  asyncapi_version: '3.0.0',
  asyncapi_servers: 1,
  asyncapi_channels: 1,
  asyncapi_messages: 1,
  asyncapi_operations_send: 0,
  asyncapi_operations_receive: 1,
  asyncapi_schemas: 4
  }       1

[Amzani]

We can leverage also the efforts @Shurtu-gal is making to improve DX CLI, for instance if we want to force users to explicitly mention versions we can show a documentation link that shows the versions map between generator and templates.

[derberg]

ok, so we look at a solution to support different version of generator in the CLI, right?

[Amzani]

Yes @derberg I think it's the most DX friendly option.

[derberg]

are you, maintainers, will work on it or you have no capacity and rather expect contribution?

[Amzani]

I think @peter-rr or @AayushSaini101 will be interested in picking it up with some support.

[Amzani]

We can submit this as a bounty issue for Q3-2024

[derberg]

ok, but then we're talking about July. So until July nobody will be able to use new AsyncAPI Generator

[peter-rr]

ok, but then we're talking about July. So until July nobody will be able to use new AsyncAPI Generator

@Amzani Should we add then this issue to the current DX Working Group's cycle?

[Amzani]

@peter-rr done

[peter-rr]

Scope update:

• [x] Discover how we can support both generators (v2 and < v2)

Working on the implementation now.

[peter-rr]

@Souvikns I'd like to continue with the implementation of the issue but I won't be available for the next 2 weeks, then I guess you better pick it up so I'm not blocking any other issues or bugs related to generator. Feel free to assign it to you. Go ahead! 👍

[Souvikns]

After some research looks like one way of solving this is reading the package.json file of the template and using v2 or v1 generator accordingly. Only thing is generator loads the template https://github.com/asyncapi/cli/blob/e8dc61095cfd08d1981a5dc21d1852bf2050130f/src/commands/generate/fromTemplate.ts#L264

Generator expects a name of the template as a string. So we have two ways to solve this in CLI or generator.

@asyncapi/generator

In generator we read the package.json file call the old logic or the new logic, i.e. generator provides support for both v2 and v3 spec at the same time.

@asyncapi/cli

In cli before passing in the template we have to download it and then check the package.json file and then accordingly call v2 or v1 of generator. Only downside with this is see is that the template will be downloaded twice as generator itself is downloading it.

@derberg what do you think I should continue with.

[derberg]

not doable in generator, thing is that generator recently reached v2 just because of dropping support for some node versions - not really possible to handle such stuff in a different way.

and yeah, pre-downloading of packages also seem like an overhead :( downloading will not take place twice though, as arborist package that we use will figure package is already available locally - still this is complex solution - installation is not trivial, there are private packages and other concepts out there that we had to implement in generator, so replicating it in CLI really do not sound like good approach.

so maybe in the end we need something in the generator? 🤔

or we just do it "manual" that it is a user of CLI that decides which generator version to use 🤔

[Souvikns]

Alright, for now we can have a flag where the user can specify which version of the generator to use, not the best solution but lets the user atleast succesfully generate from all templates.