Make it possible to generate only given template files

[derberg]

Reason/Context

I'd like to use generator not only to scaffold my project when I start working on my application. I also want to use it continuously, every time I change something in AsyncAPI document.

The problem is that regeneration means basically another generation of the entire template. This leads to a process of regenerate -> carefully go through each change introduced after regenerating and ignore changes that will break your code

More context here.

Description

A partial solution for the problem, but also not very complicated would be a configuration I could pass to the library/cli where I could specify what files to include/exclude in a generation.

Let's say I use some Node.js template. I generated my app and later added some more dependencies for my business login. Later I want to regenerate, but by telling the generator to ignore package.json and package-lock.json files.

What do you think? is the use case clear? you think it makes sense?

[Souvikns]

How about we take a page from prisma migrations. We might have to change a lot of things, but what if the generator keeps track of the previous state of the AsyncApi spec and only update what has changed. So when a new channel is added, only that part of the code is generated.

[derberg]

interesting idea. Just like git has .git we could have .generator that would keep information about the state and somehow track changes 🤔

[jonaslagoni]

Yea I agree interesting suggestion, could be a solid feature, not sure if it can be done in a maintainable way, also from the perspective of template developers. Do you foresee any problems there @Souvikns? It is one thing where you have control over the entire workflow and another when other developers need to attach into such a process 🤔

[Souvikns]

@jonaslagoni I don't have much experience with template development, but I guess It would be tough to progressively update the code according to spec changes when other developers are updating files that are being generated by the template.

I was thinking something though don't know if this approach is what we need here Right now handlers folder has business logic and it is intended to be updated by the user, but the generator updates that file also while bootstrapping. What if we take a somewhat same approach from next.js (how they make use of getStaticProps )and separate the logic that the user will write. We create some ground rules about the filename and the function name that they can export.

[magicmatatjahu]

Good idea with these hashes with previous states, but we must admit that this approach can be very difficult to write and hard to maintance (as @jonaslagoni worries about). As long-term feature it will be awesome, but at the moment we can only add some regex (as parameter) to exclude files.

What I have in mind when I say difficult to write? For example, someone can use channels in few files, we must know about that, but in which way? Parse all the content of file to AST, and by this check usage of .channels(), .channel() function from parser-js? Operating on AST is very difficult and error-prone -> maintenance problem. Someone may additionally retrieve data from parse schema in this way:

const I_DONT_KNOW_WHAT_I_AM_DOING_BUT_I_WANT_TO_BROKE_MY_TEMPLATE = asyncapi;

I_DONT_KNOW_WHAT_I_AM_DOING_BUT_I_WANT_TO_BROKE_MY_TEMPLATE.channel();

🤣 So we must also check "new" reference to asyncapi (parsed spec) object in template.

@Souvikns So another good idea, but I think that developers don't want to write separate functions (with appropriate function name) to render simple template. We should have as the best DX for developers (without any boilerplate, however small it may be). If I understood you wrong, please correct me 😅

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had recent activity :sleeping: It will be closed in 60 days if no further activity occurs. To unstale this issue, add a comment with detailed explanation. Thank you for your contributions :heart:

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had recent activity :sleeping: It will be closed in 60 days if no further activity occurs. To unstale this issue, add a comment with detailed explanation. Thank you for your contributions :heart:

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had recent activity :sleeping:

It will be closed in 120 days if no further activity occurs. To unstale this issue, add a comment with a detailed explanation.

There can be many reasons why some specific issue has no activity. The most probable cause is lack of time, not lack of interest. AsyncAPI Initiative is a Linux Foundation project not owned by a single for-profit company. It is a community-driven initiative ruled under open governance model.

Let us figure out together how to push this issue forward. Connect with us through one of many communication channels we established here.

Thank you for your patience :heart:

[magicmatatjahu]

I think that it's still relevant :) cc @derberg

[jonaslagoni]

I re-read your blog post @derberg and one thought that came to mind was, why even generate code that mixes your own implementation alongside the generated code 🤔 In my experience, I have never understood why you want to generate code alongside custom code. I have always seen it as a huge problem when you evolve your API when you mix the two (especially as we have seen with OpenAPI HTTP server code). So the opposite thought could be that you simply should not? 🤔

For example for the WebSocket server, would it not make more sense to create an interface for the generated code that you can integrate into your own application? I.e. instead of having scaffolding routes generated src/api/services/chat.js, the generated code should instead give you an interface for you to interact with. This way you will always be able to overwrite the old generated code and evolve your API without worrying about "maybe" you overwrite your code.

Example change (hope it makes sense):

const subChatMessage = undefined;
const pubChatMessage = undefined;
router.ws('/chat', async (ws, req) => {
  const path = pathParser(req.path);
  console.log(`${yellow(path)} client connected.`);
  subChatMessage(ws);
  ws.on('message', async (msg) => {
    console.log(`${yellow(path)} message was received:`);
    console.log(util.inspect(msg, { depth: null, colors: true }));
    await pubChatMessage(ws, { message: msg, path, query: req.query });
  });
});
export function setSubChatMessage(subChatMessage){
  subChatMessage = subChatMessage;
}
export function setPubChatMessage(pubChatMessage){
  pubChatMessage = pubChatMessage;
}

In this generated code you simply enable the user to interact with the generated code, and it is also here you can make sure that everything is in according to the AsyncAPI document i.e. set message types, etc.

In terms of supporting the proposed feature or a version thereof, maybe it makes sense to highlight you should try to never get yourself into such a situation, i.e. always keep generated code separated from business logic? 🤔

I am not sure if there ever is a time where such an approach simply does not make sense?

[derberg]

but I still need to "connect the dots" somehow, right? or? Like I need to require/import interface of some given name from a given path, that is custom depending on the AsyncAPI file?

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had recent activity :sleeping:

It will be closed in 120 days if no further activity occurs. To unstale this issue, add a comment with a detailed explanation.

There can be many reasons why some specific issue has no activity. The most probable cause is lack of time, not lack of interest. AsyncAPI Initiative is a Linux Foundation project not owned by a single for-profit company. It is a community-driven initiative ruled under open governance model.

Let us figure out together how to push this issue forward. Connect with us through one of many communication channels we established here.

Thank you for your patience :heart:

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had recent activity :sleeping:

It will be closed in 120 days if no further activity occurs. To unstale this issue, add a comment with a detailed explanation.

There can be many reasons why some specific issue has no activity. The most probable cause is lack of time, not lack of interest. AsyncAPI Initiative is a Linux Foundation project not owned by a single for-profit company. It is a community-driven initiative ruled under open governance model.

Let us figure out together how to push this issue forward. Connect with us through one of many communication channels we established here.

Thank you for your patience :heart:

[derberg]

@jonaslagoni you can do this but not in all cases, simple example: package.json. After generation, I want to add another dependency there. Regeneration will remove that.

[Mettbrot]

I was surprised to see that the generator creates a complete project including build files and not just stubs/interfaces. To me, the purpose of a schema document like asyncapi should foremost be to unify the interfaces and contracts between the clients/servers using the API. The implementation/project structure should not be of concern for the generator.

Take a look at the openAPI generators. They are easy to implement into every existing project.

[derberg]

@Mettbrot for interfaces, models, classes you have https://github.com/asyncapi/modelina project

generator is aimed for something more, for full clients or apps generation, like for example you can bootstrap a microservice that gives you out of the box handlers, routers and other useful features to kick off prototyping faster

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had recent activity :sleeping:

It will be closed in 120 days if no further activity occurs. To unstale this issue, add a comment with a detailed explanation.

There can be many reasons why some specific issue has no activity. The most probable cause is lack of time, not lack of interest. AsyncAPI Initiative is a Linux Foundation project not owned by a single for-profit company. It is a community-driven initiative ruled under open governance model.

Let us figure out together how to push this issue forward. Connect with us through one of many communication channels we established here.

Thank you for your patience :heart:

[derberg]

we basically need analogical functionality to existing noOverwriteGlobs option that works in the way that you say what paths should be ignored during generation. We need an option generateOnly and also support globs there, so I can say generate *.json only or something like this

[lmgyuan]

Could I be assigned this issue? Since I implemented the noOverwriteGlobs, I should be able to do this : )