Open techsavvyash opened 10 months ago
Hii @techsavvyash @RyanWalker277 I would like to give an attempt to this issue under your mentorship!! I have a few things to ask:
Thanks
Hi @amansgith, Thanks for the detailed query, i'll try to provide some clarity on your queries:
common
folder inside the repository which hosts the pre-defined components that we publish as a part of the @samagra-x/stencil
package to enable the current functionalities that stencil supports, other than that we have the examples
folder which demonstrates via working examples how to setup a particular feature that stencil provides.Hope this helps provide clarity :)
Thanks @techsavvyash this clears a lot!! i will proceed with it and will get back if anythings bugs me..
Hey @amansgith , How is it going?
Hi @techsavvyash I am currently learning about How to use queues.. it's my first time picking up such an issue where the software itself works as a boilerplate for other applications So it's going a little hard for me.. I Request if anyone else wants go ahead with the issue.. they are welcome!! Sorry for no progress.
Hey @amansgith, Thanks for taking interest, and we do realise it might be something new for you. Please let me know in case you need some support.
Hey @amansgith, Thanks for taking interest, and we do realise it might be something new for you. Please let me know in case you need some support.
Sure thing!! Thank you so much
@techsavvyash I understand the problem, and I believe I have the necessary skills. Could I have the opportunity to work on this?
@Yash-Sajwan24 please go right ahead!
Hello ,@techsavvyash I am working on this issue and I have a few queries. 1- I am thinking about setting the BullMq service inside the common/servics and making it a global module so that we can inject it as a dependency whrever required. 2- Additionally I was thinking about implementing the job queues for the file upload service to manage file upload and fallback 3- Can you give me some information about the environment variables, like the redis connection url etc. Thank you.
I'll also give it a try
@techsavvyash I tried intergrating bull in 04-monitoring sample with the help of a youtube tutorial to understand it and it works fine with the sample. I have attached a implementation video below:
https://github.com/SamagraX-Stencil/stencil/assets/91362589/62f8e08f-698e-4f25-9859-4dc3c32980ca
Bull needs to be installed in the specific component that you will create to expose the interface to setup the tooling around bull, these components might be a controller, service, interceptor, module etc. which we might find required to help bootstrap bull faster in our stencil project.
So what I understand is we need to implement bull for these components such as a controller, service, interceptor, module etc. Example: Let's say under controller we need to implement Bull for processing uploaded files asynchronously... https://github.com/SamagraX-Stencil/stencil/blob/main/packages/common/src/controllers/file-upload.controller.ts#L20
@techsavvyash I tried intergrating bull in 04-monitoring sample with the help of a youtube tutorial to understand it and it works fine with the sample. I have attached a implementation video below:
(The implementation just shows transcoding a message (or a audio file) with the help of queue)
bull-04monitoring.mp4
Bull needs to be installed in the specific component that you will create to expose the interface to setup the tooling around bull, these components might be a controller, service, interceptor, module etc. which we might find required to help bootstrap bull faster in our stencil project.
So what I understand is we need to implement bull for these components such as a controller, service, interceptor, module etc. Example: Let's say under controller we need to implement Bull for processing uploaded files asynchronously... https://github.com/SamagraX-Stencil/stencil/blob/main/packages/common/src/controllers/file-upload.controller.ts#L20
I am having issues running the instance locally --> Error: Cannot find module 'C:\Users\saksh\Videos\stencilNew\stencil\stencil\packages\common\dist\main' at Function.Module._resolveFilename (node:internal/modules/cjs/loader:1149:15) at Function.Module._load (node:internal/modules/cjs/loader:990:27) at Function.executeUserEntryPoint [as runMain] (node:internal/modules/run_main:142:12) at node:internal/main/run_main_module:28:49 from where did you spin up the server ?
Trying running it on gitpod
Trying running it on gitpod 👍 Thanks
@Savio629 the video looks good. You are on the right track, what we need is to add the components (modules, services, etc.) required to setup the bull in a Nest/Stencil app, as first class citizens in the stencil packages. Take for example how we have added the Monitoring module, Temporal module etc.
So what I understood is that I need to add job queues using bull for handling async tasks over this modules (Monitoring, file uploads,etc ), right? Or create a separate module(is this what you meant by first class citizen?) for bull for handling job queues and then integrating them inside/over the specified module? @techsavvyash
Or create a separate module(is this what you meant by first class citizen?) for bull for handling job queues and then integrating them inside/over the specified module?
this is the correct assumption @Savio629. We need to be able to setup bull in any stencil app and then bootstrap it using interceptors or whatever other component we require to any api endpoints the user might want to.
@techsavvyash Right now I was trying it with file-upload by updating the packages of file-upload Ref: https://medium.com/@raviyasas/how-to-use-bull-with-nestjs-18697f123e3c Am I on right track?
What exactly are you trying to do by manipulating the file-upload
package?
@techsavvyash Is this issue opened??
Yes @ishqDehlvi
Hello @techsavvyash, Is this issue still open? If so, could you please provide more details about it so that I can start working on it?
Please assign this issue to me under C4GT
Assigning to @Soham-27 @vishalmaurya850 You can also start working collaboratively on the issue
Hey @Soham-27 @vishalmaurya850 Thanks for showing interest in the issue, please connect with @Savio629 to get more context around the issue, but an assignment would only once you have raised a draft PR for the issue.
Hey
Hello @MohitNSamagra,
Are you involved in the Stencil project?
Hi
@techsavvyash Please assign me this issue
Interesting ticket.
Add Queues with Bull to Stencil NestJS Project
Description
The task involves integrating Bull, a JavaScript library for managing job queues in Node.js, into Stencil. This addition will enable the project to efficiently handle and manage asynchronous tasks using job queues.
Goals
Bull Integration: Successfully integrate and equip Stencil with Bull.
Job Queue Implementation: Implement job queues using Bull for at least two distinct asynchronous tasks within the Stencil.
Concurrency Control: Configure and optimize Bull job queues for efficient concurrency control, balancing workload and resource utilization.
Acceptance Criteria
Bull Setup: Install and configure the Bull library in Stencil.
Job Queue Implementation: Create and integrate at least three different job queues for distinct asynchronous tasks in the project.
Concurrency Control Testing: Perform testing to ensure that concurrency control is effectively managed, preventing issues related to job processing overload or resource exhaustion.
Documentation Update: Provide comprehensive documentation on your work. Include it in the form of a READMe.md in the specific module you create. It will be added to the official Stencil Docs.
Checklist (To be filled by the developer)
Product Name
Stencil
Organisation Name
SamagraX
Tech Skills Needed
TypeScript, APIs, Queues, Bull
Mentors
@RyanWalker277
Complexity
Hard
Domain
Category
Backend
Sub Category
APIs, Queues, Bull