Nitro Tasks API

[pi0]

Nitro tasks allow on-off operations in runtime.

Docs: https://nitro.unjs.io/guide/tasks

### Tasks
- [x] Initial Implementation (#1929)
- [x] Write documentation section (PR welcome!)
- [x] Stabilize API / return values
- [x] Integrate with Cron Tasks / Background Jobs
- [ ] Ability to run tasks using CLI against production build
- [ ] Task hooks (on dev start, production start and production build)
- [ ] Ability to schedule one time run of a task for the future
- [ ] Allow optionally allow running multiple instances of task
- [ ] https://github.com/unjs/nitro/issues/2633

### Tasks

[pi0]

@imcm7 check out task list it is on the roadmap :) Nitro tasks API will have multiple use-cases and invocation possibilities.

[pi0]

@imcm7 Would you mind to move your question to a discussion or feel free to DM me in discord (pi0) i would be happy to answer 🙏🏼

But in short, we are introducing tasks API which as you could guess for the next step of supporting standard DB layer in Nitro and allow running migrations within the runtime without this issues as most of ORMs suffer from it.

[Aareksio]

Tasks should probably be async, currently the run method expects synchronous code and is not awaited.

[manniL]

@Aareksio check https://github.com/unjs/nitro/issues/2021 ☺️

[Rigo-m]

If I use a nitro task in, let's say, Netlify will it leverage netlify background functions?

[cosbgn]

I'm also interested on this. I would like to have for example my normal code on vercel-edge but run tasks on vercel lambda (5min timeout vs 25 seconds)

[imcm7]

How to disable it, because i use dir tasks for other stuff?

[brendonmatos]

It is planned to support running these tasks from cli without running the server?

[pi0]

Nitro already has a CLI preset too! However it is tricky to tell now when we can do support it this way now.

[robin-dongbin]

Any plan for support seconds scheduling?

[pi0]

@robin-dongbin You mean this https://nitro.unjs.io/guide/tasks#scheduled-tasks or something different? (like a pattern for <1 minute schedule?)

[robin-dongbin]

@robin-dongbin You mean(意味着) this nitro.unjs.io/guide/tasks#scheduled-tasks or something different? (like a pattern(模式) for <1 minute(分钟) schedule?)

Yes, that's what I want. Is it possible? I know that cron do not support that, but Laravel has support this last year.

[go4cas]

@pi0 ... I have just tested the experimental scheduled task feature. It works really well! Thanks!

Just one question: would it be possible to have a config setting to execute a scheduled task immediately after the cron is scheduled? Example: a task running every 10 minutes: executes once the server starts, and then every 10 minutes after that? I saw in the croner docs that there is a trigger method, but not sure how it would fit into the nitro tasks?

[pi0]

@go4cas Maybe use a Nitro plugin to triger them on startup? (also init: / startup: tasks are planned to be supported to have similar effect in the future). Also you can execute a task from another.

@robin-dongbin I also wonder if you need to trigger something this frequently, maybe a setInterval inside nitro plugins to run task with runTask might be simpler method with giving full control on executation?

[JCtapuk]

@go4cas Maybe use a Nitro plugin to triger them on startup? (also init: / startup: tasks are planned to be supported to have similar effect in the future). Also you can execute a task from another.

@robin-dongbin I also wonder if you need to trigger something this frequently, maybe a setInterval inside nitro plugins to run task with runTask might be simpler method with giving full control on executation?

But won’t there be conflicts if a node in a cluster with 8 processes and all 8 processes will cause a task, but this behavior should not happen? It turns out you need to find someone who has a free worker and give him a task from the master?

[pi0]

For time being, scheduled tasks are not supported in cluster mode the exactly meet this requirement (share a safe lock among them) and it is doable.

Regardless inside your interval you can call runTask() in single node, it currently guarantees only it is running once and when we support cluster, it would be safe too.

[brendonmatos]

I apologize if this isn't the correct forum. But how is supposed to run those tasks in Nuxt project from CLI? Nuxt CLI will be implemented to invoke those tasks from the URL or we should run directly from nitro cli?

[pi0]

Yes.

[passionate-bram]

With async tasks, and enforcing task payloads to be serializable early on it will be possible in the future to dispatch tasks to queues. Shamelessly taking the concept from Laravel: https://laravel.com/docs/10.x/queues

The nice thing about that way of working is that when queues are configurable (e.g. redis, some AWS service, etc.) then the dispatch mechanic becomes really usefull for dispatching work to different parts of your architecture.

[pi0]

payload should be serializable indeed in preparation for future ideas like queues. and it is by design separated from context that is non seriazable for other stuff. PR welcome to clarify in the docs.

[GerryWilko]

@pi0 This tasks API looks fantastic thank you so much for adding this to Nitro.

Anyone who is trying to get early access to this feature and using a workspace with multiple nuxt projects may run into issues trying to get nuxt to pickup the correct nitro-imports. In my case I managed to force it to work with an overrides in the project package.json:

  "pnpm": {
    "overrides": {
      "nitropack": "npm:nitropack-nightly@latest"
    }
  }

Thought I would just post this trips anyone else up 😄

[TMBL-DEV]

Nuxt issue

My dev server (yarn dev) is running but when i do yarn run nitro task list

then it results in Missing info file: /home/timmy/projects/private/rewrite/nuxt-project/.nitro/nitro.json (is dev server running?)

what bothers me is that it's looking in .nitro folder whilst nuxt generates a .nuxt/nitro.json. So it seems to me that it is looking in the wrong folder.

Or do i need to run yarn run nitro dev for the dev server?

does anyone have a fix?

update

i fixed it by symlinking .nuxt to .nitro now i can run my nuxt server and run commands in another terminal. The only thing that i need to figure out is how to get cloudflare d1 working in the tasks context

[ProgramRipper]

@robin-dongbin You mean(意味着) this nitro.unjs.io/guide/tasks#scheduled-tasks or something different? (like a pattern(模式) for <1 minute(分钟) schedule?)

Yes, that's what I want. Is it possible? I know that cron do not support that, but Laravel has support this last year.

@robin-dongbin Yes, it is supported. You can use cron expression like */15 * * * * * (six *, different from five * of classic cron), the first * means seconds.

[robin-dongbin]

@robin-dongbin You mean(意味着) this nitro.unjs.io/guide/tasks#scheduled-tasks or something different? (like a pattern(模式) for <1 minute(分钟) schedule?)

Yes, that's what I want. Is it possible? I know that cron do not support that, but Laravel has support this last year.

@robin-dongbin Yes, it is supported. You can use cron expression like */15 * * * * * (six *, different from five * of classic cron), the first * means seconds.

That's great, thank you!

[flapili]

Hi, do you think in the future this feature could allow us to define scheduled function on gcp as example ?

[pi0]

@flapili Sure 👍🏼 if you can perhaps share an example project and some links to the docs i can help to do it faster :) (we might need a new preset that auto configured for gcp)

[flapili]

@flapili Sure 👍🏼 if you can perhaps share an example project and some links to the docs i can help to do it faster :) (we might need a new preset that auto configured for gcp)

the usecase we encounter at work is about run every day a routine to check if point of sales are closed, if not send an email, after 6 day force close the POS (constraint from France law)

here an example from google directly : https://github.com/firebase/functions-samples/blob/a8b1ab89c503419fab90b0a44b61adee8ec9923b/Node/delete-unused-accounts-cron/functions/index.js#L37

[pi0]

I see so we can probably integrate it firebase preset.

[Eckhardt-D]

Really cool feature! Works like a charm in VPS environments for scheduledTasks. I have 2 questions:

If for example, the nitro app is running in a cluster (e.g. PM2). I am assuming the scheduled tasks will trigger in each instance on the cron schedule? What would you suggest here?

Will scheduled tasks hook into platform API's for jobs, e.g. Firebase has scheduled functions and Vercel has Cron Jobs? Both which accept the cron expressions.

[Niki2k1]

@Eckhardt-D pi0 wrote about cluster mode in this comment: https://github.com/unjs/nitro/issues/1974#issuecomment-1967387326

[ivanjeremic]

I don't see in the experimental docs a way to programmatically add/remove tasks, for example, add a task programmatically that executes at a specific time, is that doable?

[pi0]

By design, no it is not doable to define or modify the tasks in runtime. The reason is that we need deterministic behavior during build to integrate with platform primitives.

We can open a way to execute (already defined) tasks btw in runtime, if that's what you might want like runTask('foobar', { in: 5 * 60 /* 5 minutes */ })

@ivanjeremic Can you explain more of the use cases that might require a behavior for add/remove if that was your question?

[ivanjeremic]

@pi0 A use case would be a platform that allows users to add / remove services that in the background add / remove tasks. I have not really good example in mind but maybe something like Hootsuite that allows its users to schedule posts for a specific time for a social media platform.

[manniL]

@ivanjeremic couldn't you run a job every X minutes that checks if some posts are queued up, then processing them if so? 🤔

[ivanjeremic]

No. The Hootsuite/Posts example was just an example there are thousends other use cases.

[ivanjeremic]

@manniL I wanted to say I'm new to tasks/cron and maybe the way I think about this problem is wrong, I think you are right and the way you described it is a simpler way of doing what I want.

[pi0]

Thanks for explaining your idea @ivanjeremic. I think in the future something like this would unblock this and i have added two tasks to tracker.

runTask('useTask', {
  scheduled: 5 * 60, /* 5 minutes */
  multiple: true,
  payload: {
    userId: 123
  }
})

(above snippet is just an example to show the idea)

[ivanjeremic]

Thanks for explaining your idea @ivanjeremic. I think in the future something like this would unblock this and i have added two tasks to tracker.

runTask('useTask', {

  scheduled: 5 * 60, /* 5 minutes */

  multiple: true,

  payload: {

    userId: 123

  }

})

(above snippet is just an example to show the idea)

Exactly how I imagined it👍🏻

[iBobik]

It would be handy to disable cron tasks in dev server by default, but keep ability to run it manually.

It is annoying when I need to debug something else and see log full of unrelated periodic tasks. It also unnecessary eats battery of laptop.

[casualmatt]

It's very nice how the tasks show up in nuxt devtools; I tried to use h3utility to send a stream back but didn't succeed because they need the event; this could be an excellent tool to keep track of the task activity.

[malgamves]

New to this so curious if it normal that when i setup a task in Nuxt to return something and use the cron scheduler in my Nuxt config, my tasks run but i don't get the return object i'm expecting.

When i run the task programmatically, say through an api route in Nuxt, i get the return object.

Edit.

Turns out i was printing out the return object programatically. It would be nice to have status modes logged (success, failure) - this would tie in really well with the queues idea. Might be interesting use cases around error handing for batching and other operations.

[jsonleex]

Thanks for the awesome feature! 🫰

Is there any way to create a new task within a task, or can tasks be created during runtime?

[AwesomePeregrineFalcon]

Do you want to add the ability to run tasks from CLI? Like

node .output/server/index.mjs db:migrate

or

node .output/server/index.mjs task run db:migrate

I really liked Nitro but this feature is kinda deal-breaker for me. As I understand, there is no way right now to make such commands because Nitro makes its own entry-point to the app, and I can't add operations before it like I would be able if I used something like express.js, so I can't add CLI

[pi0]

@AwesomePeregrineFalcon it is in the plan yes to allow CLI interaction with production build. In the meantime you can expose a (protected) API endpoint.

[juretopolak]

The only thing that i need to figure out is how to get cloudflare d1 working in the tasks context

@TMBL-DEV have you figured this out? 🙂

[AwesomePeregrineFalcon]

it is in the plan yes to allow CLI interaction with production build. In the meantime you can expose a (protected) API endpoint.

@pi0, I figured out how to make CLI. I made the custom preset. Just used Commander.js library in server/preset/entry.ts and made the listening start in a subcommand, so I start the server with

node .output/server/index.mjs server listen

and run migrations with

node .output/server/index.mjs db migrate

But now I have to build the project to use CLI :disappointed:. Anyway, it works now... And I liked everything else in Nitro.js! Is there a reason why there is no possibility to run a custom preset in dev mode? Something like:

yarn dev server listen

[dalbodeule]

Wouldn't it be possible to bind specific tasks to the Task API? For example, it might be a good idea to apply cloudflare worker's email handler or queue handler.

While looking at related issues and PRs, I thought it was a good idea and suggested it.

[luke-z]

I just came across the Tasks API and was wondering if eventually it could replace a package like bullmq for simple jobs and a worker type of setup.

Allow optionally allow running multiple instances of task

Seems to get close to this kind of feature.

Or am I somehow misunderstanding it :)

[kyng-cytro]

Hey, does the cron Synatx have support for time zones? I know Croner does.

[provok-me]

If I use a nitro task in, let's say, Netlify will it leverage netlify background functions?

@pi0 A Nitro task has only one running instance, but Netlify background functions can handle multiple instances at the same time (as I understand it in there docs).

Is there another way to use Netlify background functions with Nitro to handle multiple instances of a background function?