inngest/inngest - Githubissues

Inngest's durable functions replace queues, state management, and scheduling to enable any developer to write reliable step functions faster without touching infrastructure.

Write durable functions using any of our language SDKs
Run the Inngest Dev Server for a complete local development experience, with production parity.
Deploy your functions to your own infrastructure
Sync your application's functions with the Inngest Platform or a self-hosted Inngest server.
Inngest invokes your functions securely via HTTPS whenever triggering events are received.

An example durable function

Inngest Functions enable developers to run reliable background logic, from background jobs to complex workflows. An Inngest Function is composed of three key parts that provide robust support for retrying, scheduling, and coordinating complex sequences of operations:

Triggers - Events, Cron schedules or webhook events that trigger the function.
Flow Control - Configure how the function runs are enqueued and executed including concurrency, throttling, debouncing, rate limiting, and prioritization.
Steps - Steps are fundamental building blocks of Inngest, turning your Inngest Functions into reliable workflows that can runs for months and recover from failures.

Here is an example function that limits concurrency for each unique user id and performs two steps that will be retried on error:

export default inngest.createFunction(
  {
    id: "import-product-images",
    concurrency: {
      key: "event.data.userId",
      limit: 10
    }
  },
  { event: "shop/product.imported" },
  async ({ event, step }) => {
    // Here goes the business logic
    // By wrapping code in steps, each will be retried automatically on failure
    const s3Urls = await step.run("copy-images-to-s3", async () => {
      return copyAllImagesToS3(event.data.imageURLs);
    });
    // You can include numerous steps in your function
    await step.run("resize-images", async () => {
      await resizer.bulk({ urls: s3Urls, quality: 0.9, maxWidth: 1024 });
    })
  };
);

// Elsewhere in your code (e.g. in your API endpoint):
await inngest.send({
  name: "shop/product.imported",
  data: {
    userId: "01J8G44701QYGE0DH65PZM8DPM",
    imageURLs: [
      "https://useruploads.acme.com/q2345678/1094.jpg",
      "https://useruploads.acme.com/q2345678/1095.jpg"
    ],
  },
});

Getting started

Run the Inngest Dev Server using our CLI:

npx inngest-cli@latest dev

Open the Inngest Dev Server dashboard at http://localhost:8288:

Screenshot of the Inngest dashboard served by the Inngest Dev Server

Follow our Next.js, Node.js or Python quick start guides.

SDKs

TypeScript / JavaScript (inngest-js) - Reference
Python (inngest-py) - Reference
Go (inngestgo) - Reference
Kotlin / Java (inngest-kt)

Project Architecture

To understand how self-hosting works, it's valuable to understand the architecture and system components at a high level. We'll take a look at a simplified architecture diagram and walk through the system.

System Architecture

Event API - Receives events from SDKs via HTTP requests. Authenticates client requests via Event Keys. The Event API publishes event payloads to an internal event stream.
Event stream - Acts as buffer between the Event API and the Runner.
Runner - Consumes incoming events and performs several actions:
- Scheduling of new “function runs” (aka jobs) given the event type, creating initial run state in the State store database. Runs are added to queues given the function's flow control configuration.
- Resume functions paused via waitForEvent with matching expressions.
- Cancels running functions with matching cancelOn expressions
- Writes ingested events to a database for historical record and future replay.
Queue - A multi-tenant aware, multi-tier queue designed for fairness and various flow control methods (concurrency, throttling, prioritization, debouncing, rate limiting) and batching.
Executor - Responsible for executing functions, from initial execution, step execution, writing incremental function run state to the State store, and retries after failures.
State store (database) - Persists data for pending and ongoing function runs. Data includes initial triggering event(s), step output and step errors.
Database - Persists system data and history including Apps, Functions, Events, Function run results.
API - GraphQL and REST APIs for programmatic access and management of system resources.
Dashboard UI - The UI to manage apps, functions and view function run history.

Community

Join our Discord community for support, to give us feedback, or chat with us.
Post a question or idea to our GitHub discussion board
Read the documentation
Explore our public roadmap
Follow us on Twitter
Join our mailing list for release notes and project updates

Contributing

We embrace contributions in many forms, including documentation, typos, bug reports or fixes. Check out our contributing guide to get started. Each of our open source SDKs are open to contributions as well.

Additionally, Inngest's website documentation is available for contribution in the inngest/website repo.

Self-hosting

Self-hosting the Inngest server is possible and easy to get started with. Learn more about self-hosting Inngest in our docs guide.

License

The Inngest server and CLI are available under the Server Side Public License and delayed open source publication (DOSP) under Apache 2.0. View the license here.

All Inngest SDKs are all available under the Apache 2.0 license.

inngest / inngest

readme