inngest / inngest-py

Python SDK for Inngest: Durable functions and workflows in Python, hosted anywhere
https://www.inngest.com/docs/reference/python
Apache License 2.0
55 stars 8 forks source link

Breaking concurrency limits OR step/function confusion #179

Open 01programs opened 1 month ago

01programs commented 1 month ago

I was following the "Step parallelism in Python" in this doc and debugged for hours to find out why function that has a concurrency limit of 2 has 8+ instances running at a time. Turn out I made two mistakes in one place. I called a function as a step and I wrote "step.invoke" instead of "step.run". This should either not be possible or respect the function parameters, no?

offending piece of code:


    async def enqueue_jobs(emails, parent_job):
        jobs: list[function] = []
        for email, domain in emails:
            logging.debug(f"Email: {email}")
            jobs.append(
                lambda domain=domain: step.invoke("process_csv_row",
                                                  function=process_csv_row,
                                                  data={
                                                      "domain": domain,
                                                      "parent_job": parent_job
                                                  }))
        await step.parallel(jobs)
amh4r commented 4 weeks ago

Could you reply with both full Inngest functions? The problem is a little unclear with this code snippet