search

Azure / azure-functions-python-worker

Python worker for Azure Functions.
http://aka.ms/azurefunctions
MIT License
335 stars 103 forks source link

Durable Functions Support for Python #227

Closed asavaritayal closed 4 years ago

asavaritayal commented 5 years ago

New Feature - Looking for votes and/or user input to gauge traction.

mattc-eostar commented 4 years ago

@mattc-eostar not really. Depends on how you trigger your function. If you use http trigger - then you are limited by 4 minutes timeout. Function should execute and return output within this limit, otherwise it fails. And if you want to trigger your function from Data Factory (for example) - you are pretty much limited to http triggers.

Is this true even for functions hosted on premium plan? They advertise no limit on execution time.

anthonychu commented 4 years ago

HTTP triggers have a 230 second timeout for how long the request can stay open. If you need to use the max function duration (up to 10 mins consumption, 60 mins Premium, "unlimited" on other plans), the HTTP function can drop a message on a queue that a longer-running queue triggered function can process. Durable Functions does this behind the scenes, and adds the ability to checkpoint progress and aggregate results.

We're targeting early 2020 for a preview of Durable Functions for Python.

mattc-eostar commented 4 years ago

I see what you are saying. Yes. I was operating under the assumption my use case would be using queues regardless. And great to hear. Look forward to working with that.

gbonventre commented 4 years ago

I need Async API and Long-running functions. Also long-running functions that can do fan in and fan out from a queue would be perfect for many of the ETL applications my employer is currently working on.

markl11 commented 4 years ago

"We're targeting early 2020 for a preview of Durable Functions for Python." - given that quite a few folks can't put preview features into production, when are you actually thinking that Durable Functions for Python will be GA? (frustratingly, preview features in Azure can be preview for a looooong time in my experience so I hope this will be different)

cgillum commented 4 years ago

@markl11 we're targeting the first half of 2020 for the GA release - i.e. no later than June, assuming things go relatively smoothly with the preview.

WrittenLabyrinth commented 4 years ago

+1 working on a prototype app in python and a durable function in Azure would be awesome!

CalebHui commented 4 years ago

+1

anthonychu commented 4 years ago

Just a quick update as we've been hearing from a lot of folks who can't wait to try this. Python Durable Functions is currently on track for a preview in a month or two. It'll support the core 5 workflow patterns, with the exception of durable entities.

anthonychu commented 4 years ago

Reopening this as we're using this issue to track progress of Python Durable Functions support.

LouisMach commented 4 years ago

@anthonychu any updates on when the preview will be released? Maybe a tracking page?

anthonychu commented 4 years ago

Waiting for some bits to be released in Core Tools and deployed to the service in Azure. Could have something you can try running locally in the next week or two.

LouisMach commented 4 years ago

That's really cool! I'm planning to use durable functions to host a tensorflow model for computer vision, or do you think I should host it in AzureML and call it from a function instead?

MattConflitti commented 4 years ago

Try it and see! Python durable functions is going to open up a lot of doors given the orchestration capabilities. From smaller scale data pipelining to serving simpler models, it will be exciting to see how it stacks up in usability and pricing. Both of which are suspected to be great.

anthonychu commented 4 years ago

Hey everyone. Thanks for your patience. You can now try out Durable Functions in Python here: https://github.com/azure/azure-functions-durable-python 🎉

There are a few pieces to release in tooling and deploy to our service before we can announce a public preview. For now, check out the readme for the current limitations and instructions on how you can try it out today.

We look forward to getting some early feedback. Please create issues here. Thanks!

wasuaje commented 4 years ago

+1 Here for needing this !

Thanks for the preview

anthonychu commented 4 years ago

Python Durable Functions is now in public preview. 🎉 Thanks for your patience everybody. Please use https://github.com/Azure/azure-functions-durable-python for feedback.

Announcement: https://azure.microsoft.com/updates/durable-functions-now-supports-python/

StevenBorg commented 4 years ago

Congratulations!!!! This is great news. I've been forced to rewrite functions in C# just to get Durable Functions. The pain of switching from Python to C# was less than dealing with coordinating my own fan-out/fan-in. Thank you! Back to Python!

priyaananthasankar commented 4 years ago

That's really cool! I'm planning to use durable functions to host a tensorflow model for computer vision, or do you think I should host it in AzureML and call it from a function instead?

You can use AML to host or run ML batch processes and then use durable functions to orchestrate the pipeline through monitoring pattern. Check the samples in the repo for this use case :)

Hazhzeng commented 4 years ago

The Python Durable Functions is currently in Preview https://azure.microsoft.com/en-us/updates/durable-functions-now-supports-python/

Closing this thread as we're redirecting the related discussion to the new durable python repo https://github.com/Azure/azure-functions-durable-python

Thanks