Open javaniecampbell opened 10 months ago
Hi @javaniecampbell , thanks for reaching out. Set use_docker=true means running code generated by autogen in a docker container. It will pull python image by default, you can also specify image name. To sum up, itโs a safe code execution env.
As for Azure, AWS and GCP, it depends. If youโre using VMs which support docker, the experience should be the same as your local machine. If you want to deploy your application to containers in the cloud, then it looks like you donโt need to set use_docker to True.
Would be helpful if you could share more details about your use case.
Hi @thinkall thanks for responding greatly appreciated, the infrastructural use case, I have in mind is using autogen
as a microservice api that has isolation execution via the capability of using docker with I find as an idle security, yes I do agree it is absolutely safe and I would want to extend that safety to a real world in the cloud. VMs a pretty expensive for deployment I am wondering how I am capable of using in PaaS model. If docs could explore the deployment options and best practice for deployment would assist in how to go about the deployments ๐๐พ.
The next use case is possible of using autogen
in serverless framework like Azure Functions to be able to offload the work of the agents to be able to have a background work.
My practical application is for generating requirements based on series of questions & answers to determine what the user want for planning then create a preview of idea being asked to confirm accuracy of requirements.
Hi @thinkall thanks for responding greatly appreciated, the infrastructural use case, I have in mind is using
autogen
as a microservice api that has isolation execution via the capability of using docker with I find as an idle security, yes I do agree it is absolutely safe and I would want to extend that safety to a real world in the cloud. VMs a pretty expensive for deployment I am wondering how I am capable of using in PaaS model. If docs could explore the deployment options and best practice for deployment would assist in how to go about the deployments ๐๐พ.The next use case is possible of using
autogen
in serverless framework like Azure Functions to be able to offload the work of the agents to be able to have a background work.My practical application is for generating requirements based on series of questions & answers to determine what the user want for planning then create a preview of idea being asked to confirm accuracy of requirements.
@javaniecampbell so you would create an API which contain multi agents to solve a problem? Or is that kinda like a chat room still?
@javaniecampbell We have some prior experience deploying agents in Azure Functions, it is possible but there are some tricks involved to ensure fast spin-up times, making sure you are not caching results, etc. This isn't something we wanted to open-source "As-Is" without further security and verification guards but if it is a path you are looking to go down I can verify it will work for you.
For the purposes of future development/extension of the AutoGen framework, is the reason you want serverless execution to scale up/out on the problemset beyond what a single VM can handle? I'd love to learn more about why you want to head that direction.
Hi @thinkall thanks for responding greatly appreciated, the infrastructural use case, I have in mind is using
autogen
as a microservice api that has isolation execution via the capability of using docker with I find as an idle security, yes I do agree it is absolutely safe and I would want to extend that safety to a real world in the cloud. VMs a pretty expensive for deployment I am wondering how I am capable of using in PaaS model. If docs could explore the deployment options and best practice for deployment would assist in how to go about the deployments ๐๐พ.The next use case is possible of using
autogen
in serverless framework like Azure Functions to be able to offload the work of the agents to be able to have a background work.My practical application is for generating requirements based on series of questions & answers to determine what the user want for planning then create a preview of idea being asked to confirm accuracy of requirements.
@javaniecampbell so you would create an API which contain multi agents to solve a problem? Or is that kinda like a chat room still?
@wayliums yes I would want to create an API to be consumed by external systems which will either be triggered via an event from a queue or service bus to solve a problem in one of my use cases. The other is use it with botframework as the brains for solving a problem given using multi agents to with single responsibility to for workflow orchestration of sorts
@javaniecampbell We have some prior experience deploying agents in Azure Functions, it is possible but there are some tricks involved to ensure fast spin-up times, making sure you are not caching results, etc. This isn't something we wanted to open-source "As-Is" without further security and verification guards but if it is a path you are looking to go down I can verify it will work for you.
For the purposes of future development/extension of the AutoGen framework, is the reason you want serverless execution to scale up/out on the problemset beyond what a single VM can handle? I'd love to learn more about why you want to head that direction.
@stevetru1 awesome news, if you could share that knowledge, it would be great for those gotchas I love to learn how you accomplished it, if I can learn from your experience would be great.
For background, I have been working on project that generate microservices I want to extend it have the agents get 75% percent of the boilerplate code written that can modified if possible and saved to blob storage or git and do that in way that isn't too I/O intensive hence I thought a serverless might be the best way to manage that load that I could scale out/ fan out that workload especially if I have multiple requests that require several app platforms to generate it microservices needed or have them orchestrated etc amongst other aspects of the application I have mind.
I wanted to take advantage of several things that come with serverless approach:
Etc
I could go on but ideally it's the main workflow for budget friendly infrastructure for me currently, makes it easier to extend without troubling existing deployments for a event driven architecture I have and a level of flexibility to manage easier in scale up or out without too much boilerplate code to deliver value.
If this is solved I believe it can be some learnings for deploying in a kubernetes cluster as well.
@victordibia @pcdeadeasy for awareness
very intersting topic, is there any guideline to make it happen, step by step?
@HAL9KKK what do you need step wise instructions for? Can you explain what you want to deploy?
@pcdeadeasy we would step wise instruction about deploying these ai agents on cloud, and also about what are the best practices for the same?
@javaniecampbell We have some prior experience deploying agents in Azure Functions, it is possible but there are some tricks involved to ensure fast spin-up times, making sure you are not caching results, etc. This isn't something we wanted to open-source "As-Is" without further security and verification guards but if it is a path you are looking to go down I can verify it will work for you.
For the purposes of future development/extension of the AutoGen framework, is the reason you want serverless execution to scale up/out on the problemset beyond what a single VM can handle? I'd love to learn more about why you want to head that direction.
@stevetru1 I am having the hardest time getting AutoGen to run on an Azure Functions App. I keep getting an "unable to read database" error. Do you have any pointers for me on how to get this to work?
Maybe because caching? Have you tried setting cache_seed
to None to disable caching?
I have been playing with
autogen
and absolutely love it. I was ready to deploy it and then I realized that even though it is early stages, there was no guidance for deployment with consideration for theuse_docker
flag to be true or image specified in the cloud.Is it possible to provide some guidance on how to do the following: