metric-space-ai / octopus_server

The backend of octopus software
Other
4 stars 0 forks source link

ai service changes #482

Open asyncfncom opened 2 weeks ago

asyncfncom commented 2 weeks ago

We need a more reliable way to parse and deploy Python code to AI services.Statista wants to try to install own services. I said, maybe they wait until we optmize the plugin installation a bit.I think we should do this step by step and craft prompts for each of the step and chain them in a good way. When installing and deploying, we can then inform the user about the respective status in a progress bar and corresponding logs. We can then go and optimize each step, with for example web-search for trouble shouting.here is my idea of steps.

Parse dependencies
Install dependencies check.
Substitute API calls with corresponding backend versions. (openAI, ollama, etc) and replace tokens
Generate script version.
Run, test and iterate the script version.
Write setup function.
Correct GPU and CPU usage.
Write main functions
Write test function
Add function wrappers for flask endpoints and define inputs and outputs formats.
Run setup and test.
Deploy the service.