Azure Function not working with ChromaDB

[AaronWard]

Was unable to deploy the FastAPI to the function when installing Chromadb - two initial problems:

• venv was not consistent across local and remote evironments
• Chroma db code syntax error, that wasn't being logged during remote build.

---

Dumping all the links i focused on to troubleshoot:

• https://stackoverflow.com/questions/72554813/unable-to-deploy-basic-example-httprequest1-to-azure-function-using-vscode-pytho
• https://learn.microsoft.com/en-us/answers/questions/1163526/how-to-add-python-package-to-azure-function
• https://learn.microsoft.com/en-us/azure/azure-functions/bring-dependency-to-functions?pivots=programming-language-python
• https://github.com/Azure-Samples/fastapi-on-azure-functions/tree/main
• https://github.com/Azure/azure-functions-python-worker/issues/1338
• https://docs.trychroma.com/migration

Useful commands:

• python3.10 -m venv .venv
• source .venv/bin/activate
• func new --name awazfunctionapp-dev --template "HTTP trigger" --authlevel "anonymous"
• func azure functionapp publish awazfunctionapp --build remote
• uvicorn ApiFunction:app --reload
• az functionapp deployment source sync --name awazfunctionapp --resource-group awazfunctionapp

[AaronWard]

az functionapp config appsettings set --name awazfunctionapp --resource-group awazfunctionapp --settings SCM_DO_BUILD_DURING_DEPLOYMENT=false ENABLE_ORYX_BUILD=true

[AaronWard]

export PATH=/Applications/Docker.app/Contents/Resources/bin:$PATH

[AaronWard]

The "AlwaysOn" setting is not supported in Consumption Plan. i want this to be basically free to implement so i can't go with the docker option it seems. I will need to find an alternative to querying chroma db that doesn't use sqlite3. User a docker container would get around the dependency issues, but would require a premium plan because the the container can't be "cold started" like a regular function can.

[AaronWard]

https://gist.github.com/defulmere/8b9695e415a44271061cc8e272f3c300 https://docs.trychroma.com/troubleshooting

[AaronWard]

FUNCTIONS_WORKER_RUNTIME to from ~3.10 to python.

[AaronWard]

The dependencies require torch to run, which is pretty big - causing a OOM issue on the free-to-use runner. IE: Need to pay for basic plan 🤬

• https://azure.microsoft.com/en-gb/pricing/details/app-service/windows/
• https://learn.microsoft.com/en-us/azure/azure-monitor/autoscale/autoscale-get-started
• https://stackoverflow.com/questions/63957535/python-azure-function-exits-with-code-137

But the API works now at least

[AaronWard]

20:04:57 ****: cp: failed to close '/home/data/SitePackages/20231222200237.zip': No space left on device
20:04:57 ****: cp: failed to close '/home/data/SitePackages/20231222200237.zip': No space left on device\n/opt/Kudu/Scripts/starter.sh cp /tmp/build/expressbuild/20231222200237.zip /home/data/SitePackages/20231222200237.zip

went into the ssh tab and and rm -rf'd the site packages folder

[AaronWard]

note to self, the sentence-transformers/all-mpnet-base-v2 embeddings are too big to zip into a function app, therefore i went back to using sentence-transformers/all-MiniLM-L6-v2

[AaronWard]

Appears that sometimes the API is working and sometimes it fails to return a response. Testing with thunder client on vs code, i got a 504.0 GatewayTimeout when making this request: https://awazfunctionapp2.azurewebsites.net/query/What%20is%20autogen%?

Here are some relevant links:

• https://stackoverflow.com/questions/76852418/deployment-of-fastapi-python-azure-function-successful-but-triggers-not-showing
• https://stackoverflow.com/questions/76842742/trouble-deploying-fastapi-project-on-azure-functions-or-testing-locally

[AaronWard]

To read:

• https://learn.microsoft.com/en-us/azure/azure-functions/functions-versions?tabs=isolated-process%2Cv4&pivots=programming-language-javascript#breaking-changes-between-2x-and-3x
• https://learn.microsoft.com/en-us/azure/azure-functions/performance-reliability#make-sure-background-tasks-complete