Feature/use unstructured local api - Githubissues

i-dot-ai / redbox

Bringing Generative AI to the way the Civil Service works

https://i-dot-ai.github.io/redbox/

MIT License

85 stars 28 forks source link

Feature/use unstructured local api #917

Closed gecBurton closed 1 month ago

gecBurton commented 1 month ago

Context

As an Engineer I want to use the official Unstructured image instead of our own code so that:

our docker images are smaller
we are writing the least code ourselves

Changes proposed in this pull request

using unstructured api image
worker calls unstructured image via http

out of scope:

UnstructuredLargeChunkLoader & UnstructuredTitleLoader are nearly identical and should be merged

Guidance to review

Relevant links

build: https://github.com/i-dot-ai/redbox/actions/runs/10265038401

Things to check

[ ] I have added any new ENV vars in all deployed environments
[x] I have tested any code added or changed
[x] I have run integration tests https://github.com/i-dot-ai/redbox/actions/runs/10262498172