search

snexus / llm-search

Querying local documents, powered by LLM
MIT License
421 stars 51 forks source link

docker #75

Open MyraBaba opened 6 months ago

MyraBaba commented 6 months ago

Hi ,

Is there any docker container or any plan in near future ?

Best

snexus commented 6 months ago

Hi,

I might add support for docker in the future if there is enough demand

MyraBaba commented 6 months ago

There is a visible demand for enterprise search and llm mix.

Also one of our firm is working on the cyber security and wondering that is llm-search could be useful for logs (firewall, servers etc)

Best

On 1 Jan 2024, at 16:42, Denis Lapchev @.***> wrote:

Hi,

I might add support for docker in the future if there is enough demand

— Reply to this email directly, view it on GitHub https://github.com/snexus/llm-search/issues/75#issuecomment-1873333494, or unsubscribe https://github.com/notifications/unsubscribe-auth/AEFRZH4JN5CJK7TVUR2KDHTYMK4LTAVCNFSM6AAAAABBHPZ4P2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQNZTGMZTGNBZGQ. You are receiving this because you authored the thread.

shuther commented 5 months ago

in the docker (there is a DOCKERFILE), is it possible to install pandoc?

snexus commented 5 months ago

It is an outdated version and not in sync with the latest libraries. Will add pandoc once I update it.

shuther commented 5 months ago

thanks, indeed, I had an issue to load it. Also, is it possible to remove the dependency on cuda? I expect the embeddings and llm to run on a different machine (openAI or through litellm)

snexus commented 5 months ago

CUDA is also required for sparse embeddings (SPLADE in this project), which are not optional at the moment. Theoretically, they can run on CPU but would have a large performance hit. Are there any sparse embeddings that can be served via lite llm?