feat: kotaemon as service

[phv2312]

Description

• Provide convinient method to call kotaemon without gradio App

Type of change

• [x] New features (non-breaking change).
• [ ] Bug fix (non-breaking change).
• [ ] Breaking change (fix or feature that would cause existing functionality not to work as expected).

Checklist

• [x] I have performed a self-review of my code.
• [ ] I have added thorough tests if it is a core feature.
• [ ] There is a reference to the original bug report and related work.
• [ ] I have commented on my code, particularly in hard-to-understand areas.
• [ ] The feature is well documented.

[phv2312]

How to use:

from kotaemon.deps.container import Container
from ktem.index.file.pipelines import DocumentRetrievalPipeline, IndexDocumentPipeline

"""
1. Indexing
"""
indexing_pipeline: IndexDocumentPipeline = Container.indexings.get("default").get()
print(indexing_pipeline)
stream = indexing_pipeline.stream(
    ["<your_filepath>"],
    reindex=True
)
for response in stream:
    print(response)

"""
2. Retriever
"""
crud = Container.crud
docids = crud.list_docids()

retriever_pipeline: DocumentRetrievalPipeline = Container.retrievers.get("default").get()
print(retriever_pipeline)

responses= retriever_pipeline.run(
    text="What is the main content?",
    doc_ids=docids
)
for response in responses:
    print(response.id_)
    print(response.text)