Closed Shreyanand closed 1 year ago
Check out this pull request on
See visual diffs & provide feedback on Jupyter Notebooks.
Powered by ReviewNB
@codificat @suppathak Are there things that I can add to this notebook that would help with #24? If not we could start on that notebook...
I think you could probably add some more markdown cells explaining what is going on and what choices are being made among the available options. Also it would be nice to include links to relevant docs where appropriate.
Sounds good, will add that!
The evaluation approach here, using an LLM for evaluation, differs from what is presented in #16. That's fine, it is an interesting approach to evaluate, but I am wondering if it belongs there instead? Or, possibly, in the "next" notebook that you mention, #24.
Yes, the metrics notebook #16 should include a description of this as well. It's quite an interesting approach.
I mean: that part already helps with what we want for #24, but we probably don't want to elaborate much more on that here: the goal of this notebook should be to highlight what langchain offers, similar to what the hugging face and haystack notebooks did.
Agreed, so I'll split this notebook into two. The first one being a small notebook showing langchain and openai with appropriate links and markdown. The second one being a WIP notebook for #24.
That one question answer looks interesting:
Query: How can I federate metrics to a centralized Prometheus Cluster? Answer: To federate metrics to a centralized Prometheus Cluster, you can deploy a Prometheus instance and configure it to send alerts to themselves, and use Prometheus' federated metrics feature and the Prometheus Operator. SOURCES: Tommer Amber's guide: https://medium.com/@tamber/2-mini-how-to-guides-for-prometheus-on-openshift-federation-custom-infrastructure-alerting-8ec70061405d
so, it is pointing to an external source, not to any of the docs provided.
I believe it would be helpful to get some more insight into what is going on here, i.e. to be able to observe which documents have been identified as relevant during the chain, their contents (after being split), what prompts (including query and context) are being sent, etc.
Yes! I skimmed over the medium article and it seemed to be relevant (tbh, I was happy the link even worked :D) I'll look at it in more detail in the demo notebook.
But this would be a separate issue.
This one LGTM
Related to #9 This PR adds a notebook that uses openai embedding and Langchain framework for the QA task.