feat: Add model and tokenizer kwargs to `TransformersSimilarityRanker`, `SentenceTransformersDocumentEmbedder`, `SentenceTransformersTextEmbedder`

deepset-ai / haystack

:mag: AI orchestration framework to build customizable, production-ready LLM applications. Connect components (models, vector DBs, file converters) to pipelines or agents that can interact with your data. With advanced retrieval methods, it's best suited for building RAG, question answering, semantic search or conversational agent chatbots.

https://haystack.deepset.ai

Apache License 2.0

16.94k stars 1.85k forks source link

feat: Add model and tokenizer kwargs to `TransformersSimilarityRanker`, `SentenceTransformersDocumentEmbedder`, `SentenceTransformersTextEmbedder` #8145

Closed sjrl closed 1 month ago

sjrl commented 1 month ago

Related Issues

fixes #8055

Proposed Changes:

Adds model_kwargs and tokenizer_kwargs to the components TransformersSimilarityRanker, SentenceTransformersDocumentEmbedder, SentenceTransformersTextEmbedder

How did you test it?

Expanded existing unit tests

Notes for the reviewer

Checklist

I have read the contributors guidelines and the code of conduct
I have updated the related issue with new insights and changes
I added unit tests and updated the docstrings
I've used one of the conventional commit types for my PR title: fix:, feat:, build:, chore:, ci:, docs:, style:, refactor:, perf:, test:.
I documented my code
I ran pre-commit hooks and fixed any issue

coveralls commented 1 month ago

Pull Request Test Coverage Report for Build 10210727170

Warning: This coverage report may be inaccurate.

This pull request's base commit is no longer the HEAD commit of its target branch. This means it includes changes from outside the original pull request, including, potentially, unrelated coverage changes.

For more information on this, see Tracking coverage changes with pull request builds.
To avoid this issue with future PRs, see these Recommended CI Configurations.
For a quick fix, rebase this PR at GitHub. Your next report should be accurate.

Details

0 of 0 changed or added relevant lines in 0 files are covered.
32 unchanged lines in 5 files lost coverage.
Overall coverage increased (+0.02%) to 90.115%

Files with Coverage Reduction	New Missed Lines	%
components/embedders/sentence_transformers_document_embedder.py	2	96.49%
components/embedders/sentence_transformers_text_embedder.py	2	95.92%
components/rankers/transformers_similarity.py	6	91.11%
components/retrievers/sentence_window_retriever.py	9	83.33%
components/fetchers/link_content.py	13	79.79%
<!--	Total:	32	-->

Totals
Change from base Build 10197271394:	0.02%
Covered Lines:	6883
Relevant Lines:	7638

💛 - Coveralls

sjrl commented 1 month ago

Hey @dfokina let me know if the docstrings for the new parameters look okay!