Make transformers optional + allow pinecone-text[dense] - Githubissues

pinecone-io / canopy

Retrieval Augmented Generation (RAG) framework and context engine powered by Pinecone

https://www.pinecone.io/

Apache License 2.0

976 stars 121 forks source link

Make transformers optional + allow pinecone-text[dense] #266

Closed igiloh-pinecone closed 10 months ago

igiloh-pinecone commented 10 months ago

Problem

transformers shouldn't be a required dependency. It was accidentally added as such.
No way to install pinecone-text with its [dense] extra dependency

Solution

Made transformers optional, only installed with a new anyscale extra
Added a new torch extra (torch is the heaviest installation, so it made more sense as the extra's name than transformers or sentence-transformers, which might be ambiguous)

Type of Change

[x] Breaking change (fix or feature that would cause existing functionality to not work as expected)

Might be slightly breaking for users who currently use the Anyscale Tokenizer, and would need to re-install with the extra.

Test Plan

No affect on current tests