How should we handle optional dependencies?

[vasinov]

As the size of the framework grows, how should we think about handling non-core, aka driver, dependencies?

[collindutter]

I think we want to leverage poetry extras, the main challenge is how to group the extras. Maybe something like this?

llms-openai = [openai, tiktoken]
llms-cohere = [cohere]
llms-anthropic = [anthropic]
llms-huggingface = [huggingface-hub, transformers]

drivers-sql-base = [sqlalchemy]
drivers-sql-aws = [boto3]
drivers-sql-snowflake = [sqlalchemy, snowflake]

drivers-memory-aws = [boto3]
drivers-memory-pinecone = [pinecone]

drivers-vector-marqo = [marqo]
drivers-vector-pinecone = [pinecone]

[dylanholmes]

As an alternative option, how about using namespace packages to split the repo into mulitple repos? (There are obviously pros and cons to this decision though)

Let me know if you want me to dig into this idea more and give a more specific example.

[vasinov]

@dylanholmes, we had the framework split up into three repos at some point but ultimately decided to go back to one, as namespacing introduced more development and usage challenges than it resolved.

[dylanholmes]

Makes sense! I feel like splitting up a code base most often adds complexity, which usually needs be justified by some other motivator -- like splitting up ownership or working around licensing constraints imposed by module specific dependencies.

If the only goal is to avoid adding unnecessary transitive dependencies to griptape applications, then I think @collindutter's suggestion is the way to go. 👍

[erik-hasse]

Just adding my voice in here to say that I'm very interested in using griptape, but the large dependency list (and sqlalchemy<2 specifically #162) makes it impossible for me at the moment. The dependency grouping @collindutter posted would work well for me.

[collindutter]

I created a PR for this change, would love feedback from those who are interested!

[collindutter]

Closing, implemented in https://github.com/griptape-ai/griptape/pull/298