search

pinecone-io / pinecone-datasets

An open-source dataset library for pre-embedded dataset: create your own data catalog, or use Pinecone's public datasets.
https://pinecone-io.github.io/pinecone-datasets/
32 stars 12 forks source link

Support loading from arbitrary path #10

Closed igiloh-pinecone closed 1 year ago

igiloh-pinecone commented 1 year ago
  1. Changed load logic - lazy create Dataset() object. Retrieve DFs only when needed (in a cached manner).
  2. Added option to load a dataset from any arbitrary path

@miararoy I did some manual testing (see screenshots below) but haven't run the unit tests or added new ones for the new functionality. Leaving that to you 😬

igiloh-pinecone commented 1 year ago

Cached lazy loading:

image
igiloh-pinecone commented 1 year ago

Load from arbitrary path:

image