search

gowthamchanduthota / scikit-learn-aws-s3-extension

To seamlessly extend the capabilities of the scikit-learn ecosystem by integrating dynamic cloud-based data management using AWS S3, enabling machine learning practitioners and researchers to efficiently manage large datasets and model outputs without the need for extensive local storage. This integration will streamline the workflow of model train
0 stars 0 forks source link

As an User/Developer, I need an ability within scikit-learn module to connect to AWS S3 with primary pull and push objects features from s3. #1

Open gowthamchanduthota opened 1 month ago

gowthamchanduthota commented 1 month ago

Estimation Time: Large (8 Days)

Acceptance Criteria:

  1. Inbuilt ability in scikit-learn module to connect to AWS s3 using decorators/class.
  2. User should be able to provide his credentials securely.
  3. Feature function to Pull objects (images, files) from S3 Bucket.
  4. Function to push reports, logs, files to S3 Bucket
gowthamchanduthota commented 4 weeks ago

Added S3Helper Class, which includes -

  1. Set S3 Client and use user creds optionally.
  2. downloading images using the path and display optionally.
  3. List all objects in the bucket or specified path.