search

Trusted-AI / AIF360

A comprehensive set of fairness metrics for datasets and machine learning models, explanations for these metrics, and algorithms to mitigate bias in datasets and models.
https://aif360.res.ibm.com/
Apache License 2.0
2.46k stars 840 forks source link

Design and implement a mechanism to download datasets from Cloud Object store #342

Open anupamamurthi opened 2 years ago

anupamamurthi commented 2 years ago

This issue may require collaboration/inspiration from https://github.com/Trusted-AI/AIF360/issues/341

Sometimes data may reside in cloud object store or amazon s3. Some companies may prefer to store "internal/confidential" data in a COS bucket.

It will be good to have a mechanism to implement a "COSStore" which will help us download data from COS to our local machine.

It will be a fun exercise to poke around IBM COS (under the hoods, it uses aws s3 - so it maybe possible to leverage aws s3 api's)

https://www.ibm.com/cloud/object-storage

Python IBM COS SDK: https://github.com/IBM/ibm-cos-sdk-python

Please refer to #341 to learn about how to get started, why we care, possible implementation strategy etc

AditSaxena commented 2 years ago

@anupamamurthi Hi I would like to work on this issue. Can you assign it to me ?