seperated data loaders classes

aidecentralized / sonar

SONAR - Self-Organizing Network of Aggregated Representations

MIT License

7 stars 27 forks source link

seperated data loaders classes #14

Closed hardlyhuman closed 2 months ago

hardlyhuman commented 2 months ago

This PR is solving issue #7 by refactoring the data_utils.py file to dynamically load only the necessary data loader libraries instead of importing all of them upfront. This change prevents the need to load all the libraries in memory thus reducing unnecessary dependencies.

Changes:

Seperated Data Loader Classes
Implemented dynamic imports for data loaders
Updated the organization of data_utils.py to align with dynamic import strategy.

hardlyhuman commented 2 months ago

Tested with Cifar10 for 2 rounds of FL and it was successful. Would you recommend to test with all other datasets as well once?

Also, I have reduced the training set size drastically since I am runing it on my PC and primarily testing the functionality rather than ML accuracy part.

tremblerz commented 2 months ago

should be good enough, thanks!