This pull request includes significant changes to the data handling and dependencies in the examples/mnist-pytorch/client project. The most important changes involve replacing the use of torchvision with direct data downloads, updating the data loading logic, and modifying the environment dependencies.
Data Handling Improvements:
examples/mnist-pytorch/client/data.py: Replaced torchvision dataset download with direct data download using requests. This includes setting a random split ID and downloading the corresponding data file from Scaleout's own public bucket.
examples/mnist-pytorch/client/data.py: Updated the load_data function to dynamically find the correct data split ID and verify the existence of the data file.
examples/mnist-pytorch/client/data.py: Removed the splitset and split functions, as the data splitting is now handled by the pre-downloaded data files.
Dependency Updates:
examples/mnist-pytorch/client/python_env.yaml: Removed torchvision from the dependencies and updated numpy versions to ensure compatibility with different platforms and Python versions.
This pull request includes significant changes to the data handling and dependencies in the
examples/mnist-pytorch/clientproject. The most important changes involve replacing the use oftorchvisionwith direct data downloads, updating the data loading logic, and modifying the environment dependencies.Data Handling Improvements:
examples/mnist-pytorch/client/data.py: Replacedtorchvisiondataset download with direct data download usingrequests. This includes setting a random split ID and downloading the corresponding data file from Scaleout's own public bucket.examples/mnist-pytorch/client/data.py: Updated theload_datafunction to dynamically find the correct data split ID and verify the existence of the data file.examples/mnist-pytorch/client/data.py: Removed thesplitsetandsplitfunctions, as the data splitting is now handled by the pre-downloaded data files.Dependency Updates:
examples/mnist-pytorch/client/python_env.yaml: Removedtorchvisionfrom the dependencies and updatednumpyversions to ensure compatibility with different platforms and Python versions.