aws / sagemaker-python-sdk

A library for training and deploying machine learning models on Amazon SageMaker
https://sagemaker.readthedocs.io/
Apache License 2.0
2.09k stars 1.14k forks source link

How can I use the custom generator to train a TensorFlow model on SageMaker? #4462

Open dsnsabari opened 7 months ago

dsnsabari commented 7 months ago

I am trying to train a TensorFlow custom classification model using a custom data generator on SageMaker Studio. However, the notebook did not utilize the GPU to train the model.

My instance type: image

I have stored the images in a zip file on an S3 bucket and unzipped them during training, using the temporary path in the custom generator for training.

I want to add data augmentation while training the model.

The example below loads the images in numpy format and passes them directly to the training model. I want to load the images from the S3 bucket in zip format and train with a custom generator and augmentation.

Link to the example

mohanasudhan commented 7 months ago

We are looking into it.