aws / amazon-sagemaker-examples

Example 📓 Jupyter notebooks that demonstrate how to build, train, and deploy machine learning models using 🧠 Amazon SageMaker.
https://sagemaker-examples.readthedocs.io
Apache License 2.0
9.8k stars 6.67k forks source link

How do you use the custom generator to train the TensorFlow model on PageMaker? #4574

Open dsnsabari opened 4 months ago

dsnsabari commented 4 months ago

I am trying to train a TensorFlow custom classification model using a custom data generator on SageMaker Studio. However, the notebook did not utilize the GPU to train the model.

My instance type: image

I have stored the images in a zip file on an S3 bucket and unzipped them during training, using the temporary path in the custom generator for training.

I want to add data augmentation while training the model.

The example below loads the images in numpy format and passes them directly to the training model. I want to load the images from the S3 bucket in zip format and train with a custom generator and augmentation.

Link to the example