search

aws / amazon-sagemaker-examples

Example 📓 Jupyter notebooks that demonstrate how to build, train, and deploy machine learning models using 🧠 Amazon SageMaker.
https://sagemaker-examples.readthedocs.io
Apache License 2.0
10k stars 6.74k forks source link

Notebook exceeds memory?; OS Error : Cannot allocate memory #1469

Open ngluna opened 4 years ago

ngluna commented 4 years ago

In the notebook: https://github.com/awslabs/amazon-sagemaker-examples/blob/master/sagemaker_batch_transform/introduction_to_batch_transform/batch_transform_pca_dbscan_movie_clusters.ipynb

After training and within the "Batch Predictions PCA" I ran into a:
OSError: [Errno 12] Cannot allocate memory

How to recreate error:

  1. Git clone: https://github.com/awslabs/amazon-sagemaker-examples.git
  2. Use default kernel: conda_python3
  3. "Run All"

What I tried: Shut down & restarted notebook instance.

In the terminal I ran: free -h

which returned:

          total       used       free     shared    buffers     cached
Mem:   3.9G       3.7G       202M        64K        97M       526M

It might be that the dataset copied from S3 is causing me to exceed memory?

berkayalan commented 1 year ago

Did you find any solution for that?