search

minerva-ml / open-solution-data-science-bowl-2018

Open solution to the Data Science Bowl 2018
https://www.kaggle.com/c/data-science-bowl-2018
MIT License
157 stars 43 forks source link

Issue with pytorch multiprocessing #46

Closed etheleon closed 6 years ago

etheleon commented 6 years ago

Hi I'm running the latest version of the master branch c61edd4, locally.

I kept running into a double free or corruption error and after searching around found that by

installing libtcmalloc-minimal4 and adding it to the LD_PRELOAD prevents the error from occurring again. 

sudo apt-get install libtcmalloc-minimal4
export LD_PRELOAD="/usr/lib/libtcmalloc_minimal.so.4"

I'm running this in a nvidia docker container using pytorch 0.3 instead of 2. but otherwise everything else is based on the requirements.txt

this issue has been flagged out both in tensorflow, lightgbm and pytorch

https://github.com/tensorflow/tensorflow/issues/6968 https://github.com/Microsoft/LightGBM/issues/546 https://github.com/ikostrikov/pytorch-a3c/issues/2

jakubczakon commented 6 years ago

@etheleon thanks for the suggestion. As I understand it solved the problem? If so I will add it to the wiki.

etheleon commented 6 years ago

Yea!

On Tue, Mar 13, 2018, 6:34 PM Jakub notifications@github.com wrote:

@etheleon https://github.com/etheleon thanks for the suggestion. As I understand it solved the problem? If so I will add it to the wiki.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/neptune-ml/data-science-bowl-2018/issues/46#issuecomment-372620109, or mute the thread https://github.com/notifications/unsubscribe-auth/ACvGemQ6uHXY2_ZXHOhg4PlGpqYQ-7lqks5td6DGgaJpZM4SoIQW .