search

chickenbestlover / RNN-Time-series-Anomaly-Detection

RNN based Time-series Anomaly detector model implemented in Pytorch.
Apache License 2.0
1.2k stars 318 forks source link

Running in Docker #4

Open drillep opened 6 years ago

drillep commented 6 years ago

I'm trying to run the code on CPU inside a docker container, I believe all my dependencies are in place but I'm receiving the following error.

File "1_train_predictor.py", line 77, in train_dataset = TimeseriesData.batchify(args,TimeseriesData.trainData, args.batch_size) File "/home/RNN-Time-series-Anomaly-Detection/preprocess_data.py", line 65, in batchify batched_data = batched_data.to(device(args.device)) RuntimeError: Cannot initialize CUDA without ATen_cuda library. PyTorch splits its backend into two shared libraries: a CPU library and a CUDA library; this error has occurred because you are trying to use some CUDA functionality, but the CUDA library has not been loaded by the dynamic linker for some reason. The CUDA library MUST be loaded, EVEN IF you don't directly use any symbols from the CUDA library!

This issue appears to be tied into the Pytorch version I am using: pip3 install http://download.pytorch.org/whl/cpu/torch-0.4.1-cp35-cp35m-linux_x86_64.whl After rebuilding the Docker container for my desktop system with GPU using libnccl-dev=2.2.13-1+cuda9.0 my error changed:

RuntimeError: cuda runtime error (35) : CUDA driver version is insufficient for CUDA runtime version at /pytorch/aten/src/THC/THCGeneral.cpp:74

Both of these errors appear to be a result of trying to build the Docker container with a Windows or Mac host OS. Nvidia Docker requires a Linux host.

I was able to run the model using p2.xlarge instance on AWS with Deep Learning AMI using a GPU but the same instance failed to run with a CPU flag.

BehnamTaki commented 5 years ago

I'm trying to run the code on CPU inside a docker container, I believe all my dependencies are in place but I'm receiving the following error.

File "1_train_predictor.py", line 77, in train_dataset = TimeseriesData.batchify(args,TimeseriesData.trainData, args.batch_size) File "/home/RNN-Time-series-Anomaly-Detection/preprocess_data.py", line 65, in batchify batched_data = batched_data.to(device(args.device)) RuntimeError: Cannot initialize CUDA without ATen_cuda library. PyTorch splits its backend into two shared libraries: a CPU library and a CUDA library; this error has occurred because you are trying to use some CUDA functionality, but the CUDA library has not been loaded by the dynamic linker for some reason. The CUDA library MUST be loaded, EVEN IF you don't directly use any symbols from the CUDA library!

This issue appears to be tied into the Pytorch version I am using: pip3 install http://download.pytorch.org/whl/cpu/torch-0.4.1-cp35-cp35m-linux_x86_64.whl After rebuilding the Docker container for my desktop system with GPU using libnccl-dev=2.2.13-1+cuda9.0 my error changed:

RuntimeError: cuda runtime error (35) : CUDA driver version is insufficient for CUDA runtime version at /pytorch/aten/src/THC/THCGeneral.cpp:74

Both of these errors appear to be a result of trying to build the Docker container with a Windows or Mac host OS. Nvidia Docker requires a Linux host.

I was able to run the model using p2.xlarge instance on AWS with Deep Learning AMI using a GPU but the same instance failed to run with a CPU flag.

It means there is no way to fix it on windows?

leonardobegher commented 5 years ago

Could you please post the command line you run to execute the network? to see, if there is anything wrong there!