Check failed: error == cudaSuccess (30 vs. 0) unknown error

[ProGamerGov]

I'm not sure what is causing this error:

I am using your AMI, and I ran git pull in the style_transfer directory.

Loading VGG16_SOD_finetune.caffemodel.
WARNING: Logging before InitGoogleLogging() is written to STDERR
E1026 21:52:14.375412  1926 common.cpp:113] Cannot create Cublas handle. Cublas won't be available.
E1026 21:52:14.379542  1926 common.cpp:120] Cannot create Curand generator. Curand won't be available.
F1026 21:52:14.386206  1926 cudnn_conv_layer.cpp:52] Check failed: error == cudaSuccess (30 vs. 0)  unknown error
*** Check failure stack trace: ***

Loading vgg19.caffemodel.
WARNING: Logging before InitGoogleLogging() is written to STDERR
E1026 21:53:59.387698  1934 common.cpp:113] Cannot create Cublas handle. Cublas won't be available.
E1026 21:53:59.392220  1934 common.cpp:120] Cannot create Curand generator. Curand won't be available.
F1026 21:53:59.400104  1934 cudnn_conv_layer.cpp:52] Check failed: error == cudaSuccess (30 vs. 0)  unknown error
*** Check failure stack trace: ***

[crowsonkb]

Is it the new one? The most recent one has a bug I couldn't figure out. Things like nvidia-smi won't even work unless root runs it first. Then it'll work for ordinary users. What happens if you run nvidia-smi?

[ProGamerGov]

ubuntu@ip-Address:~/style_transfer$ nvidia-smi 2>&1 | tee ~/mylog.log

NVIDIA-SMI has failed because it couldn't communicate with the NVIDIA driver. Make sure that the latest NVIDIA driver is installed and running.

ubuntu@ip-Address:~/style_transfer$

Then with sudo:

ubuntu@ip-Address:~/style_transfer$ sudo nvidia-smi 2>&1 | tee ~/mylog.log

Fri Oct 28 20:54:14 2016       
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 370.28                 Driver Version: 370.28                    |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|===============================+======================+======================|
|   0  GRID K520           On   | 0000:00:03.0     Off |                  N/A |
| N/A   27C    P0    43W / 125W |      0MiB /  4036MiB |      0%      Default |
+-------------------------------+----------------------+----------------------+

+-----------------------------------------------------------------------------+
| Processes:                                                       GPU Memory |
|  GPU       PID  Type  Process name                               Usage      |
|=============================================================================|
|  No running processes found                                                 |
+-----------------------------------------------------------------------------+

ubuntu@ip-Address:~/style_transfer$

[ProGamerGov]

Trying: ubuntu@ip-Address:~/style_transfer$ sudo python3 style_transfer.py results in:

Traceback (most recent call last):
  File "style_transfer.py", line 25, in <module>
    import numpy as np
ImportError: No module named 'numpy'

[ProGamerGov]

I tried to get it working by installing the missing modules via Pip3:

sudo apt-get install python3-pip

sudo pip3 install numpy

sudo pip3 install image

 sudo pip3 install posix-ipc

sudo pip3 install scipy

I stopped when I saw it was missing the 'Caffe' module:

Process ForkProcess-1:
Traceback (most recent call last):
  File "/usr/lib/python3.4/multiprocessing/process.py", line 254, in _bootstrap
    self.run()
  File "/usr/lib/python3.4/multiprocessing/process.py", line 93, in run
    self._target(*self._args, **self._kwargs)
  File "style_transfer.py", line 1371, in init_model
    import caffe
ImportError: No module named 'caffe'

[crowsonkb]

That's really weird... You can fix the 'caffe' error by setting the PYTHONPATH environment variable to pycaffe's build directory. It's like /home/ubuntu/caffe/python or such. I really should have a configuration file or something that sets the PYTHONPATH that users can just edit if needed...

[crowsonkb]

I made a new AMI with a fixed nVidia driver: https://us-west-2.console.aws.amazon.com/ec2/v2/home?region=us-west-2#LaunchInstanceWizard:ami=ami-ad8b2dcd

[ProGamerGov]

The newly fixed AMI seems to work properly now.