Error running example from docker image

rycolab commented 4 years ago

After pulling and launching the tase docker container, I am unable to run the basic example script. It exits with the following error:

meistecl@su-lee:~/Documents/repositories/taso$ docker/run_docker.sh tasoml/cuda100
WORKSPACE: /usr/TASO
IMAGE NAME: tasoml/cuda100
DOCKER BINARY: nvidia-docker
root@su-lee:/usr/TASO# python examples/resnet50.py 
Cuda failure: 2
/usr/TASO/src/cudnn/ops_cudnn.cu:42
Aborting...
root@su-lee:/usr/TASO#

Additionally, if I copy a sample onnx model file to the container (just a generic .onnx chosen from their GitHub repo) and run taso.load() on the file, it exits with the same error.

Thank you for your help!

jiazhihao commented 4 years ago

Hi, can you let us know your GPU version? Currently, TASO pre-allocate around 8GB of GPU memory to allow measuring the cost of large operators, it seems TASO ran out of memory on your machine.

cimeister commented 4 years ago

I'm running this on a single GeForce RTX 2070

jiazhihao / TASO

Error running example from docker image #26