search

sassoftware / sas-viya-programming

Code samples and materials to help you learn to access SAS Viya services by writing programs in Python and other open-source languages.
Apache License 2.0
138 stars 134 forks source link

CUDA module cannot be loaded #21

Open roy12369 opened 6 years ago

roy12369 commented 6 years ago

Hi @bensloane ,

I applied the python and showed the error. Does it mean that I must have GPU in my viya server? If so,how can I change to run on the cpu? cuda

bensloane commented 6 years ago

Hey @roy12369 is there any particular reason you are running this as a python script? I would first just try running it as is in Jupyter. Looks like you receive the error when you start training the model and by default in this notebook i have gpu=dict(devices={0,1}) in the call to dltrain. I would remove this line and see if that helps at all. Thanks.

roy12369 commented 6 years ago

Hi @bensloane , Would it cause any different result when running it as python script or Jupyter? If so,I would advice my customer to run the code on the Jupyter. Looks like the issue somehow relates to the GPU.

bensloane commented 6 years ago

No it would not but the notebook was built to be interactive and useful for exploring the fashion mnist data. You just lose a little of that investigative workflow outside of Jupyter but of course you can still run this as a script more just curious as to why. Yes just remove that line stated above and you will train on the cpu.

roy12369 commented 6 years ago

Hi @bensloane , Got it! Many Thanks! I thought my customer just edited the code on spyder and saved it as a python script. Then just tried to send me the error screen with running the python script.

nadolsw commented 5 years ago

Hello, I am a current SAS employee attempting to stand up a demo using the SAS ML CAS Free Trial accessed via this link: https://www.sas.com/en_us/software/machine-learning-cloud.html

I have encountered the same error listed above (ERROR: CUDA module cannot be loaded.) and was hoping to get some guidance on how to resolve or who to contact for further assistance as this comment chain is the only reference I can find online related to the error.

Here is the code I am attempting to submit: 74 ods output OptIterHistory=ObjectModeliter; 75 proc cas; 76 dlTrain / table={name='SmallImageDatashuffled', where='PartInd=1'} model='ConVNN' 77 modelWeights={name='ConVTrainedWeights_d', replace=1} 78 bestweights={name='ConVbestweights', replace=1} 79 inputs='image' 80 target='label' nominal={'label'} 81 GPU=True 82 ValidTable={name='SmallImageDatashuffled', where='PartInd=2'} 83 optimizer={minibatchsize=80,
84 algorithm={method='ADAM', lrpolicy='Step', gamma=0.6, stepsize=5 85 beta1=0.9, beta2=0.999, learningrate=.01}
86 maxepochs=25} 87 seed=12345 88 ; 89 run; NOTE: Active Session now CASAUTO. ERROR: CUDA module cannot be loaded. ERROR: The action stopped due to errors.

Note that the code dos run if I change GPU=False but I'm attempting to leverage the GPU rather than CPU for this task. Any help is much appreciated!

bensloane commented 5 years ago

Are you running on a server that has CUDA installed? Using the trial environment does not include a GPU as far as I know. Try changing GPU=False. Also I recommend checking the dlpy repo for more demo examples!

nadolsw commented 5 years ago

Ah, that would explain it. Do you happen to know if there are any commands which could be submitted in order to verify whether or not the cloud server has a GPU available?

bensloane commented 5 years ago

You want to see if CUDA is installed so there's a couple places to check. By no means is this an exhaustive list but you could try the following:

nvidia-smi nvcc --version

If you don't have luck with those take a look at some of the CUDA docs

nadolsw commented 5 years ago

Wonderful - thank you so much!