Open psyhtest opened 7 years ago
caffe::P2PSync has been removed...
This is not terribly important as we can always use the time_gpu
command of program:caffe
...
Yes looks like caffe::P2PSync has been remove because it not stable at least it was not stable for opencl version as it was metiononed at some BVLC/caffe/issues
Actually, it is important since I use it for crowd-benchmarking, but it seems like the latest versions work fine?
Hi, trying to benchmark a GPU workstation with package:lib-caffe-bvlc-master-cuda-universal
using explore-batch-size-libs-models-benchmarking.py
.
As is, it proposes only CPU packages. I changed program='caffe-time'
to program='caffe-time-cuda'
, but now it fails with the error above.
Is there a workaround to get the benchmark running?
Edit: it might be a problem with lib-caffe: the tags it reads are u'lib,caffe,vcpu'
Thx
Hi @uriv, I've committed a new script just for benchmarking Caffe with CUDA and cuDNN. The key difference is this:
< program='caffe-time'
< cmd_key='default'
---
> program='caffe'
> cmd_key='time_gpu'
Please give it a try!
I've checked tags for package:lib-caffe-bvlc-master-cuda-universal
- they seem to be fine. Did you mean a different package? (I do recall fixing the vcpu
tag in a non-CPU package lately in... CK-TensorFlow:-))
Thanks, @psyhtest. I'll give the patch a try when I get the chance. I've installed package:lib-caffe-bvlc-master-cuda-universal
. If there is a command that prints more info, I can reply with the output.
@uriv, you don't need to patch anything. Just use the new explore-batch-size-libs-models-benchmarking-cuda.py
script.
Please post the output of:
$ ck show env --tags=lib,caffe
Hi @psyhtest,
The benchmark worked with program='caffe', cmd_key='time_gpu'
, after fixing an non-ASCII character '\xc2'
error by appending # -*- coding: utf-8 -*-
to ck-analytics/module/math.variation/module.py
(ref).
Here is the output of ck show env --tags=lib,caffe
:
Env UID: Target OS: Bits: Name: Version: Tags:
21aeea91f24ce4a9 linux-64 64 BVLC Caffe framework (cuda) master-4efdf7ee 64bits,bvlc,caffe,host-os-linux-64,lib,target-os-linux-64,v0,v0.0,vcuda,vmaster,vno-cudnn
vno-cudnn
probably implies that it runs without cudnn, though cudnn was detected during caffe installation. Any suggestions on how to troubleshoot this?
Thanks
@uriv The vno-cudnn
tag means that package:lib-caffe-bvlc-master-cuda-universal
doesn't use cuDNN. In contrast, package:lib-caffe-bvlc-master-cudnn-universal
is tagged with 'vcudnn'. Basically, this is to distinguish the two packages when needed, as both are tagged with vcuda
.
We used to tag the former with vcuda
only and the latter with vcudnn
only. But as this particular program:caffe-time-cuda
can work with both, we added the vcuda
tag also to the latter. So then we needed to add the vno-cudnn
and vcudnn
tags just in case. Hope this answers your question.
@gfursin Could you please check @uriv's comment about fixing an non-ASCII character?
@uriv The vno-cudnn tag means that package:lib-caffe-bvlc-master-cuda-universal doesn't use cuDNN. In contrast, package:lib-caffe-bvlc-master-cudnn-universal is tagged with 'vcudnn'. Basically, this is to distinguish the two packages when needed, as both are tagged with vcuda.
@psyhtest Thanks for the clarification no this.
Yes, I fixed non-ASCII character - I removed it at all and added ASCII... Thanks for noting ...
This is with: