cat't load pretrained model in gpu model

[dfcv24]

http://caffe2.ai/docs/tutorial-loading-pre-trained-models.html

with open("init_net.pb") as f: init_net = f.read() with open("predict_net.pb") as f: predict_net = f.read() print workspace.has_gpu_support p = workspace.Predictor(init_net, predict_net) results = p.run([img])

I changed my old caffe model to caffe2, and use workspace.predictor but find hundreds of times slower when inference,I find it use cpu rather than gpu. And I have no idea the implementation of gpu. can someone help me ?

[Yangqing]

@salexspb @bwasti

Ah right, to support flexible device placement, Caffe2 now explicitly requires one to set device affinity. Bram, Alex - would you guys like to make the predictor interface accept device specifications? Under the hood we can definitely route things to GPUs (i.e. loading things to GPU, and do computation on GPU too)

[salexspb]

@bwasti mentioned that he is going to take care of this along with examples for mobile format saving / loading.

[sergey-serebryakov]

I was able to achieve custom placing by assigning device_option of NetDef structure:

def load_net_def(def_file, device_opts=None):
    if def_file is None:
        return None
    net_def = NetDef()
    with open(def_file) as fobj:
        net_def.ParseFromString(fobj.read())
        if device_opts is not None:
            net_def.device_option.CopyFrom(device_opts)
    return net_def

[Yangqing]

@sergey-serebryakov yup, that is the right way to do it. We'll definitely want to fold it into the predictor interface.

(Just in case you are wondering, we use C++ predictor and explicitly specify devices for serving, in a similar way as you do here)

[zhangguanqun]

With Caffe, switch from CPU to GPU is pretty simple, but with Caffe2, I haven't done yet. Better to write a tutorial that how to run a network with GPU.

[KeyKy]

https://github.com/caffe2/models/issues/5 could anyone help me?

from caffe2.proto import caffe2_pb2
from caffe2.python import core, workspace, models
import numpy as np

net_def = caffe2_pb2.NetDef()
with open('./predict_net.pb') as f:
    net_def.ParseFromString(f.read())
device_opts = caffe2_pb2.DeviceOption()
device_opts.device_type = 1
device_opts.cuda_gpu_id = 3
net_def.device_option.CopyFrom(device_opts)
predict_net = net_def.SerializeToString()

with open("./init_net.pb") as f:
    init_net = f.read()

p = workspace.Predictor(init_net, predict_net)

img = np.zeros([10, 3, 227, 227], dtype=np.float32)
results = p.run([img])

However, i get wrong type for the Blob instance. Blob contains caffe2::Tensor<caffe2::CPUContext while caller expects caffe2::Tensor

[zhangguanqun]

我也没有搞定 你搞定了吗？

2017年4月27日星期四，康洋 notifications@github.com 写道：

caffe2/models#5 https://github.com/caffe2/models/issues/5 could anyone help me?

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/caffe2/caffe2/issues/323#issuecomment-297709965, or mute the thread https://github.com/notifications/unsubscribe-auth/APnF9cGzdgnQuKDwJhL6WhRpo1P8lORSks5r0JUogaJpZM4NCkSs .

[KeyKy]

@zhangguanqun , Maybe i found a solution, here is my code

import numpy as np
#import caffe2.python._import_c_extension as C
from caffe2.proto import caffe2_pb2
import os, time

import caffe2.python._import_c_extension as C

base_path = '/data2/models/bvlc_alexnet'

device_opts = caffe2_pb2.DeviceOption()
device_opts.device_type = caffe2_pb2.CUDA
device_opts.cuda_gpu_id = 0

init_def = caffe2_pb2.NetDef()
with open(os.path.join(base_path, 'init_net.pb'), 'r') as f:
    init_def.ParseFromString(f.read())
    init_def.device_option.CopyFrom(device_opts)
    C.run_net_once(init_def.SerializeToString())

net_def = caffe2_pb2.NetDef()
with open(os.path.join(base_path, 'predict_net.pb'), 'r') as f:
    net_def.ParseFromString(f.read())
    net_def.device_option.CopyFrom(device_opts)
    C.create_net(net_def.SerializeToString())

C.feed_blob('data', np.random.rand(1, 3, 227, 227).astype(np.float32),
        device_opts.SerializeToString())

start = time.time()
for i in range(1000):
    C.run_net('AlexNet', 1)
end = time.time()
print('Run time per RunNet: {}'.format((end - start) / 1000))

[RobertBiehl]

@KeyKy Any update on that?

[tdp2110]

@sergey-serebryakov : do you know what kind of DeviceOption to set for CUDA? I've tried DeviceOption(caffe2_pb2.CUDA) but that hits @KeyKy's error.

[sergey-serebryakov]

@tdp2110 This works for me:

device_opts = core.DeviceOption(caffe2_pb2.CUDA, gpu_id)

where gpu_id is a valid integer identifier of a GPU you want to use.

[jnulzl]

@dfcv24 @zhangguanqun Hello,I want to load pretrained model in gpu mode and then predict a image by Predictor interface, but I cannot do it. do you make it????? Can you give me a simple example?? Thank you so much!