ckpt to pb - Githubissues

cccxinli commented 5 years ago

您好，我看网络最后输出的是final_box,final_score这几个变量，但是这几个变量没有命名，所以我如果要做模型固化的操作，是不是需要手动加上名字再重新训练？

yangxue0827 commented 5 years ago

https://github.com/DetectionTeamUCAS/FPN_Tensorflow/blob/master/libs/export_pbs/exportPb.py @cccxinli

cccxinli commented 5 years ago

非常感谢

cccxinli commented 5 years ago

您好，我把模型放到C++下检测，发现检测速度慢了十几倍，请问您有做过C++下的调用吗？

BigWilky commented 5 years ago

请问生成后的pb为什么会有两个？不应该是一个的吗？

cccxinli commented 5 years ago

应该是那句replace导致的，能用的是带Frozen的那个。

BigWilky commented 5 years ago

嗯，对，我刚刚试了是replace的关系。不过话说你生成的pd文件可以直接部署吗，我用tf serving docker部署的时候一直不成功，错误信息如下： `2019-08-01 08:51:57.152804: I tensorflow_serving/model_servers/server.cc:82] Building single TensorFlow model file config: model_name: my_model model_base_path: /models/my_model

2019-08-01 08:51:57.152913: I tensorflow_serving/model_servers/server_core.cc:461] Adding/updating models.

2019-08-01 08:51:57.152925: I tensorflow_serving/model_servers/server_core.cc:558] (Re-)adding model: my_model

2019-08-01 08:51:57.253586: I tensorflow_serving/core/basic_manager.cc:739] Successfully reserved resources to load servable {name: my_model version: 1}

2019-08-01 08:51:57.253666: I tensorflow_serving/core/loader_harness.cc:66] Approving load for servable version {name: my_model version: 1}

2019-08-01 08:51:57.253709: I tensorflow_serving/core/loader_harness.cc:74] Loading servable version {name: my_model version: 1}

2019-08-01 08:51:57.253767: I external/org_tensorflow/tensorflow/contrib/session_bundle/bundle_shim.cc:363] Attempting to load native SavedModelBundle in bundle-shim from: /models/my_model/1

2019-08-01 08:51:57.253844: I external/org_tensorflow/tensorflow/cc/saved_model/reader.cc:31] Reading SavedModel from: /models/my_model/1

2019-08-01 08:51:57.381834: I external/org_tensorflow/tensorflow/cc/saved_model/reader.cc:54] Reading meta graph with tags { serve }

2019-08-01 08:51:57.388025: I external/org_tensorflow/tensorflow/cc/saved_model/loader.cc:259] SavedModel load for tags { serve }; Status: fail. Took 134178 microseconds.

2019-08-01 08:51:57.388045: E tensorflow_serving/util/retrier.cc:37] Loading servable: {name: my_model version: 1} failed: Not found: Could not find meta graph def matching supplied tags: { serve }. To inspect available tag-sets in the SavedModel, please use the SavedModel CLI: `saved_model_cli``

cccxinli commented 5 years ago

我直接编译的C++ tensorflow API部署的，没有用tf serving docker。

BigWilky commented 5 years ago

哦哦哦，C++ 的API 怎么样，速度的问题解决了吗

cccxinli commented 5 years ago

没有完全解决

pan-lj commented 5 years ago

@cccxinli 请问你c++和python预测结果是一样的吗，为什么我的c++下准确率降低了很多，方便的话可以看下你c++调用的代码吗？

cccxinli commented 5 years ago

我看看你c++下的检测代码？用没用到一行output.flat_outer_dim？

---Original--- From: "pan-lj"<notifications@github.com> Date: Mon, Sep 30, 2019 20:46 PM To: "DetectionTeamUCAS/FPN_Tensorflow"<FPN_Tensorflow@noreply.github.com>; Cc: "Mention"<mention@noreply.github.com>;"Thunder"<792512955@qq.com>; Subject: Re: [DetectionTeamUCAS/FPN_Tensorflow] ckpt to pb (#78)

@cccxinli 请问你c++和python预测结果是一样的吗，为什么我的c++下准确率降低了很多，方便的话可以看下你c++调用的代码吗？

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

pan-lj commented 5 years ago

没用到啊 `img = imread(image_path + imgVec[i], 1); cout << "Process img: " << imgVec[i] << endl; //cvtColor(img, img, COLOR_BGR2RGB); if (img.empty()) { cout << "can't open the image!!!!!!!" << endl; return -1; }

    //创建一个tensor作为输入网络的接口
    Tensor resized_tensor(DT_UINT8, TensorShape({ img.rows,img.cols,3 })); //DT_FLOAT

    //将Opencv的Mat格式的图片存入tensor
    CVMat_to_Tensor(img, &resized_tensor, input_height, input_width);

    //cout << resized_tensor.DebugString() << endl;

    /*-----------------------------------用网络进行测试-----------------------------------------*/
    //cout << endl << "<-------------Running the model with test_image--------------->" << endl;
    //前向运行，输出结果一定是一个tensor的vector
    vector<tensorflow::Tensor> outputs;
    /*imshow("1", img);
    waitKey(0);*/
    double startTime = getTickCount();
    Status status_run = session->Run({ { "input_img", resized_tensor } }, { "DetResults" }, {}, &outputs);
    double endTime = getTickCount();
    double totalTime = (endTime - startTime) / getTickFrequency() * 1000; //获取的单位是ms`

`void CVMat_to_Tensor(Mat img, Tensor output_tensor, int input_rows, int input_cols) { //创建一个指向tensor的内容的指针 uint8 p = output_tensor->flat().data();

//创建一个Mat，与tensor的指针绑定,改变这个Mat的值，就相当于改变tensor的值
cv::Mat tempMat(img.rows, img.cols, CV_8UC3, p);
img.convertTo(tempMat, CV_8UC3);

//    waitKey(0);

}`

cccxinli commented 5 years ago

不太懂你最后为什么把指针p和mat绑定？output里存的不是一堆框，类别，置信度的信息吗？

---Original--- From: "pan-lj"<notifications@github.com> Date: Mon, Sep 30, 2019 21:00 PM To: "DetectionTeamUCAS/FPN_Tensorflow"<FPN_Tensorflow@noreply.github.com>; Cc: "Mention"<mention@noreply.github.com>;"Thunder"<792512955@qq.com>; Subject: Re: [DetectionTeamUCAS/FPN_Tensorflow] ckpt to pb (#78)

没用到啊 img = imread(image_path + imgVec[i], 1); cout << "Process img: " << imgVec[i] << endl; //cvtColor(img, img, COLOR_BGR2RGB); if (img.empty()) { cout << "can't open the image!!!!!!!" << endl; return -1; } //创建一个tensor作为输入网络的接口 Tensor resized_tensor(DT_UINT8, TensorShape({ img.rows,img.cols,3 })); //DT_FLOAT //将Opencv的Mat格式的图片存入tensor CVMat_to_Tensor(img, &resized_tensor, input_height, input_width); //cout << resized_tensor.DebugString() << endl; /*-----------------------------------用网络进行测试-----------------------------------------*/ //cout << endl << "<-------------Running the model with test_image--------------->" << endl; //前向运行，输出结果一定是一个tensor的vector vector<tensorflow::Tensor> outputs; /*imshow("1", img); waitKey(0);*/ double startTime = getTickCount(); Status status_run = session->Run({ { "input_img", resized_tensor } }, { "DetResults" }, {}, &outputs); double endTime = getTickCount(); double totalTime = (endTime - startTime) / getTickFrequency() * 1000; //获取的单位是ms
void CVMat_to_Tensor(Mat img, Tensor* output_tensor, int input_rows, int input_cols) { //创建一个指向tensor的内容的指针 uint8 *p = output_tensor->flat().data(); //创建一个Mat，与tensor的指针绑定,改变这个Mat的值，就相当于改变tensor的值 cv::Mat tempMat(img.rows, img.cols, CV_8UC3, p); img.convertTo(tempMat, CV_8UC3); // waitKey(0); }

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

pan-lj commented 5 years ago

@cccxinli 那个是网上找的把输入图片mat转为tensor的代码，那个命名可能有点问题，output_tensor应该是input_tensor，最后c++预测结果里置信度都特别低

pan-lj commented 5 years ago

`void CVMat_to_Tensor(Mat img, Tensor input_tensor, int input_rows, int input_cols) { //创建一个指向tensor的内容的指针 uint8 p = input_tensor->flat().data();

//创建一个Mat，与tensor的指针绑定,改变这个Mat的值，就相当于改变tensor的值
cv::Mat tempMat(img.rows, img.cols, CV_8UC3, p);
img.convertTo(tempMat, CV_8UC3);

}

int main() { /--------------------------------配置关键信息------------------------------/ string model_path = "D:\VS2015\FasterRCNN_Res101_FPN_Frozen_1.8.pb"; string image_path = "D:\VS2015\DefectImage\"; int input_height = 608; int input_width = 608;

/*--------------------------------创建session------------------------------*/
Session* session;
GraphDef graph_def;
SessionOptions opts;
TF_CHECK_OK(ReadBinaryProto(Env::Default(), model_path, &graph_def));

// Set GPU options
//opts.config.mutable_gpu_options()->set_per_process_gpu_memory_fraction(0.9);
opts.config.mutable_gpu_options()->set_allow_growth(true);

// create a new session
TF_CHECK_OK(NewSession(opts, &session));

// Load graph into session
TF_CHECK_OK(session->Create(graph_def));

cout << "<----Successfully created session and load graph.------->" << endl;

/*---------------------------------载入测试图片-------------------------------------*/
//cout << endl << "<------------loading test_image-------------->" << endl;
vector<string> imgVec = { "00002.jpg", "00104.jpg", "00207.jpg", "00398.jpg", "00460.jpg", "00553.jpg", "00645.jpg", 
                                            "00781.jpg", "00868.jpg", "01000.jpg", "01131.jpg", "01194.jpg", "01263.jpg", "01407.jpg"};

for (int i = 0; i < imgVec.size(); ++i) {
    Mat img;
    img = imread(image_path + imgVec[i], 1);
    cout << "Process img: " << imgVec[i] << endl;
    //cvtColor(img, img, COLOR_BGR2RGB);
    if (img.empty())
    {
        cout << "can't open the image!!!!!!!" << endl;
        return -1;
    }

    //创建一个tensor作为输入网络的接口
    Tensor resized_tensor(DT_UINT8, TensorShape({ img.rows,img.cols,3 })); //DT_FLOAT
    //将Opencv的Mat格式的图片存入tensor
    CVMat_to_Tensor(img, &resized_tensor, input_height, input_width);
    //cout << resized_tensor.DebugString() << endl;

    //cout << endl << "<-------------Running the model with test_image--------------->" << endl;
    vector<tensorflow::Tensor> outputs;
    double startTime = getTickCount();
    Status status_run = session->Run({ { "input_img", resized_tensor } }, { "DetResults" }, {}, &outputs);
    double endTime = getTickCount();
    double totalTime = (endTime - startTime) / getTickFrequency() * 1000; //获取的单位是ms
    cout << totalTime << endl;
    if (!status_run.ok()) {
        cout << "ERROR: RUN failed..." << std::endl;
        cout << status_run.ToString() << "\n";
        cout << "failed" << endl;
        return -1;
    }

    int pre_num = outputs[0].dim_size(0);  // 模型预测的目标数量
    //cout << pre_num << endl;
    auto tmap = outputs[0].tensor<float, 2>();
    float probability = 0.001;  //自己设定的score阈值
    for (int pre_i = 0; pre_i < pre_num; pre_i++)
    {
        if (tmap(pre_i, 1) < probability)
        {
            break;
        }
        cout << "Class ID: " << tmap(pre_i, 0) - 1 << endl;
        cout << "Probability: " << tmap(pre_i, 1) << endl;
        string id = to_string(int(tmap(pre_i, 0)) - 1);
        int xmin = int(tmap(pre_i, 2));
        int ymin = int(tmap(pre_i, 3));
        int xmax = int(tmap(pre_i, 4));
        int ymax = int(tmap(pre_i, 5));
        //cout << "Xmin is: " << xmin << endl;
        //cout << "Ymin is: " << ymin << endl;
        //cout << "Xmax is: " << xmax << endl;
        //cout << "Ymax is: " << ymax << endl;
        rectangle(img, Point(xmin, ymin), Point(xmax, ymax), Scalar(255, 0, 0), 1, 1, 0);
        putText(img, id, Point(xmin, ymin), FONT_HERSHEY_COMPLEX, 1.0, Scalar(255, 0, 0), 1);
    }
    imshow("img", img);
    waitKey(0);
}
return 0;

}`

cccxinli commented 5 years ago

auto tmap = outputs[0].tensor<float, 2>(); 这行代码我这用的是auto xxx=outputs[0].flat_outer_dims<float,2>(); 不知道是不是这个原因

------------------ 原始邮件 ------------------ 发件人: "pan-lj"<notifications@github.com>; 发送时间: 2019年9月30日(星期一) 晚上9:14 收件人: "DetectionTeamUCAS/FPN_Tensorflow"<FPN_Tensorflow@noreply.github.com>; 抄送: "墨明棋妙"<792512955@qq.com>;"Mention"<mention@noreply.github.com>; 主题: Re: [DetectionTeamUCAS/FPN_Tensorflow] ckpt to pb (#78)

`void CVMat_to_Tensor(Mat img, Tensor input_tensor, int input_rows, int input_cols) { //创建一个指向tensor的内容的指针 uint8 p = input_tensor->flat().data(); //创建一个Mat，与tensor的指针绑定,改变这个Mat的值，就相当于改变tensor的值 cv::Mat tempMat(img.rows, img.cols, CV_8UC3, p); img.convertTo(tempMat, CV_8UC3);
}

int main() { /--------------------------------配置关键信息------------------------------/ string model_path = "D:\VS2015\FasterRCNN_Res101_FPN_Frozen_1.8.pb"; string image_path = "D:\VS2015\DefectImage\"; int input_height = 608; int input_width = 608; /--------------------------------创建session------------------------------/ Session session; GraphDef graph_def; SessionOptions opts; TF_CHECK_OK(ReadBinaryProto(Env::Default(), model_path, &graph_def)); // Set GPU options //opts.config.mutable_gpu_options()->set_per_process_gpu_memory_fraction(0.9); opts.config.mutable_gpu_options()->set_allow_growth(true); // create a new session TF_CHECK_OK(NewSession(opts, &session)); // Load graph into session TF_CHECK_OK(session->Create(graph_def)); cout << "<----Successfully created session and load graph.------->" << endl; /---------------------------------载入测试图片-------------------------------------/ //cout << endl << "<------------loading test_image-------------->" << endl; vector<string> imgVec = { "00002.jpg", "00104.jpg", "00207.jpg", "00398.jpg", "00460.jpg", "00553.jpg", "00645.jpg", "00781.jpg", "00868.jpg", "01000.jpg", "01131.jpg", "01194.jpg", "01263.jpg", "01407.jpg"}; for (int i = 0; i < imgVec.size(); ++i) { Mat img; img = imread(image_path + imgVec[i], 1); cout << "Process img: " << imgVec[i] << endl; //cvtColor(img, img, COLOR_BGR2RGB); if (img.empty()) { cout << "can't open the image!!!!!!!" << endl; return -1; } //创建一个tensor作为输入网络的接口 Tensor resized_tensor(DT_UINT8, TensorShape({ img.rows,img.cols,3 })); //DT_FLOAT //将Opencv的Mat格式的图片存入tensor CVMat_to_Tensor(img, &resized_tensor, input_height, input_width); //cout << resized_tensor.DebugString() << endl; //cout << endl << "<-------------Running the model with test_image--------------->" << endl; vector<tensorflow::Tensor> outputs; double startTime = getTickCount(); Status status_run = session->Run({ { "input_img", resized_tensor } }, { "DetResults" }, {}, &outputs); double endTime = getTickCount(); double totalTime = (endTime - startTime) / getTickFrequency() 1000; //获取的单位是ms cout << totalTime << endl; if (!status_run.ok()) { cout << "ERROR: RUN failed..." << std::endl; cout << status_run.ToString() << "\n"; cout << "failed" << endl; return -1; } int pre_num = outputs[0].dim_size(0); // 模型预测的目标数量 //cout << pre_num << endl; auto tmap = outputs[0].tensor<float, 2>(); float probability = 0.001; //自己设定的score阈值 for (int pre_i = 0; pre_i < pre_num; pre_i++) { if (tmap(pre_i, 1) < probability) { break; } cout << "Class ID: " << tmap(pre_i, 0) - 1 << endl; cout << "Probability: " << tmap(pre_i, 1) << endl; string id = to_string(int(tmap(pre_i, 0)) - 1); int xmin = int(tmap(pre_i, 2)); int ymin = int(tmap(pre_i, 3)); int xmax = int(tmap(pre_i, 4)); int ymax = int(tmap(pre_i, 5)); //cout << "Xmin is: " << xmin << endl; //cout << "Ymin is: " << ymin << endl; //cout << "Xmax is: " << xmax << endl; //cout << "Ymax is: " << ymax << endl; rectangle(img, Point(xmin, ymin), Point(xmax, ymax), Scalar(255, 0, 0), 1, 1, 0); putText(img, id, Point(xmin, ymin), FONT_HERSHEY_COMPLEX, 1.0, Scalar(255, 0, 0), 1); } imshow("img", img); waitKey(0); } return 0;
}`

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

pan-lj commented 5 years ago

@cccxinli 其他地方还有什么不一样的吗，我改成flat_outer_dims还是一样置信度很低

cccxinli commented 5 years ago

图片转tensor不一样我用的是网上一个ReadTensorFromImageFile函数

------------------ 原始邮件 ------------------ 发件人: "pan-lj"<notifications@github.com>; 发送时间: 2019年9月30日(星期一) 晚上9:22 收件人: "DetectionTeamUCAS/FPN_Tensorflow"<FPN_Tensorflow@noreply.github.com>; 抄送: "墨明棋妙"<792512955@qq.com>;"Mention"<mention@noreply.github.com>; 主题: Re: [DetectionTeamUCAS/FPN_Tensorflow] ckpt to pb (#78)

@cccxinli 其他地方还有什么不一样的吗，我改成flat_outer_dims还是一样置信度很低

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

pan-lj commented 5 years ago

我找找试试，可以加个好友吗

AllentDan commented 5 years ago

@cccxinli 同用c++ api部署的，但是之前的项目编译的tensorflow版本低于1.10，导致这次推理时候不能用nms。不知道是否方便分享一下你编译的文件，毕竟windows下编译一次tensorflow太折磨人了

cccxinli commented 5 years ago

加我

---Original--- From: "AllentDan"<notifications@github.com> Date: Tue, Oct 8, 2019 13:43 PM To: "DetectionTeamUCAS/FPN_Tensorflow"<FPN_Tensorflow@noreply.github.com>; Cc: "Mention"<mention@noreply.github.com>;"Thunder"<792512955@qq.com>; Subject: Re: [DetectionTeamUCAS/FPN_Tensorflow] ckpt to pb (#78)

@cccxinli 同用c++ api部署的，但是之前的项目编译的tensorflow版本低于1.10，导致这次推理时候不能用nms。不知道是否方便分享一下你编译的文件，毕竟windows下编译一次tensorflow太折磨人了

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

DetectionTeamUCAS / FPN_Tensorflow

ckpt to pb #78