How to generate training data with "LMDB" as a suffix?

[samylee]

Hi @miao0913 , Thanks for your code. When I check your "train.prototxt", the data type is "Data", so how to generate training data with "LMDB" as a suffix? By the way, where is the "CrowdData" layer when I check the "caffe.proto"?

Thank you very much!

Samylee

[miao0913]

Hi,

Thanks for you interest. LMDB is a common data format used in Caffe, I believe you can find plenty of resources of it on the Internet. e.g. https://stackoverflow.com/questions/30980338/lmdb-files-and-how-they-are-used-for-caffe-deep-learning-network

Please ignore the CrowdData layer as it is not mentioned in the prototxt.

Cheers MJ

2018-03-13 8:01 GMT+01:00 samylee notifications@github.com:

Hi @miao0913 https://github.com/miao0913 , Thanks for your code. When I check your "train.prototxt", the data type is "Data", so how to generate training data with "LMDB" as a suffix? By the way, where is the "CrowdData" layer when I check the "caffe.proto"?

Thank you very much!

Samylee

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/miao0913/SaCNN-CrowdCounting-Tencent_Youtu/issues/2, or mute the thread https://github.com/notifications/unsubscribe-auth/APmD0QHInlE_4QfM50My99HH-SA89f7Nks5td262gaJpZM4SoLHs .

-- Dr. Miaojing Shi Research Scientist LINKMEDIA - Inria Rennes Email: miaojing.shi@inria.fr CC: miao0913@gmail.com

Web: https://sites.google.com/site/miaojingshi/

[samylee]

Hi @miao0913 , Thanks for your answer, I sent an email to your email address(miao0913@gmail.com). I am looking forward to your reply! Thank you very much!

samylee

[wangwen39]

when i run the script :sh train_sacnn.sh it has one error like this:F0319 10:59:38.220983 11551 db_lmdb.hpp:15] Check failed: mdb_status == 0 (2 vs. 0) No such file or directory Does this mean i need to change the data format to lmdb? I just download the dataset which provided in the readmetxt. Thanks for your replay

[wangwen39]

or can you share the dataset of shanghaitech in LMDB data format ? Thank you so much !

[miao0913]

Hi,

Yes, you have to create lmdb in order to use my prototxt. But you do not necessarily need to use the lmdb format. You can choose one data format you are familiar with in Caffe, and change the corresponding data layer.

Generating lmdb data is a very basic operation in Caffe, I am sure you can find plenty of resources online.

Cheers

2018-03-19 4:07 GMT+01:00 wangwen notifications@github.com:

when i run the script :sh train_sacnn.sh it has one error like this:F0319 10:59:38.220983 11551 db_lmdb.hpp:15] Check failed: mdb_status == 0 (2 vs. 0) No such file or directory Does this mean i need to change the data format to lmdb? I just download the dataset which provided in the readmetxt. Thanks for your replay

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/miao0913/SaCNN-CrowdCounting-Tencent_Youtu/issues/2#issuecomment-374088581, or mute the thread https://github.com/notifications/unsubscribe-auth/APmD0R-xUienOLsqpJAFQEiy6rDQcSaZks5tfyEBgaJpZM4SoLHs .

-- Dr. Miaojing Shi Research Scientist LINKMEDIA - Inria Rennes Email: miaojing.shi@inria.fr CC: miao0913@gmail.com

Web: https://sites.google.com/site/miaojingshi/

[wangwen39]

Actually i need to generate the density map for the groundtruth and then transform the density map into lmdb format , am I right? can you share the code of generating density map ? i am a newer of crowdcounting , thanks for you sharing sincerely! Best wishes!

[wangwen39]

i got the density map, then i try to save them as jpg image in MATLAB, but i found the resolution is different from the original images , how could you save the density map without losing accuracy? please ,thanks for your reply!

[Zhanlgu]

You'd better save the density map as .mat, not jpg.

[wangwen39]

Then transfrom the .mat to LMDB?

[wangwen39]

i want to use caffe data layer ,so i need to change the format of the input to LMDB, and i only know caffe has tool of convert_image to transfrom image to LMDB ,so how can i do this?

[miao0913]

Hi

Check out this piece of code. For the rest you have to do it yourself.

void serialize_label(cv::Mat& map, std::string& out) { //map caffe::Datum datum; datum.set_channels(1); datum.set_height(map.rows); datum.set_width(map.cols); //datum.set_float_data(std::string((char)map.data), map.cols map.rows); float ptr = map.ptr(); for (int i = 0; i < map.cols map.rows; i ++) datum.add_float_data(ptr[i]);

datum.set_encoded(false);

// serialize
datum.SerializeToString(&out);

}

2018-03-21 10:11 GMT+01:00 wangwen notifications@github.com:

i want to use caffe data layer ,so i need to change the format of the input to LMDB, and i only know caffe has tool of convert_image to transfrom image to LMDB ,so how can i do this?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/miao0913/SaCNN-CrowdCounting-Tencent_Youtu/issues/2#issuecomment-374873076, or mute the thread https://github.com/notifications/unsubscribe-auth/APmD0fP0HXWP-wuGdVhbjVOXP8n5U_u7ks5tghlEgaJpZM4SoLHs .

-- Dr. Miaojing Shi Research Scientist LINKMEDIA - Inria Rennes Email: miaojing.shi@inria.fr CC: miao0913@gmail.com

Web: https://sites.google.com/site/miaojingshi/

[wangwen39]

Thanks ,i will try it!

[ying135]

@wangwen39 Hi!Have you figured out how to convert mat to lmdb?I can't do it with this piece of code.Can u help me?Thanks!!

[wangwen39]

@ying135 no need for you to convert mat to lmdb, I added a new caffe layer, you can refer to this blog （https://blog.csdn.net/seven_first/article/details/52598810）Hope this will be helpful for you!

[ying135]

@wangwen39 Thank u very much!I will read it seriously！

[ou-zhi-hui]

@wangwen39 do you know how to convert mat or csv to lmdb? thank you！