Cannot load image, STB Reason: expected marker

[lengly]

When I try to train it on COCO:

./darknet detector train cfg/coco.data cfg/yolo.cfg weights/darknet19_448.conv.23 -gpus 0,1,2,3

I see such error

Region Avg IOU: 0.495555, Class: 0.534914, Obj: 0.116917, No Obj: 0.007393, Avg Recall: 0.468750,  count: 32
Region Avg IOU: 0.268316, Class: 0.294134, Obj: 0.117317, No Obj: 0.008529, Avg Recall: 0.222222,  count: 36
Region Avg IOU: 0.384779, Class: 0.296443, Obj: 0.101438, No Obj: 0.007914, Avg Recall: 0.343750,  count: 32
Syncing... Cannot load image "/home/walle/lyd/datasets/COCO/coco/images/train2014/COCO_train2014_000000167126.jpg"
STB Reason: expected marker
Done!
2544: 32.557770, 34.544533 avg, 0.004000 rate, 3.863461 seconds, 325632 images
Loaded: 0.000039 seconds

This error happend randomly, anyone knows how to solve it?

[RichardYinguo]

Have you solved the problem? I meet the same

[lengly]

@RichardYinguo Something wrong with that image, please delete it in the list. don't know why...

[RichardYinguo]

thanks

[jeremy-rutman]

These appear to be images that got truncated. opencv loads them fine with original dimensions, (so a check in parse.py won't prevent the problem) but the darknet code dies on them.

I found 2 examples and uploaded to https://ibb.co/njvyf8 https://ibb.co/fQ3Uno

[mhaghighat]

I faced the same issue. Is the COCO_train2014_000000167126.jpg the only image that causes this error or are there others down the road? If this is the only one, we can simply remove it from the trainvalno5k.txt.

UPDATE: Yes, it is the only damaged image. You can simply remove it from the list.

[jeremy-rutman]

any truncated image will do this due to the c code used for loading the image. You can add a check to the code eg. a try/catch, or hope there are no more truncated images in your dataset

[leicao-me]

If you take a look at the image that can’t be loaded - COCO_train2014_000000167126.jpg, you will see it’s damaged.

Solution: Download the undamaged version of the image and replace the damaged https://msvocds.blob.core.windows.net/images/262993_z.jpg

You are welcome. :)

References: Damaged Image: https://blog.csdn.net/chengyq116/article/details/80258821 Replacement: https://github.com/karpathy/neuraltalk2/issues/4

[usb1508]

Did someone solve the problem, in which the solution doesn't involve redownloading the damaged image? Because I initially used the same images to train, and it worked fine. But the next time I reset the session of VM, this particular error pops up. Is there some way this could be avoided as currently, I need to unzip the dataset every time I use a new session.

[jeremy-rutman]

You can fix the original C code which is lacking a check for this error

On Thu, Jun 27, 2019 at 10:41 AM utkarsh1508 notifications@github.com wrote:

Did someone solve the problem, in which the solution doesn't involve redownloading the damaged image? Because I initially used the same images to train, and it worked fine. But the next time I reset the session of VM, this particular error pops up. Is there some way this could be avoided as currently, I need to unzip the dataset every time I use a new session.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/pjreddie/darknet/issues/312?email_source=notifications&email_token=AA5PJQEN4QDHBRQKL46JD3TP4RVMFA5CNFSM4EDZ2AD2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODYWHXRY#issuecomment-506231751, or mute the thread https://github.com/notifications/unsubscribe-auth/AA5PJQH7E2K2QQ5JNDL3IA3P4RVMFANCNFSM4EDZ2ADQ .

[chenjiahe01]

I've work on it all day.Finally I make it!! Find "darknet/src/image.h" and open it in editor. change the function image load_image_stb(char *filename, int channels) { int w, h, c; unsigned char *data = stbi_load(filename, &w, &h, &c, channels); if (!data) { fprintf("shit!")//stderr, "Cannot load image \"%s\"\nSTB Reason: %s\n", filename, stbi_failure_reason()); exit(0); } if(channels) c = channels; int i,j,k; image im = make_image(w, h, c); for(k = 0; k < c; ++k){ for(j = 0; j < h; ++j){ for(i = 0; i < w; ++i){ int dst_index = i + w*j + w*h*k; int src_index = k + c*i + c*w*j; im.data[dst_index] = (float)data[src_index]/255.; } } } free(data); return im; } to: image load_image_stb(char *filename, int channels) { int w, h, c; unsigned char *data = stbi_load(filename, &w, &h, &c, channels); if (!data) { //fprintf("shit!")//stderr, "Cannot load image \"%s\"\nSTB Reason: %s\n", filename, stbi_failure_reason()); //exit(0); image i; return i; } if(channels) c = channels; int i,j,k; image im = make_image(w, h, c); for(k = 0; k < c; ++k){ for(j = 0; j < h; ++j){ for(i = 0; i < w; ++i){ int dst_index = i + w*j + w*h*k; int src_index = k + c*i + c*w*j; im.data[dst_index] = (float)data[src_index]/255.; } } } free(data); return im; } Now, if the image to be loaded is truncated. The load_image program will return an empty image instead of exit. Then we can useif(im.w==0) to skip the image automatically or what.

[dmitrygashnikov]

Find "darknet/src/image.h" and open it in editor. change the function

Thx for solution, will try it later today. One correction: not find those class in "darknet/src/image.h" by myself. In my version its located in "darknet/src/image.c" And reminder for thouse, who will try this solution - do not foorget to rebuild project after making changes.

P.S. Sorry for my english

[dranaivo]

I've work on it all day.Finally I make it!! Find "darknet/src/image.h" and open it in editor. change the function image load_image_stb(char *filename, int channels) { int w, h, c; unsigned char *data = stbi_load(filename, &w, &h, &c, channels); if (!data) { fprintf("shit!")//stderr, "Cannot load image \"%s\"\nSTB Reason: %s\n", filename, stbi_failure_reason()); exit(0); } if(channels) c = channels; int i,j,k; image im = make_image(w, h, c); for(k = 0; k < c; ++k){ for(j = 0; j < h; ++j){ for(i = 0; i < w; ++i){ int dst_index = i + w*j + w*h*k; int src_index = k + c*i + c*w*j; im.data[dst_index] = (float)data[src_index]/255.; } } } free(data); return im; } to: image load_image_stb(char *filename, int channels) { int w, h, c; unsigned char *data = stbi_load(filename, &w, &h, &c, channels); if (!data) { //fprintf("shit!")//stderr, "Cannot load image \"%s\"\nSTB Reason: %s\n", filename, stbi_failure_reason()); //exit(0); image i; return i; } if(channels) c = channels; int i,j,k; image im = make_image(w, h, c); for(k = 0; k < c; ++k){ for(j = 0; j < h; ++j){ for(i = 0; i < w; ++i){ int dst_index = i + w*j + w*h*k; int src_index = k + c*i + c*w*j; im.data[dst_index] = (float)data[src_index]/255.; } } } free(data); return im; } Now, if the image to be loaded is truncated. The load_image program will return an empty image instead of exit. Then we can useif(im.w==0) to skip the image automatically or what.

@chenjiahe01 As I am not really good with C, can you elaborate a bit on the use of the condition if (im.w==0) to skip the image, namely where in the file can you use it and if possible an example of snippet. Thank you!

[HtR212]

@dranaivo You can check if (im.w==0) in the function test_detector inside "darknet/examples/detector.c". You also have to modify the load_image function inside "darknet/src/image.c" to make it return out before the if statement to avoid the resizing part.