yahoo / CaffeOnSpark

Distributed deep learning on Hadoop and Spark clusters.
Apache License 2.0
1.27k stars 358 forks source link

when run the CaffeOnSpark,i encountered a problem,it is wrong when using the opencv to decode data. #270

Closed flyfrommiwang closed 6 years ago

flyfrommiwang commented 6 years ago

hi, i use the caffeonspark,but i encountered a problem.the message is that



Corrupt JPEG data: premature end of data segment 17/07/20 16:56:42 INFO Executor: Finished task 0.0 in stage 21.0 (TID 15). 1271 bytes result sent to driver #

A fatal error has been detected by the Java Runtime Environment:

#

SIGSEGV (0xb) at pc=0x00007f8180e4e6cf, pid=21125, tid=0x00007f8124d25700

#

JRE version: Java(TM) SE Runtime Environment (8.0_121-b13) (build 1.8.0_121-b13)

Java VM: Java HotSpot(TM) 64-Bit Server VM (25.121-b13 mixed mode linux-amd64 compressed oops)

Problematic frame:

C [libopencv_highgui.so.2.4+0xa26cf]17/07/20 16:56:42 INFO BlockManagerInfo: Removed broadcast_3_piece0 on 204.204.30.182:58793 in memory (size: 73.0 B, free: 4.1 GB)

decompress_onepass+0x14f #

Failed to write core dump. Core dumps have been disabled. To enable core dumping, try "ulimit -c unlimited" before starting Java again

# [thread 140192646326016 also had an error]

An error report file with more information is saved as:

/home/cos/R-FCN/hs_err_pid21125.log

#

If you would like to submit a bug report, please visit:

http://bugreport.java.com/bugreport/crash.jsp

The crash happened outside the Java Virtual Machine in native code.

See problematic frame for where to report the bug.

#



then ,i view the source code, and i mark DEBUG.it may be wrong when using the opencv to decode my data.Source Code


if (sample_encoded) mat.decode(Mat.CV_LOAD_IMAGE_UNCHANGED) )


the source code show that it need to use "Mat.CV_LOAD_IMAGE_UNCHANGED" to decode,but it is going wrong.So what should i do ? i don't what's going on with my data.Can you give me some suggestions,thanks!

junshi15 commented 6 years ago

This line says if the dimensions are given, it will just load it as it is. If you want to load the image othewise, don't provide the dimensions.

flyfrommiwang commented 6 years ago

thank you for your answer , This issue has been solved