Assertion Failure in Sam2p at rule.cpp:514

[Ngiong]

Hello, We are currently working on a new fuzz testing feature, and we found a crash in sam2p.

Step to Reproduce We used CXXFLAGS="-g -O0" ./configure to configure sam2p, and build using make, and run it with:

./sam2p -c:zip -pdf:0 -pdf:b2 -c:zip PNM: -pdf:0 -3 <attached_file> EPS:

Attachment: poc_0005.txt

Environment

• OS: Ubuntu 18.04.4 LTS
• Compiler: gcc 7.5.0
• sam2p commit version: b953f63

Additional context Here's the stack trace: stack_trace_0005.txt

Program received signal SIGABRT, Aborted.
__GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:51
51  ../sysdeps/unix/sysv/linux/raise.c: No such file or directory.
#0  __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:51
#1  0x00007ffff769b8b1 in __GI_abort () at abort.c:79
#2  0x00007ffff768b42a in __assert_fail_base (fmt=0x7ffff7812a38 "%s%s%s:%u: %s%sAssertion `%s' failed.\n%n", assertion=assertion@entry=0x5555555b3c68 "cacheHints.PredictorBPC!=0 && \"doSampleFormat already called\"", file=file@entry=0x5555555b3c5d "rule.cpp", line=line@entry=514, function=function@entry=0x5555555b4740 <Rule::OutputRule::appendDecoderSpec(GenBuffer::Writable&) const::__PRETTY_FUNCTION__> "void Rule::OutputRule::appendDecoderSpec(GenBuffer::Writable&) const") at assert.c:92
#3  0x00007ffff768b4a2 in __GI___assert_fail (assertion=0x5555555b3c68 "cacheHints.PredictorBPC!=0 && \"doSampleFormat already called\"", file=0x5555555b3c5d "rule.cpp", line=514, function=0x5555555b4740 <Rule::OutputRule::appendDecoderSpec(GenBuffer::Writable&) const::__PRETTY_FUNCTION__> "void Rule::OutputRule::appendDecoderSpec(GenBuffer::Writable&) const") at assert.c:101
#4  0x000055555557a9d8 in Rule::OutputRule::appendDecoderSpec (this=0x5555557fd820, out=...) at rule.cpp:514
#5  0x000055555557c1fa in Rule::writeTTE (out=..., outpal=..., outstream=..., template_=0x5555557ea058 " getinterval\n/F T`F def\n/m 0 def\n{255 div m 1 add dup 3 eq{pop setrgbcolor\n`w `h false[1 0 0 -1 0 `h]F imagemask\n0}if/m exch def}forall\n`t}\n%%BeginData:;\nexec\n`S\n%%EndData\nend restore showpage\n%%Trail"..., or_=0x5555557fd820, sf=0x7fffffffdcc0, stream_writer=0x55555557b7d6 <Rule::writePalData(GenBuffer::Writable&, GenBuffer::Writable&, Image::SampledInfo*)>, strings=0x7fffffffdac0) at rule.cpp:888
#6  0x000055555557d864 in Rule::writeTTT (out=..., outpal=..., outstream=..., template_key=0x5555555af520 "l23tran2", or_=0x5555557fd820, sf=0x7fffffffdcc0, stream_writer=0x55555557b7d6 <Rule::writePalData(GenBuffer::Writable&, GenBuffer::Writable&, Image::SampledInfo*)>, strings=0x7fffffffdac0) at rule.cpp:1093
#7  0x000055555556161a in out_l23_work (out=..., or_=0x5555557fd820, sf=0x7fffffffdcc0) at appliers.cpp:326
#8  0x000055555557b52d in Rule::applyProfile (out=..., rule_list=0x5555557fd520, sf=0x7fffffffdcc0) at rule.cpp:680
#9  0x000055555555fe93 in run_sam2p_engine (sout=..., serr=..., argv1=0x7fffffffdeb0, helpp=false) at sam2p_main.cpp:1079
#10 0x0000555555560512 in main (argv=0x7fffffffdea8) at sam2p_main.cpp:1148

Thank you.