Error when trying to use other benchmarks

[mlulaj]

Hello, I am trying to use this tool to fuzz other designs. I already run it for the given benchmarks and the results are really satisfiable . What I have to do now, is use this tool , to verify the coverage of AES CIPHER Core, which is provided in both VHDL and Verilog. My first Issue was to integrate this design to this tool, given that the RFUZZ needs a (.fir) file as input, so the intermediate representation . I already read on the official page of FIRRTL that there is a synthesizer (Yosys) that converts from Verilog to FirRtl. I converted my design to FirRtl but it gives me an error when I try to instrument it : make run cd instrumentation ; sbt -ivy /home/maku/Desktop/Fuzzing/rfuzz/.ivy2 "runMain hardwareafl.firrtltransforms.CustomTop -i /home/maku/Desktop/Fuzzing/rfuzz/benchmarks/Top_PipelinedCipher.fir -o /home/maku/Desktop/Fuzzing/rfuzz/build/Top_PipelinedCipher.v -X verilog -ll info -fct hardwareafl.firrtltransforms.NoDedupTransform,hardwareafl.firrtltransforms.ReplaceMemsTransform,hardwareafl.firrtltransforms.SplitMuxConditions,hardwareafl.firrtltransforms.ProfilingTransform,firrtl.passes.wiring.WiringTransform,hardwareafl.firrtltransforms.AddMetaResetTransform " [info] Loading settings from plugins.sbt ... [info] Loading project definition from /home/maku/Desktop/Fuzzing/rfuzz/instrumentation/project [info] Loading settings from build.sbt ... [info] Set current project to instrumentation (in build file:/home/maku/Desktop/Fuzzing/rfuzz/instrumentation/) [info] Running hardwareafl.firrtltransforms.CustomTop -i /home/maku/Desktop/Fuzzing/rfuzz/benchmarks/Top_PipelinedCipher.fir -o /home/maku/Desktop/Fuzzing/rfuzz/build/Top_PipelinedCipher.v -X verilog -ll info -fct hardwareafl.firrtltransforms.NoDedupTransform,hardwareafl.firrtltransforms.ReplaceMemsTransform,hardwareafl.firrtltransforms.SplitMuxConditions,hardwareafl.firrtltransforms.ProfilingTransform,firrtl.passes.wiring.WiringTransform,hardwareafl.firrtltransforms.AddMetaResetTransform [error] (run-main-0) java.lang.StackOverflowError [error] java.lang.StackOverflowError [error] at java.util.regex.Pattern$5.isSatisfiedBy(Pattern.java:5253) [error] at java.util.regex.Pattern$5.isSatisfiedBy(Pattern.java:5253) [error] at java.util.regex.Pattern$CharProperty.match(Pattern.java:3778) [error] at java.util.regex.Pattern$Curly.match0(Pattern.java:4252) [error] at java.util.regex.Pattern$Curly.match(Pattern.java:4236) [error] at java.util.regex.Pattern$Ques.match(Pattern.java:4184) [error] at java.util.regex.Pattern$GroupHead.match(Pattern.java:4660) [error] at java.util.regex.Pattern$BmpCharProperty.match(Pattern.java:3800) [error] at java.util.regex.Pattern$Curly.match0(Pattern.java:4274) [error] at java.util.regex.Pattern$Curly.match(Pattern.java:4236) [error] at java.util.regex.Matcher.match(Matcher.java:1270) [error] at java.util.regex.Matcher.matches(Matcher.java:604)

I dont know if the problem is of synthesizer, which is not converting good the verilog file. I tried It also with small designs, but the result is the same. Maybe I should modify something to the Rfuzz ?

Thank you

[ekiwi]

Hi @mlulaj ,

thanks for trying out rfuzz! Would you be able to share the .fir file + maybe the original Verilog with me? There might be a bug in our code that was never exercised by our benchmarks.

Kevin

[mlulaj]

@ekiwi thank you very much for your reply. Should I send the files via email, because here I can not upload files. It doesnt allow me

[scanakci]

Hi @ekiwi,

Similar to this issue, I also wanted to fuzz some RTLs written in verilog. I am not familiar with FIRRTL or chisel. However, I generated a FIRRTL representation of some basic designs using YOSYS.

I moved the *.fir file into benchmarks folder and make FIR=benchmarkname.fir DUT=benchmarkname run. Unfortunately, I encounter an error:

[info] running pynq.HarnessGenerator /root/rfuzz/build/examplemodule_InstrumentationInfo.toml /root/rfuzz/build/examplemodule.toml /root/rfuzz/build/examplemodule.e2e.toml inToml: /root/rfuzz/build/examplemodule_InstrumentationInfo.toml [error] scala.MatchError: Left((List(general, coverage),Could not resolve value)) (of class scala.util.Left)

I attached the full stack trace. Also, you can find the Verilog and FIRRTL files in the attachment as well.

rfuzz_files.zip

Let me know if you need more information,

Thanks!

[ekiwi]

Let me know if you need more information,

Thanks, I hope I fill find time to look into this. Feel free to ping me on this issue once in a while if I do not respond. I am a bit busy, but still very interested in trying to get this working for you.

[ekiwi]

Hi @scanakci ,

I updated the main branch of this repository to work with the latest stable versions of Chisel and Firrtl (no more submodules!). I also fixed some bugs, were a failure to parse the FileInfo (which is useful for debugging) lead to the whole process being terminated. Now such a failure just leads to an output of ??? in the generated TOML.

There is still at least one remaining problem which prevents the gates.fir file from working in rfuzz: In gates.fir there are no mux(cond, a, b) statements and thus there is nothing to instrument coverage for. I never considered this case when I created rfuzz together with my co-authors and there is quite a bit of code that makes assumptions about there being at least one covered signal.

Can you try out the latest master branch with your generated files that have at least one mux in them?

Thanks for the helpeful bug report and attaching all the files I needed to reproduce!

Kevin

[scanakci]

Hi @ekiwi,

Thank you so much for you quick response and looking into this issue. It is my bad to provide you with an example that does not include a MUX :).

I will try it ASAP (either today or tomorrow) and let you know.

Thanks,

Sadullah

[scanakci]

Hi @ekiwi ,

I just tried the recent version. I am currently getting errors when I am trying already available benchmarks. In a fresh container (Ubuntu 18.04), I followed the steps in README file and make run gave me this:

======== Starting Transform firrtl.passes.CheckChirrtl$ ======== [error] (run-main-0) firrtl.passes.CheckHighFormLike$ResetInputException: : [module SparseMem] Abstract Reset not allowed as top-level input: reset [error] firrtl.passes.CheckHighFormLike$ResetInputException: : [module SparseMem] Abstract Reset not allowed as top-level input: reset [error] stack trace is suppressed; run last Compile / bgRunMain for the full output [error] Nonzero exit code: 1 [error] (Compile / runMain) Nonzero exit code: 1 [error] Total time: 3 s, completed Jul 5, 2020 8:32:05 PM Makefile:62: recipe for target '/root/rfuzz/build/Sodor3Stage_InstrumentationInfo.toml' failed make: *** [/root/rfuzz/build/Sodor3Stage_InstrumentationInfo.toml] Error 1

Please let me know if I need to perform some additional steps after your recent changes.

[ekiwi]

[error] (run-main-0) firrtl.passes.CheckHighFormLike$ResetInputException: : [module SparseMem] Abstract Reset not allowed as top-level input: reset

Yeah, this is a bug in the pass that replaces memories with ones that can easily be reset and related to the new abstract Reset type in the latest chisel version. I am going to look into this later tonight (hopefully).

[ekiwi]

@scanakci I fixed the "Abstract Reset" issue.

[scanakci]

@ekiwi thank you so much for allocating time for this issue. Your recent commit fixed the Abstract Reset issue. Also, I generated a FIRRTL file using YOSYS and rfuzz can generate the necessary files without having compilation issues. Now, by looking at the runtime output, it looks like that rfuzz can successfully fuzz the RTL that I obtained with YOSYS. I will try more complicated examples during this week but it looks like that the issue is solved :+1:

[ekiwi]

I will try more complicated examples during this week but it looks like that the issue is solved

Sounds great! Keep me posted.

If you want to modify rfuzz and need some help, we can set up a call if you want to. Just reach out to me through my Berkeley email.