Can't run everything.sh

[benjaminjellison]

Trying to run and set it up as described in readme, I get this error when running everything.sh:

[everything] PANDA taint analysis prospective bug mining -- input /bin/ls -- logging to /home/lava/lava/file_lava/logs/bug_mining--bin-ls.log
/usr/bin/python /home/lava/lava/scripts/bug_mining.py /home/lava/lava/file_lava.json /bin/ls
command failed! exit code was 1
========== end of logfile /home/lava/lava/file_lava/logs/bug_mining--bin-ls.log: ========== 

bug_mining.py /home/lava/lava/file_lava.json /bin/ls

[bug_mining.py] Entering /home/lava/lava.

Traceback (most recent call last):
  File "/home/lava/lava/scripts/bug_mining.py", line 128, in <module>
    command_args, installdir, isoname, project["expect_prompt"], rr=qemu_use_rr)
TypeError: create_recording() takes at least 8 arguments (8 given)

I can't find the create_recording definition to see what the problem is.

[moyix]

I think this is caused by this recent commit to PANDA https://github.com/panda-re/panda/commit/87822fabedc74610d49f7eb143ba069df7da7b4f which adds an extra option to the create_recording function. I'll try to update the bug_mining.py script to match soon (you should be able to have a look at that commit to see what it needs if you want to fix it before I have time to get to it).

[moyix]

OK it was easier than I thought so I've committed what I hope is a fix (here: https://github.com/panda-re/lava/commit/1532f181248266ee5dc3f5e8ce83198174faf110). Could you check if it fixes your issue?

[benjaminjellison]

That fixes that issue, but now I get a timeout error later in the script:

[everything] PANDA taint analysis prospective bug mining -- input /bin/ls -- logging to /home/lava/lava/file_lava/logs/bug_mining--bin-ls.log
/usr/bin/python /home/lava/lava/scripts/bug_mining.py /home/lava/lava/file_lava.json /bin/ls
command failed! exit code was 1
========== end of logfile /home/lava/lava/file_lava/logs/bug_mining--bin-ls.log: ========== 

bug_mining.py /home/lava/lava/file_lava.json /bin/ls

[bug_mining.py] Entering /home/lava/lava.

[run_guest.py] Running qemu with args:

/home/lava/panda/build/i386-softmmu/qemu-system-i386 /home/lava/lava/wheezy_panda2.qcow2 -monitor unix:/tmp/tmp6PZ6vo/monitor,server,nowait -serial unix:/tmp/tmp6PZ6vo/serial,server,nowait -loadvm root -display none
QEMU 2.8.50 monitor - type 'help' for more information
(qemu)

root@debian-i386:~# Traceback (most recent call last):
  File "/home/lava/lava/scripts/bug_mining.py", line 129, in <module>
    command_args, installdir, cdrom, isoname, project["expect_prompt"], rr=qemu_use_rr)
  File "/home/lava/panda/panda/scripts/run_guest.py", line 161, in create_recording
    expect_prompt=expect_prompt, extra_args=extra_args) as qemu:
  File "/home/lava/panda/panda/scripts/run_guest.py", line 113, in __enter__
    self.console.expect(self.expect_prompt)
  File "/home/lava/panda/panda/scripts/expect.py", line 58, in expect
    raise TimeoutExpired()
expect.TimeoutExpired

I'll dig to see if I can raise the timeout time, unless it shouldn't be causing a timeout to begin with.

[moyix]

Hmm, it's possible further changes will be needed. I'll try to take a look soon. Sorry about this!

[benjaminjellison]

Thanks! For your information, I increased the timeout to 5, then 10 minutes, and it still timed out.

[benjaminjellison]

Had any chance to check this out yet? I'm at SourceFest and was hoping to use LAVA (it only lasts 3 more days). If not I can concentrate on something else, so if you could let me know one way or the other.

[moyix]

Yes, I finally got around to this today. I've pushed a couple of fixes and everything.sh now works out of the box on file again. Please give it a shot and let me know if they fix things for you!

[benjaminjellison]

It looks like it's working now. The bug injection took almost 10 minutes, is that expected? I only ask because the original timeout was 30 seconds, so I wanted to double check that minutes is a reasonable amount of time for the script to run.

[moyix]

The timeout in the script is just for running the command under PANDA; after that it does a taint analysis and then mines the taint log to construct bugs. It's expected that taint+bug mining will take around 10 minutes, yes; the amount of time is dependent on how big the program is (or more precisely, the length of the execution trace). So if you're trying to get a faster develop/test cycle, you could try using a smaller program too.

The other thing to note is that most of that time is taken up building the database of injectable bugs. If you have a look at the arguments to everything.sh, it's possible to just do the injection step on a database you've already created, which should be much faster.

[AndrewFasano]

@benjaminjellison sounds like your issue was resolved, but if you're still having any problems, we just released a new version of LAVA and things should definitely work better now. We use a submodule of PANDA now so future changes there won't break LAVA again. Let us know if you have any more issues with it!