search

tracer-x / TracerX

TracerX Symbolic Virtual Machine
https://tracer-x.github.io/
Other
31 stars 11 forks source link

TracerX setup on Test-Comp under Ubuntu22 #401

Open rasoolmaghareh opened 11 months ago

rasoolmaghareh commented 11 months ago

We will discuss the steps and issues for running TracerX on Test-comp here.

rasoolmaghareh commented 11 months ago

Steps from @yxliang01

Note this instruction aims to recreate a very similar environment (to the extent possible as per descriptions the orgranizers have given) to test-comp23, so the steps are bit different from Rasool's previous steps which aimed to setup for the lastest development version.

  1. Ubuntu22 server edition
  2. git clone --branch=testcomp23 --depth=1 --shallow-submodules --recurse-submodules https://gitlab.com/sosy-lab/test-comp/bench-defs
  3. Follow the dockerfile at scripts/test/Dockerfile.user.2023 for the list of packages and perform apt install accordingly
  4. cd bench-defs/
  5. pip3 install --user --requirement=scripts/requirements.txt
  6. pip3 install --user ./benchexec/
  7. echo 'PATH=$PATH:$HOME/.local/bin/' >> ~/.bashrc
  8. Follow benchexec/doc/INSTALL.md section "Setting up Cgroups" and "Setting up Cgroups on Machines with systemd"
  9. sed -i 's/--zipResultFiles//g' scripts/configure.sh
  10. sudo usermod -aG benchexec id --user --name
  11. reboot and all done

Notes

  1. test-comp seems to use the server edition of Ubuntu22. Using desktop edition in theory can work, but it's known that both of them have different default settings. To the best of my knowledge, using desktop edition might be related to some of our earlier failure and they use different default kernel. So, you might need to do more if using the desktop edition.
  2. the steps above should be done using normal user (not root)
  3. requirement: cpu execution threads >= 8, RAM (not counting swap) >= 15GB, better to have storage >= 50GB
  4. To Rasool: step 9 is to resolve a potential bug in their script, however, in your instructions, you seem to not have this issue. May I confirm this (to know if I've made wrong patch to their codebase)?
  5. I've ignored the optional setup for benchexec as I work with hardware that doesn't seem to support them and it's of limited power. But they can make benchmarking more reliable, so I would recommend to try them.
  6. I recommend not to run on virtual machines since it requires correct timer/cache settings etc to make the results reliable and I'm unsure how yet. But, if you've managed to achive reliable benchmarking in VMs, it would be great if you can share.
  7. Using PPA to install benchexec is easier, but the scripts were developed for a version that seems to be unavailable on PPA. So, I recommend following the steps above to install benchexec instead. Let me know if you've any questions, XL
rasoolmaghareh commented 11 months ago

Regarding this point:

To Rasool: step 9 is to resolve a potential bug in their script, however, in your instructions, you seem to not have this issue. May I confirm this (to know if I've made wrong patch to their codebase)?

I didn't have this issue.

ArpitaDutta commented 11 months ago

Hi @yxliang01,

As per your instructions, I have installed the Ubuntu22 server version for testcomp23. While installation I have chosen the default options for the 1. Network connections, 2. Storage configurations and 3. SSH setup. I have not done any manual settings.

But now while performing the git clone --branch=testcomp23 --depth=1 --shallow-submodules --recurse-submodules https://gitlab.com/sosy-lab/test-comp/bench-defs The server is throwing the following error: fatal: unable to access 'https://gitlab.com/sosy-lab/test-comp/bench-defs/': Could not resolve host: gitlab.com

Do you have any suggestions to resolve this?

rasoolmaghareh commented 11 months ago

It looks like a network or access issue. Try to see if this link https://gitlab.com/sosy-lab/test-comp/bench-defs/ can be opened in a browser.

ArpitaDutta commented 11 months ago

Yes, it was due to the network issue.

ArpitaDutta commented 11 months ago

A small update in the instructions given by XL.

For Step 3, we need to follow the dockerfile at scrtips/test/Dockerfile.test to get the list of packages using apt install.

rasoolmaghareh commented 11 months ago

Were you able to fully install it?

ArpitaDutta commented 11 months ago

The installation is almost completed.

However, for most of the tools I am getting the following warning/error messages.

ArpitaDutta commented 11 months ago

I am trying to fix these issues with the help of XL.

ArpitaDutta commented 9 months ago

These are the installation steps for TestComp-23 locally. The instructions are taken from XL email. I have made additions to Steps 3 and 4. The Step 3 and Step 4 installations are related to dependencies and they are of utmost importance.

  1. Use Ubuntu22 server edition ( I have used Ubuntu Server 22.04.3 LTS at my end )
  2. git clone --branch=testcomp23 --depth=1 --shallow-submodules --recurse-submodules https://gitlab.com/sosy-lab/test-comp/bench-defs
  3. Follow the dockerfile at scripts/test/Dockerfile.user.2023 for the list of packages and perform apt install accordingly
  4. We need to follow the dockerfile at scrtips/test/Dockerfile.test to get the list of packages using apt install [Step 3 and 4 are most important for getting all the dependencies for the testcomp23 setup]
  5. cd bench-defs/
  6. pip3 install --user --requirement=scripts/requirements.txt
  7. pip3 install --user ./benchexec/
  8. echo 'PATH=$PATH:$HOME/.local/bin/' >> ~/.bashrc
  9. Follow benchexec/doc/INSTALL.md section "Setting up Cgroups" and "Setting up Cgroups on Machines with systemd"
  10. sed -i 's/--zipResultFiles//g' scripts/configure.sh
  11. sudo usermod -aG benchexec id --user --name
  12. reboot and all done