search

pepper-project / pequin

A system for verifying outsourced computations, and applying SNARKs. Simplified release of the main Pepper codebase.
Other
122 stars 46 forks source link

Tutorial "mm_pure_arith" doesn't work #46

Closed Eliott closed 5 years ago

Eliott commented 5 years ago

I am following the GETTINGSTARTED.md tutorial and run the command:

./pepper_compile_and_setup_V.sh mm_pure_arith mm_pure_arith.vkey mm_pure_arith.pkey

..but it fails with the following error:

+ compiling common objs

============================================
=== Compiling computation to constraints ===
============================================

+ compile apps/mm_pure_arith.c --metrics -b 0 -w 10240 -t ZAATAR -db-hash-func ggh -db-num-addresses 1024 -ram-cell-num-bits 32 -db-hash-num-bits 1216 -db-thr-num-addresses-naive 32768 -fast-ram-address-width 32 -fast-ram-word-width 64
metric_num_lines_in_sfdl mm_pure_arith 46
metric_num_lines_in_source mm_pure_arith 46
make[1]: Entering directory '/home/e/Downloads/pequin/compiler/frontend'
ant compile
Buildfile: /home/e/Downloads/pequin/compiler/frontend/build.xml

compile:

BUILD SUCCESSFUL
Total time: 0 seconds
make[1]: Leaving directory '/home/e/Downloads/pequin/compiler/frontend'
# vericomp/hash/ggh
compile: reading input: EOF
Makefile:113: recipe for target 'bin/mm_pure_arith.params' failed
make: *** [bin/mm_pure_arith.params] Error 1

==========================================
===== Running setup (key generation) =====
==========================================

./pepper_compile_and_setup_V.sh: line 35: bin/pepper_verifier_mm_pure_arith: No such file or directory

I have also tried a few of the other .c files in the "apps" folder, but they all give the same basic error.

I am running it on Ubuntu version 16.04.

What am I doing wrong? Thank you.

maxhowald commented 5 years ago

Were you able to follow all of the steps to install the dependencies here?

If not, can you post the output of any errors you get when running those commands?

Eliott commented 5 years ago

Thank you for the reply.

I followed the Debian instructions in the link and everything is installed OK. However, I wasn't able to install openjdk-7-jdk and instead had to install openjdk-8-jdk.

After running thirdparty/install_pepper_deps.sh, the following was written to std-err:

threads.c: In function ‘_papi_hwi_shutdown_global_threads’:
threads.c:461:16: warning: variable ‘our_tid’ set but not used [-Wunused-but-set-variable]
  unsigned long our_tid;
                ^
papi_preset.c: In function ‘papi_load_derived_events’:
papi_preset.c:1058:6: warning: variable ‘preset’ set but not used [-Wunused-but-set-variable]
  int preset = 0;
      ^
threads.c: In function ‘_papi_hwi_shutdown_global_threads’:
threads.c:461:16: warning: variable ‘our_tid’ set but not used [-Wunused-but-set-variable]
  unsigned long our_tid;
                ^
papi_preset.c: In function ‘papi_load_derived_events’:
papi_preset.c:1058:6: warning: variable ‘preset’ set but not used [-Wunused-but-set-variable]
  int preset = 0;
      ^
ar: creating libpapi.a
zero_omp.c: In function ‘main’:
zero_omp.c:118:6: warning: variable ‘maxthr’ set but not used [-Wunused-but-set-variable]
  int maxthr, retval;
      ^
warning: no files found matching 'examples'
warning: no files found matching 'docs'
warning: no files found matching 'bin'
warning: no files found matching '*' under directory 'docs'
warning: no files found matching '*' under directory 'examples'
warning: no previously-included files matching '*.pyc' found under directory 'cheetah'
warning: no previously-included files matching '*~' found under directory 'cheetah'
warning: no previously-included files matching '*.aux' found under directory 'cheetah'
warning: no previously-included files matching '*~' found under directory 'docs'
warning: no previously-included files matching '*.aux' found under directory 'docs'
zip_safe flag not set; analyzing archive contents...
Cheetah.ImportManager: module references __file__
Cheetah.ImportManager: module references __path__
Cheetah.Template: module references __file__
Cheetah.ImportHooks: module references __file__
Cheetah.NameMapper: module MAY be using inspect.stack
/home/e/.local/lib/python2.7/site-packages/setuptools/command/bdist_egg.py:395: UnicodeWarning: Unicode equal comparison failed to convert both arguments to Unicode - interpreting them as being unequal
  symbols = dict.fromkeys(iter_symbols(code))
Cheetah.Tests.Template: module references __file__
configure: WARNING: old version of Kyoto Cabinet was detected
HEAD is now at dc78fda... Update libff, libfqfft, gtest, xbyak submodule versions
error: patch failed: CMakeLists.txt:105
error: CMakeLists.txt: patch does not apply

I had also run:

cd buffetfsm/llvm-build ; make check-all

and got the following to std-out:

llvm[0]: Running test suite
make[1]: Entering directory '/home/e/Downloads/pequin/compiler/buffetfsm/llvm-build/test'
Making LLVM 'lit.site.cfg' file...
Making LLVM unittest 'lit.site.cfg' file...
make -C /home/e/Downloads/pequin/compiler/buffetfsm/llvm-build/test/../tools/clang/test lit.site.cfg Unit/lit.site.cfg
make[2]: Entering directory '/home/e/Downloads/pequin/compiler/buffetfsm/llvm-build/tools/clang/test'
Making Clang 'lit.site.cfg' file...
Making Clang 'Unit/lit.site.cfg' file...
make[2]: Leaving directory '/home/e/Downloads/pequin/compiler/buffetfsm/llvm-build/tools/clang/test'
make -C /home/e/Downloads/pequin/compiler/buffetfsm/llvm-build/test/../tools/clang/tools/extra/test lit.site.cfg
make[2]: Entering directory '/home/e/Downloads/pequin/compiler/buffetfsm/llvm-build/tools/clang/tools/extra/test'
Making lit.site.cfg for Clang extra tools...
make[2]: Leaving directory '/home/e/Downloads/pequin/compiler/buffetfsm/llvm-build/tools/clang/tools/extra/test'
( ulimit -t 600 ; ulimit -d 512000 ; ulimit -m 512000 ; ulimit -s 8192 ; \
  /usr/bin/python /home/e/Downloads/pequin/compiler/buffetfsm/llvm/utils/lit/lit.py -s -v . /home/e/Downloads/pequin/compiler/buffetfsm/llvm-build/test/../tools/clang/test /home/e/Downloads/pequin/compiler/buffetfsm/llvm-build/test/../tools/clang/tools/extra/test )
-- Testing: 18163 tests, 4 threads --
Testing: 0 .. 10.. 20.. 30.. 40.. 50.. 60.. 70.. 80.. 90
FAIL: LLVM-Unit :: ADT/Release+Asserts/ADTTests/APIntTest.LargeAPIntConstruction (17180 of 18163)
******************** TEST 'LLVM-Unit :: ADT/Release+Asserts/ADTTests/APIntTest.LargeAPIntConstruction' FAILED ********************
Note: Google Test filter = APIntTest.LargeAPIntConstruction
[==========] Running 1 test from 1 test case.
[----------] Global test environment set-up.
[----------] 1 test from APIntTest
[ RUN      ] APIntTest.LargeAPIntConstruction
terminate called after throwing an instance of 'std::bad_alloc'
  what():  std::bad_alloc
0  libLLVM-3.5svn.so 0x00007f19c5386c45 llvm::sys::PrintStackTrace(_IO_FILE*) + 37
1  libLLVM-3.5svn.so 0x00007f19c5387092
2  libpthread.so.0   0x00007f19c4349390
3  libc.so.6         0x00007f19c32d5428 gsignal + 56
4  libc.so.6         0x00007f19c32d702a abort + 362
5  libstdc++.so.6    0x00007f19c390f84d __gnu_cxx::__verbose_terminate_handler() + 365
6  libstdc++.so.6    0x00007f19c390d6b6
7  libstdc++.so.6    0x00007f19c390d701
8  libstdc++.so.6    0x00007f19c390d919
9  libstdc++.so.6    0x00007f19c390debc
10 libstdc++.so.6    0x00007f19c390df19 operator new[](unsigned long) + 9
11 libLLVM-3.5svn.so 0x00007f19c53477a0 llvm::APInt::initSlowCase(unsigned int, unsigned long, bool) + 48
12 ADTTests          0x000000000049c44b
13 ADTTests          0x000000000060b721 testing::Test::Run() + 785
14 ADTTests          0x000000000060d050 testing::TestInfo::Run() + 816
15 ADTTests          0x000000000060d793 testing::TestCase::Run() + 451
16 ADTTests          0x0000000000616589 testing::internal::UnitTestImpl::RunAllTests() + 1353
17 ADTTests          0x000000000061602a testing::UnitTest::Run() + 106
18 ADTTests          0x0000000000627aa6 main + 70
19 libc.so.6         0x00007f19c32c0830 __libc_start_main + 240
20 ADTTests          0x000000000042bae9 _start + 41

********************
Testing: 0 .. 10.. 20.. 30.. 40.. 50.. 60.. 70.. 80.. 90
FAIL: LLVM-Unit :: ADT/Release+Asserts/ADTTests/APIntTest.nearestLogBase2 (17200 of 18163)
******************** TEST 'LLVM-Unit :: ADT/Release+Asserts/ADTTests/APIntTest.nearestLogBase2' FAILED ********************
Note: Google Test filter = APIntTest.nearestLogBase2
[==========] Running 1 test from 1 test case.
[----------] Global test environment set-up.
[----------] 1 test from APIntTest
[ RUN      ] APIntTest.nearestLogBase2
terminate called after throwing an instance of 'std::bad_alloc'
  what():  std::bad_alloc
0  libLLVM-3.5svn.so 0x00007f6120f9fc45 llvm::sys::PrintStackTrace(_IO_FILE*) + 37
1  libLLVM-3.5svn.so 0x00007f6120fa0092
2  libpthread.so.0   0x00007f611ff62390
3  libc.so.6         0x00007f611eeee428 gsignal + 56
4  libc.so.6         0x00007f611eef002a abort + 362
5  libstdc++.so.6    0x00007f611f52884d __gnu_cxx::__verbose_terminate_handler() + 365
6  libstdc++.so.6    0x00007f611f5266b6
7  libstdc++.so.6    0x00007f611f526701
8  libstdc++.so.6    0x00007f611f526919
9  libstdc++.so.6    0x00007f611f526ebc
10 libstdc++.so.6    0x00007f611f526f19 operator new[](unsigned long) + 9
11 libLLVM-3.5svn.so 0x00007f6120f607a0 llvm::APInt::initSlowCase(unsigned int, unsigned long, bool) + 48
12 ADTTests          0x000000000049d3ae
13 ADTTests          0x000000000060b721 testing::Test::Run() + 785
14 ADTTests          0x000000000060d050 testing::TestInfo::Run() + 816
15 ADTTests          0x000000000060d793 testing::TestCase::Run() + 451
16 ADTTests          0x0000000000616589 testing::internal::UnitTestImpl::RunAllTests() + 1353
17 ADTTests          0x000000000061602a testing::UnitTest::Run() + 106
18 ADTTests          0x0000000000627aa6 main + 70
19 libc.so.6         0x00007f611eed9830 __libc_start_main + 240
20 ADTTests          0x000000000042bae9 _start + 41

********************
Testing: 0 .. 10.. 20.. 30.. 40.. 50.. 60.. 70.. 80.. 90.. 
Testing Time: 473.68s
********************
Failing Tests (2):
    LLVM-Unit :: ADT/Release+Asserts/ADTTests/APIntTest.LargeAPIntConstruction
    LLVM-Unit :: ADT/Release+Asserts/ADTTests/APIntTest.nearestLogBase2

  Expected Passes    : 17976
  Expected Failures  : 99
  Unsupported Tests  : 86
  Unexpected Failures: 2
Makefile:100: recipe for target 'check-local-all' failed
make[1]: Leaving directory '/home/e/Downloads/pequin/compiler/buffetfsm/llvm-build/test'
/home/e/Downloads/pequin/compiler/buffetfsm/llvm/Makefile.rules:1809: recipe for target 'check-all' failed

and to std-err:

lit.py: lit.cfg:201: note: using clang: '/home/e/Downloads/pequin/compiler/buffetfsm/llvm-build/Release+Asserts/bin/clang'
lit.py: lit.cfg:329: note: Did not find clang-interpreter in /home/e/Downloads/pequin/compiler/buffetfsm/llvm-build/Release+Asserts/bin:/home/e/Downloads/pequin/compiler/buffetfsm/llvm-build/Release+Asserts/bin
lit.py: discovery.py:190: warning: test suite 'Extra Tools Unit Tests' contained no tests

1 warning(s) in tests.
make[1]: *** [check-local-all] Error 1
make: *** [check-all] Error 2
maxhowald commented 5 years ago

The errors from building llvm for buffet shouldn't cause any problems for most computations, and the other build errors appear to be caused by running the install_pepper_deps.sh script twice.

Looking back at the original error, it looks like zcc is failing somewhere around here.

Can you try running these commands manually (starting from the pequin/compiler directory) and posting the output? Or, maybe just comment out the exit 1s in this block and running the original script again, since the Go code isn't actually required for the mm_pure_arith example.

Alternatively, as a workaround, you could try running Ubuntu 14.04 in a virtual machine or use the build_docker.sh script in this repo if you're familiar with Docker.

Eliott commented 5 years ago

Thank you,

Running the commands manually, the command go build vericomp worked OK, but go install vericomp/merkle/GenMerkle returned the error code 2.

Commenting out the exit 1s resulted in the following error:

...
# vericomp/hash/ggh
compile: reading input: EOF
Error: A JNI error has occurred, please check your installation and try again
Exception in thread "main" java.lang.ClassFormatError: Truncated class file
at java.lang.ClassLoader.defineClass1(Native Method)
at java.lang.ClassLoader.defineClass(ClassLoader.java:763)
at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
at java.net.URLClassLoader.defineClass(URLClassLoader.java:468)
at java.net.URLClassLoader.access$100(URLClassLoader.java:74)
at java.net.URLClassLoader$1.run(URLClassLoader.java:369)
at java.net.URLClassLoader$1.run(URLClassLoader.java:363)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:362)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
at sun.launcher.LauncherHelper.checkAndLoadMain(LauncherHelper.java:495)
Makefile:113: recipe for target 'bin/mm_pure_arith.params' failed
make: *** [bin/mm_pure_arith.params] Error 1
...

This error occurs on line 246 of zcc. And running the "No optimize" else-branch resulted in the same error but on line 353.

The values of the variables when executing line 246 are:

Variable Value
$TIMER /usr/bin/time -o tmptime -f
$classname mm_pure_arith
$javaargs -Xmx1400M
$compileropts --verbose --bigconstraints --seed=12345 -OZAATAR -B10240 --defs mm_pure_arith.c.defines
$inlining m
$path mm_pure_arith.c

Thank you again Max for the support.

Eliott commented 5 years ago

Thank you Max for the help. I have it working now...

I tried 14.04 in a VM and was having trouble getting it to work there too, but after playing around a bit, I found it was because I didn't have pkg-config installed (which caused install_pepper_deps.sh to fail around here). After installing pkg-config, everything worked on the VM.

I went back to 16.04 and explored the java.lang.ClassFormatError: Truncated class file error some more and found that the frontend/bin/zcc/ZCC.class file was actually empty. I deleted the bin and gensrc folders and recompiled the frontend and got the class-file to generate (I don't know why it didn't compile correctly the first time).

I then got another error, the pepper/obj/libv/computation_p.o was causing problems, so I deleted the obj folder and then, finally, re-running ./pepper_compile_and_setup_V.sh worked!

So I don't know what prevented these files from compiling correctly the first time, but it must have been something I done wrong. Anyway, it's all working for me now!

Thank you again for the support.

maxhowald commented 5 years ago

Glad you got it working! And thanks for the info.

Removing the files was good thinking, but you can also try running make clean-all in the pepper directory to blow away everything, in case you run in to similar problems in the future.