search

vmware-archive / cascade

A Just-In-Time Compiler for Verilog from VMware Research
Other
433 stars 44 forks source link

Most regression tests fail on FreeBSD #211

Closed yurivict closed 4 years ago

yurivict commented 4 years ago 
[==========] Running 294 tests from 16 test suites.
[----------] Global test environment set-up.
[----------] 4 tests from array
[ RUN      ] array.run_2
/usr/ports/cad/cascade-compiler/work/cascade-3f0c4988aacc8bde4b2661c6666c44eeb3024573/test/harness.cc:83: Failure
Value of: c.bad()
  Actual: true
Expected: false
[  FAILED  ] array.run_2 (6 ms)
[ RUN      ] array.run_3
/usr/ports/cad/cascade-compiler/work/cascade-3f0c4988aacc8bde4b2661c6666c44eeb3024573/test/harness.cc:83: Failure
Value of: c.bad()
  Actual: true
Expected: false
[  FAILED  ] array.run_3 (6 ms)
[ RUN      ] array.run_4
/usr/ports/cad/cascade-compiler/work/cascade-3f0c4988aacc8bde4b2661c6666c44eeb3024573/test/harness.cc:83: Failure
Value of: c.bad()
  Actual: true
Expected: false
[  FAILED  ] array.run_4 (6 ms)

Some succeed:

[       OK ] type_check.fail_array_1 (5 ms)
[ RUN      ] type_check.fail_array_2
[       OK ] type_check.fail_array_2 (5 ms)
[ RUN      ] type_check.fail_array_3
[       OK ] type_check.fail_array_3 (6 ms)
[ RUN      ] type_check.fail_array_4
[       OK ] type_check.fail_array_4 (5 ms)
[ RUN      ] type_check.fail_array_5
[       OK ] type_check.fail_array_5 (5 ms)
[ RUN      ] type_check.fail_array_6
[       OK ] type_check.fail_array_6 (5 ms)
eschkufz commented 4 years ago

Well... this isn't saying much... those type_check.fail_xxx tests only pass because they're expected to fail.

I added some code to the last pull request to print out cascade's error stream whenever a test fails. Can you try rerunning your regression tests to see what the errors look like?

yurivict commented 4 years ago

Now the log looks like this:

[==========] Running 294 tests from 16 test suites.
[----------] Global test environment set-up.
[----------] 4 tests from array
[ RUN      ] array.run_2
Parse Error:
  > Unable to locate file share/cascade/march/regression/minimal.v
  > In final line of user input:
    syntax error, unexpected <unparseable>
/usr/ports/cad/cascade-compiler/work/cascade-5a27bf54d8b9d6e708192d59d99623f428a603b4/test/harness.cc:87: Failure
Value of: c.bad()
  Actual: true
Expected: false
[  FAILED  ] array.run_2 (6 ms)
[ RUN      ] array.run_3
Parse Error:
  > Unable to locate file share/cascade/march/regression/minimal.v
  > In final line of user input:
    syntax error, unexpected <unparseable>
/usr/ports/cad/cascade-compiler/work/cascade-5a27bf54d8b9d6e708192d59d99623f428a603b4/test/harness.cc:87: Failure
Value of: c.bad()
  Actual: true
Expected: false
[  FAILED  ] array.run_3 (7 ms)
[ RUN      ] array.run_4
Parse Error:
  > Unable to locate file share/cascade/march/regression/minimal.v
  > In final line of user input:
    syntax error, unexpected <unparseable>
/usr/ports/cad/cascade-compiler/work/cascade-5a27bf54d8b9d6e708192d59d99623f428a603b4/test/harness.cc:87: Failure
Value of: c.bad()
  Actual: true
Expected: false
[  FAILED  ] array.run_4 (6 ms)
yurivict commented 4 years ago

I think your file search is still broken.

yurivict commented 4 years ago

I used out-of-source build in /usr/ports/cad/cascade-compiler/work/.build, executable run_regression is located there, but installation was done into /usr/ports/cad/cascade-compiler/work/stage/usr/local/...

yurivict commented 4 years ago

It looks for share/cascade/test/regression/bitcoin/bitcoin_2.v, didn't you just remove it from the installation? -)

eschkufz commented 4 years ago

Yeah, I just removed the test and benchmark files from the installation. The regression tests will only work if you run them the cascade source directory since they'll all be looking for the share/ directory relative to the current working directory.

yurivict commented 4 years ago

You need to make regression tests to look for these files in the build directory.

eschkufz commented 4 years ago

Gotcha. Let me see what I can do about that.

eschkufz commented 4 years ago

Try the most recent commit to issue-211. I updated cmake to copy the share/cascade/ directory to the build directory. If you run make regression from inside your build directory it should work, regardless of where your build directory is relative to your source directory.

yurivict commented 4 years ago

Now regression target test fails when run as a target:

    Start 1: regression
1/1 Test #1: regression .......................***Timeout 1500.02 sec

0% tests passed, 1 tests failed out of 1

Total Test time (real) = 1500.03 sec

The following tests FAILED:
      1 - regression (Timeout)

but ./test/run_regression succeeds:

<...skipped...>
[----------] 5 tests from verilator64
[ RUN      ] verilator64.array
[       OK ] verilator64.array (1664 ms)
[ RUN      ] verilator64.bitcoin
[       OK ] verilator64.bitcoin (24736 ms)
[ RUN      ] verilator64.mips32
[       OK ] verilator64.mips32 (1788 ms)
[ RUN      ] verilator64.nw
[       OK ] verilator64.nw (137 ms)
[ RUN      ] verilator64.regex
[       OK ] verilator64.regex (914 ms)
[----------] 5 tests from verilator64 (29239 ms total)

[----------] Global test environment tear-down
[==========] 294 tests from 16 test suites ran. (184993 ms total)
[  PASSED  ] 294 tests.
eschkufz commented 4 years ago

That sounds like progress. Can you show me the command lines you're using that get you from a fresh checkout to this error? I want to try duplicating your directory structure to see if I can reproduce on my end.

yurivict commented 4 years ago

configure is done with

/bin/mkdir -p /usr/ports/cad/cascade-compiler/work/.build
cd /usr/ports/cad/cascade-compiler/work/.build; /usr/bin/env ac_cv_path_PERL=/usr/local/bin/perl ac_cv_path_PERL_PATH=/usr/local/bin/perl  PERL_USE_UNSAFE_INC=1 XDG_DATA_HOME=/usr/ports/cad/cascade-compiler/work  XDG_CONFIG_HOME=/usr/ports/cad/cascade-compiler/work  HOME=/usr/ports/cad/cascade-compiler/work PATH=/usr/ports/cad/cascade-compiler/work/.bin:/home/yuri/bin:/sbin:/bin:/usr/sbin:/usr/bin:/usr/local/sbin:/usr/local/bin SHELL=/bin/sh CONFIG_SHELL=/bin/sh CMAKE_PREFIX_PATH="/usr/local" /usr/local/bin/cmake -DCMAKE_C_COMPILER:STRING="cc"  -DCMAKE_CXX_COMPILER:STRING="c++"  -DCMAKE_C_FLAGS:STRING="-O2 -pipe -fno-omit-frame-pointer  -fstack-protector-strong -isystem /usr/local/include -fno-strict-aliasing "  -DCMAKE_C_FLAGS_DEBUG:STRING="-O2 -pipe -fno-omit-frame-pointer  -fstack-protector-strong -isystem /usr/local/include -fno-strict-aliasing "  -DCMAKE_C_FLAGS_RELEASE:STRING="-O2 -pipe -fno-omit-frame-pointer  -fstack-protector-strong -isystem /usr/local/include -fno-strict-aliasing "  -DCMAKE_CXX_FLAGS:STRING="-O2 -pipe -fno-omit-frame-pointer -fstack-protector-strong -isystem /usr/local/include -fno-strict-aliasing -fno-omit-frame-pointer  -isystem /usr/local/include "  -DCMAKE_CXX_FLAGS_DEBUG:STRING="-O2 -pipe -fno-omit-frame-pointer -fstack-protector-strong -isystem /usr/local/include -fno-strict-aliasing -fno-omit-frame-pointer  -isystem /usr/local/include "  -DCMAKE_CXX_FLAGS_RELEASE:STRING="-O2 -pipe -fno-omit-frame-pointer -fstack-protector-strong -isystem /usr/local/include -fno-strict-aliasing -fno-omit-frame-pointer  -isystem /usr/local/include "  -DCMAKE_EXE_LINKER_FLAGS:STRING=" -Wl,-rpath=/usr/local/lib -fstack-protector-strong -L/usr/local/lib "  -DCMAKE_MODULE_LINKER_FLAGS:STRING=" -Wl,-rpath=/usr/local/lib -fstack-protector-strong -L/usr/local/lib "  -DCMAKE_SHARED_LINKER_FLAGS:STRING=" -Wl,-rpath=/usr/local/lib -fstack-protector-strong -L/usr/local/lib "  -DCMAKE_INSTALL_PREFIX:PATH="/usr/local"  -DCMAKE_BUILD_TYPE:STRING="Release"  -DTHREADS_HAVE_PTHREAD_ARG:BOOL=YES  -DCMAKE_INSTALL_RPATH_USE_LINK_PATH=YES  -DCMAKE_VERBOSE_MAKEFILE:BOOL=ON -DBUILD_TESTING:BOOL=OFF -GNinja /usr/ports/cad/cascade-compiler/work/cascade-25e1050ab41c242c77014d7436998611e0b3fb82

Build is done with 

(cd /usr/ports/cad/cascade-compiler/work/.build; if ! /usr/bin/env FREEBSD_WRKSRC=/usr/ports/cad/cascade-compiler/work/cascade-25e1050ab41c242c77014d7436998611e0b3fb82 FREEBSD_LOCALBASE=/usr/local PERL_USE_UNSAFE_INC=1 XDG_DATA_HOME=/usr/ports/cad/cascade-compiler/work  XDG_CONFIG_HOME=/usr/ports/cad/cascade-compiler/work  HOME=/usr/ports/cad/cascade-compiler/work PATH=/usr/ports/cad/cascade-compiler/work/.bin:/home/yuri/bin:/sbin:/bin:/usr/sbin:/usr/bin:/usr/local/sbin:/usr/local/bin NO_PIE=yes MK_DEBUG_FILES=no MK_KERNEL_SYMBOLS=no SHELL=/bin/sh NO_LINT=YES DESTDIR=/usr/ports/cad/cascade-compiler/work/stage PREFIX=/usr/local  LOCALBASE=/usr/local  CC="cc" CFLAGS="-O2 -pipe -fno-omit-frame-pointer  -fstack-protector-strong -isystem /usr/local/include -fno-strict-aliasing "  CPP="cpp" CPPFLAGS="-fno-omit-frame-pointer -isystem /usr/local/include"  LDFLAGS=" -Wl,-rpath=/usr/local/lib -fstack-protector-strong -L/usr/local/lib " LIBS=""  CXX="c++" CXXFLAGS="-O2 -pipe -fno-omit-frame-pointer -fstack-protector-strong -isystem /usr/local/include -fno-strict-aliasing -fno-omit-frame-pointer  -isystem /usr/local/include "  MANPREFIX="/usr/local" BSD_INSTALL_PROGRAM="install  -s -m 555"  BSD_INSTALL_LIB="install  -s -m 0644"  BSD_INSTALL_SCRIPT="install  -m 555"  BSD_INSTALL_DATA="install  -m 0644"  BSD_INSTALL_MAN="install  -m 444" ninja   -j8 -v all; then  if [ -n "Try to set MAKE_JOBS_UNSAFE=yes and rebuild before reporting the failure to the maintainer." ] ; then  echo "===> Compilation failed unexpectedly.";  (echo "Try to set MAKE_JOBS_UNSAFE=yes and rebuild before reporting the failure to the maintainer.") | /usr/bin/fmt 75 79 ;  fi;  false;  fi)

test is done with

(cd /usr/ports/devel/benchmark/work/.build; if ! /usr/bin/env XDG_DATA_HOME=/usr/ports/devel/benchmark/work  XDG_CONFIG_HOME=/usr/ports/devel/benchmark/work  HOME=/usr/ports/devel/benchmark/work PATH=/usr/ports/devel/benchmark/work/.bin:/home/yuri/bin:/sbin:/bin:/usr/sbin:/usr/bin:/usr/local/sbin:/usr/local/bin NO_PIE=yes MK_DEBUG_FILES=no MK_KERNEL_SYMBOLS=no SHELL=/bin/sh NO_LINT=YES DESTDIR=/usr/ports/devel/benchmark/work/stage PREFIX=/usr/local  LOCALBASE=/usr/local  CC="cc" CFLAGS="-O2 -pipe -fno-omit-frame-pointer  -fstack-protector-strong -fno-strict-aliasing "  CPP="cpp" CPPFLAGS="-fno-omit-frame-pointer"  LDFLAGS=" -fstack-protector-strong " LIBS=""  CXX="c++" CXXFLAGS="-O2 -pipe -fno-omit-frame-pointer -fstack-protector-strong -fno-strict-aliasing -fno-omit-frame-pointer  -Wno-thread-safety-analysis -Wno-sign-compare "  MANPREFIX="/usr/local" BSD_INSTALL_PROGRAM="install  -s -m 555"  BSD_INSTALL_LIB="install  -s -m 0644"  BSD_INSTALL_SCRIPT="install  -m 555"  BSD_INSTALL_DATA="install  -m 0644"  BSD_INSTALL_MAN="install  -m 444" ninja   -j8 -v all; then  if [ -n "Try to set MAKE_JOBS_UNSAFE=yes and rebuild before reporting the failure to the maintainer." ] ; then  echo "===> Compilation failed unexpectedly.";  (echo "Try to set MAKE_JOBS_UNSAFE=yes and rebuild before reporting the failure to the maintainer.") | /usr/bin/fmt 75 79 ;  fi;  false;  fi)

Sorry, these commands are auto-generated by the framework, and are very long.

eschkufz commented 4 years ago

No worries, this is really helpful.

I think I know what's going on. Cascade is resolving include directives correctly, but some of our regression tests use the $fopen() system task, which expects a path. So you can run the regression tests from anywhere and the includes will work as expected but the fopens will only work if you run the tests from <path/to/build> (that's what our Makefile does, but it looks like it's not what your build chain does). I'll add a feature similar to include path resolution for fopen and we should be all set.

On Sat, Jan 4, 2020 at 7:52 PM yuri@FreeBSD notifications@github.com wrote:

configure is done with

/bin/mkdir -p /usr/ports/cad/cascade-compiler/work/.build cd /usr/ports/cad/cascade-compiler/work/.build; /usr/bin/env ac_cv_path_PERL=/usr/local/bin/perl ac_cv_path_PERL_PATH=/usr/local/bin/perl PERL_USE_UNSAFE_INC=1 XDG_DATA_HOME=/usr/ports/cad/cascade-compiler/work XDG_CONFIG_HOME=/usr/ports/cad/cascade-compiler/work HOME=/usr/ports/cad/cascade-compiler/work PATH=/usr/ports/cad/cascade-compiler/work/.bin:/home/yuri/bin:/sbin:/bin:/usr/sbin:/usr/bin:/usr/local/sbin:/usr/local/bin SHELL=/bin/sh CONFIG_SHELL=/bin/sh CMAKE_PREFIX_PATH="/usr/local" /usr/local/bin/cmake -DCMAKE_C_COMPILER:STRING="cc" -DCMAKE_CXX_COMPILER:STRING="c++" -DCMAKE_C_FLAGS:STRING="-O2 -pipe -fno-omit-frame-pointer -fstack-protector-strong -isystem /usr/local/include -fno-strict-aliasing " -DCMAKE_C_FLAGS_DEBUG:STRING="-O2 -pipe -fno-omit-frame-pointer -fstack-protector-strong -isystem /usr/local/include -fno-strict-aliasing " -DCMAKE_C_FLAGS_RELEASE:STRING="-O2 -pipe -fno-omit-frame-pointer -fstack-protector-strong -isystem /usr/local/include -fno-strict-aliasing " -DCMAKE_CXX_FLAGS:STRING="-O2 -pipe -fno-omit-frame-pointer -fstack-protector-strong -isystem /usr/local/include -fno-strict-aliasing -fno-omit-frame-pointer -isystem /usr/local/include " -DCMAKE_CXX_FLAGS_DEBUG:STRING="-O2 -pipe -fno-omit-frame-pointer -fstack-protector-strong -isystem /usr/local/include -fno-strict-aliasing -fno-omit-frame-pointer -isystem /usr/local/include " -DCMAKE_CXX_FLAGS_RELEASE:STRING="-O2 -pipe -fno-omit-frame-pointer -fstack-protector-strong -isystem /usr/local/include -fno-strict-aliasing -fno-omit-frame-pointer -isystem /usr/local/include " -DCMAKE_EXE_LINKER_FLAGS:STRING=" -Wl,-rpath=/usr/local/lib -fstack-protector-strong -L/usr/local/lib " -DCMAKE_MODULE_LINKER_FLAGS:STRING=" -Wl,-rpath=/usr/local/lib -fstack-protector-strong -L/usr/local/lib " -DCMAKE_SHARED_LINKER_FLAGS:STRING=" -Wl,-rpath=/usr/local/lib -fstack-protector-strong -L/usr/local/lib " -DCMAKE_INSTALL_PREFIX:PATH="/usr/local" -DCMAKE_BUILD_TYPE:STRING="Release" -DTHREADS_HAVE_PTHREAD_ARG:BOOL=YES -DCMAKE_INSTALL_RPATH_USE_LINK_PATH=YES -DCMAKE_VERBOSE_MAKEFILE:BOOL=ON -DBUILD_TESTING:BOOL=OFF -GNinja /usr/ports/cad/cascade-compiler/work/cascade-25e1050ab41c242c77014d7436998611e0b3fb82

Build is done with

(cd /usr/ports/cad/cascade-compiler/work/.build; if ! /usr/bin/env FREEBSD_WRKSRC=/usr/ports/cad/cascade-compiler/work/cascade-25e1050ab41c242c77014d7436998611e0b3fb82 FREEBSD_LOCALBASE=/usr/local PERL_USE_UNSAFE_INC=1 XDG_DATA_HOME=/usr/ports/cad/cascade-compiler/work XDG_CONFIG_HOME=/usr/ports/cad/cascade-compiler/work HOME=/usr/ports/cad/cascade-compiler/work PATH=/usr/ports/cad/cascade-compiler/work/.bin:/home/yuri/bin:/sbin:/bin:/usr/sbin:/usr/bin:/usr/local/sbin:/usr/local/bin NO_PIE=yes MK_DEBUG_FILES=no MK_KERNEL_SYMBOLS=no SHELL=/bin/sh NO_LINT=YES DESTDIR=/usr/ports/cad/cascade-compiler/work/stage PREFIX=/usr/local LOCALBASE=/usr/local CC="cc" CFLAGS="-O2 -pipe -fno-omit-frame-pointer -fstack-protector-strong -isystem /usr/local/include -fno-strict-aliasing " CPP="cpp" CPPFLAGS="-fno-omit-frame-pointer -isystem /usr/local/include" LDFLAGS=" -Wl,-rpath=/usr/local/lib -fstack-protector-strong -L/usr/local/lib " LIBS="" CXX="c++" CXXFLAGS="-O2 -pipe -fno-omit-frame-pointer -fstack-protector-strong -isystem /usr/local/include -fno-strict-aliasing -fno-omit-frame-pointer -isystem /usr/local/include " MANPREFIX="/usr/local" BSD_INSTALL_PROGRAM="install -s -m 555" BSD_INSTALL_LIB="install -s -m 0644" BSD_INSTALL_SCRIPT="install -m 555" BSD_INSTALL_DATA="install -m 0644" BSD_INSTALL_MAN="install -m 444" ninja -j8 -v all; then if [ -n "Try to set MAKE_JOBS_UNSAFE=yes and rebuild before reporting the failure to the maintainer." ] ; then echo "===> Compilation failed unexpectedly."; (echo "Try to set MAKE_JOBS_UNSAFE=yes and rebuild before reporting the failure to the maintainer.") | /usr/bin/fmt 75 79 ; fi; false; fi)

test is done with

(cd /usr/ports/devel/benchmark/work/.build; if ! /usr/bin/env XDG_DATA_HOME=/usr/ports/devel/benchmark/work XDG_CONFIG_HOME=/usr/ports/devel/benchmark/work HOME=/usr/ports/devel/benchmark/work PATH=/usr/ports/devel/benchmark/work/.bin:/home/yuri/bin:/sbin:/bin:/usr/sbin:/usr/bin:/usr/local/sbin:/usr/local/bin NO_PIE=yes MK_DEBUG_FILES=no MK_KERNEL_SYMBOLS=no SHELL=/bin/sh NO_LINT=YES DESTDIR=/usr/ports/devel/benchmark/work/stage PREFIX=/usr/local LOCALBASE=/usr/local CC="cc" CFLAGS="-O2 -pipe -fno-omit-frame-pointer -fstack-protector-strong -fno-strict-aliasing " CPP="cpp" CPPFLAGS="-fno-omit-frame-pointer" LDFLAGS=" -fstack-protector-strong " LIBS="" CXX="c++" CXXFLAGS="-O2 -pipe -fno-omit-frame-pointer -fstack-protector-strong -fno-strict-aliasing -fno-omit-frame-pointer -Wno-thread-safety-analysis -Wno-sign-compare " MANPREFIX="/usr/local" BSD_INSTALL_PROGRAM="install -s -m 555" BSD_INSTALL_LIB="install -s -m 0644" BSD_INSTALL_SCRIPT="install -m 555" BSD_INSTALL_DATA="install -m 0644" BSD_INSTALL_MAN="install -m 444" ninja -j8 -v all; then if [ -n "Try to set MAKE_JOBS_UNSAFE=yes and rebuild before reporting the failure to the maintainer." ] ; then echo "===> Compilation failed unexpectedly."; (echo "Try to set MAKE_JOBS_UNSAFE=yes and rebuild before reporting the failure to the maintainer.") | /usr/bin/fmt 75 79 ; fi; false; fi)

Sorry, these commands are auto-generated by the framework, and are very long.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/vmware/cascade/issues/211?email_source=notifications&email_token=AAE3RLXSXN542URGXBGZ3ELQ4FKP3A5CNFSM4KCULMSKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEIDH27A#issuecomment-570850684, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAE3RLRM6D4WS5X537NTKYLQ4FKP3ANCNFSM4KCULMSA .

eschkufz commented 4 years ago

Give the most recent commit to issue-211 a shot. Fingers crossed that does the trick.

yurivict commented 4 years ago

Yes, tests pass now in issue-211.

eschkufz commented 4 years ago

Great! I'll pull the changes into master.

On Sun, Jan 5, 2020 at 12:34 PM yuri@FreeBSD notifications@github.com wrote:

Yes, tests pass now in issue-211.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/vmware/cascade/issues/211?email_source=notifications&email_token=AAE3RLSUSLU6X5GR4EIGR3DQ4I73RA5CNFSM4KCULMSKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEID64YY#issuecomment-570945123, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAE3RLSUC4FAJF23TG7LU3LQ4I73RANCNFSM4KCULMSA .