search

cpc / openasip

Open Application-Specific Instruction Set processor tools (OpenASIP)
http://openasip.org
Other
138 stars 41 forks source link

New version 2.0 oa-selftest -v test error #211

Closed dingzex closed 1 year ago

dingzex commented 1 year ago

安装后运行 oa-selftest -v 时，发生如下错误： Traceback (most recent call last): File "/home/carbon/local/bin/oa-selftest", line 597, in test_proge_and_pig self.assertEqual(stderr, '') AssertionError: "HDBManager.cc:4391: In function 'int HDB[241 chars]d)\n" != ''

Ran 8 tests in 133.643s

FAILED (failures=1)

dingzex commented 1 year ago

sqlite3 -version is : 3.31.1 2020-01-27 19:55:54 3bfa9cc97da10598521b342961df8f5f68c7388fa117345eeb516eaa837balt1

pjaaskel commented 1 year ago

Hi, which Linux distribution you are using?

dingzex commented 1 year ago

Hi, which Linux distribution you are using?

ubuntu 20.04

pjaaskel commented 1 year ago

Same here, and it works, hmm.

bojle commented 1 year ago

Any updates on this issue @pjaaskel ? I am struggling with this too (ubuntu version 22.10, sqlite version 3.39.3)

pjaaskel commented 1 year ago

Can you test with the latest revision? That at least works here:

$ oa-selftest -v
Testing OpenASIP installation, this can take up to two hours.
Time to go for a lunch? Use -v to see the progress.
test_noOpt (__main__.TestCompiler)
Test C compilation with -O0 ... ok
test_opt (__main__.TestCompiler)
Test C compilation with -O3 ... ok
test_custom_operation_from_c_code (__main__.TestCustomOps)
Test creating a simple custom operation with simulation code and ... ok
test_basic_plugins (__main__.TestExplorer)
Test the ImplementationSelector and Evaluate explorer plugins (this ... ok
test_default_plugins_found (__main__.TestExplorer)
Check that the default Explorer plugins are listed. ... ok
test_proge_and_pig (__main__.TestProGeAndPIG)
Test the processor generator and the program image generator (with ... ok
test_compiled_simulation (__main__.TestSimulator)
Test the compiled engine of the simulator ... ok
test_debugging_simulation (__main__.TestSimulator)
Test the debugging engine of the simulator ... ok

----------------------------------------------------------------------
Ran 8 tests in 211.219s

OK
...
$ sqlite3 --version
3.31.1 2020-01-27 19:55:54 3bfa9cc97da10598521b342961df8f5f68c7388fa117345eeb516eaa837balt1
$ cat /etc/issue
Ubuntu 20.04.5 LTS \n \l
bojle commented 1 year ago

Hey, what do you mean by latest revision? I've tried both, llvm 15 (from main branch) and 16 builds (from llvm_16 branch). Both builds suffer from the same issue (SQL logic error in HDBManager). For the sake of it, I degraded my sqlite to 3.31.1 to match yours, the problem persists. Here's what i got from oa-selftest -v on llvm_16 based build:

Testing OpenASIP installation, this can take up to two hours.
Time to go for a lunch? Use -v to see the progress.
test_noOpt (main.TestCompiler)
Test C compilation with -O0 ... FAIL
test_opt (main.TestCompiler)
Test C compilation with -O3 ... ok
test_custom_operation_from_c_code (main.TestCustomOps)
Test creating a simple custom operation with simulation code and ... ok
test_basic_plugins (main.TestExplorer)
Test the ImplementationSelector and Evaluate explorer plugins (this ... ok
test_default_plugins_found (main.TestExplorer)
Check that the default Explorer plugins are listed. ... ok
test_proge_and_pig (main.TestProGeAndPIG)
Test the processor generator and the program image generator (with ... FAIL
test_compiled_simulation (main.TestSimulator)
Test the compiled engine of the simulator ... FAIL
test_debugging_simulation (main.TestSimulator)
Test the debugging engine of the simulator ... FAIL

======================================================================
FAIL: test_noOpt (main.TestCompiler)
Test C compilation with -O0

Traceback (most recent call last):
File "/home/vicharak/hdd/openasip/openasip/llvm/bin/oa-selftest", line 200, in test_noOpt
self._test_compilation("-O0 --swfp")
File "/home/vicharak/hdd/openasip/openasip/llvm/bin/oa-selftest", line 193, in _test_compilation
self.assertEqual(stderr, "")
AssertionError: 'Traceback (most recent call last):\n Fil[355 chars]ed\n' != ''

Traceback (most recent call last):
File "/home/vicharak/hdd/openasip/openasip/llvm/bin/oacc", line 1700, in <module>
emulationLinkedFileName = linkEmulationFuncs(optimizedFileName, namePrefix)
File "/home/vicharak/hdd/openasip/openasip/llvm/bin/oacc", line 780, in linkEmulationFuncs
" -f " + internalizeCmd + " -o " + outputName +
NameError: name 'internalizeCmd' is not defined
======================================================================
FAIL: test_proge_and_pig (main.TestProGeAndPIG)
Test the processor generator and the program image generator (with

Traceback (most recent call last):
File "/home/vicharak/hdd/openasip/openasip/llvm/bin/oa-selftest", line 597, in test_proge_and_pig
self.assertEqual(stderr, '')
AssertionError: "[HDBManager.cc:4385](http://hdbmanager.cc:4385/): In function 'int HDB[261 chars]d)\n" != ''

[HDBManager.cc:4385](http://hdbmanager.cc:4385/): In function 'int HDB::HDBManager::addBooleanColumn(const std::string&, const std::string&)': DEBUG: SQL logic error
generateprocessor: [HDBManager.cc:4387](http://hdbmanager.cc:4387/): int HDB::HDBManager::addBooleanColumn(const std::string&, const std::string&): Assertion `false' failed.
Aborted (core dumped)
======================================================================
FAIL: test_compiled_simulation (main.TestSimulator)
Test the compiled engine of the simulator

Traceback (most recent call last):
File "/home/vicharak/hdd/openasip/openasip/llvm/bin/oa-selftest", line 265, in test_compiled_simulation
self._test_simulation("-q")
File "/home/vicharak/hdd/openasip/openasip/llvm/bin/oa-selftest", line 255, in _test_simulation
self.assertEqual(stdout.strip(), "Hello \nTTA World!")
AssertionError: 'Unable to load program. Cannot find suita[60 chars]zed.' != 'Hello \nTTA World!'

Unable to load program. Cannot find suitable reader implementation for file.
Simulation not initialized.+ Hello
TTA World!
======================================================================
FAIL: test_debugging_simulation (main.TestSimulator)
Test the debugging engine of the simulator

Traceback (most recent call last):
File "/home/vicharak/hdd/openasip/openasip/llvm/bin/oa-selftest", line 261, in test_debugging_simulation
self._test_simulation("")
File "/home/vicharak/hdd/openasip/openasip/llvm/bin/oa-selftest", line 255, in _test_simulation
self.assertEqual(stdout.strip(), "Hello \nTTA World!")
AssertionError: 'Unable to load program. Cannot find suita[60 chars]zed.' != 'Hello \nTTA World!'

Unable to load program. Cannot find suitable reader implementation for file.
Simulation not initialized.+ Hello
TTA World!
Ran 8 tests in 22.101s

FAILED (failures=4)

I dug a little deep and found the values of table and newcolum passed to addBooleanColumn when it crashes. table = rf_architecture and column = zero_register. Could the problem be with this table not being present as it is a "logic error"?

pjaaskel commented 1 year ago

I mean did you try OpenASIP main branch or just the release branch? @nrother this sounds it could be related to your DB fixes.

nrother commented 1 year ago

Hm, could be, but #213 doesn't touch anything called rf_architecture or zero_register... Is there a way to get more output from SQlite what the actual problem is?

bojle commented 1 year ago

Well, I seem to have fixed it mostly. Not really sure how, I did downgrade sqlite to 3.31.1 and did a clean re-compile. oa-selftest still isn't clean. I'll post the output when I get back to my computer.

pjaaskel commented 1 year ago

Is this with the master branch or the 2.0 release branch?

bojle commented 1 year ago

This is with the llvm_16 branch, but compiled with install_llvm_15.sh. Here's the oa-selftest

Testing OpenASIP installation, this can take up to two hours.
Time to go for a lunch? Use -v to see the progress.
test_noOpt (__main__.TestCompiler)
Test C compilation with -O0 ... FAIL
test_opt (__main__.TestCompiler)
Test C compilation with -O3 ... ok
test_custom_operation_from_c_code (__main__.TestCustomOps)
Test creating a simple custom operation with simulation code and ... ok
test_basic_plugins (__main__.TestExplorer)
Test the ImplementationSelector and Evaluate explorer plugins (this ... ok
test_default_plugins_found (__main__.TestExplorer)
Check that the default Explorer plugins are listed. ... ok
test_proge_and_pig (__main__.TestProGeAndPIG)
Test the processor generator and the program image generator (with ... FAIL
test_compiled_simulation (__main__.TestSimulator)
Test the compiled engine of the simulator ... FAIL
test_debugging_simulation (__main__.TestSimulator)
Test the debugging engine of the simulator ... FAIL

======================================================================
FAIL: test_noOpt (__main__.TestCompiler)
Test C compilation with -O0
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/vicharak/hdd/openasip/openasip/llvm/bin/oa-selftest", line 200, in test_noOpt
    self._test_compilation("-O0 --swfp")
  File "/home/vicharak/hdd/openasip/openasip/llvm/bin/oa-selftest", line 193, in _test_compilation
    self.assertEqual(stderr, "")
AssertionError: 'Traceback (most recent call last):\n  Fil[355 chars]ed\n' != ''
- Traceback (most recent call last):
-   File "/home/vicharak/hdd/openasip/openasip/llvm/bin/oacc", line 1700, in <module>
-     emulationLinkedFileName = linkEmulationFuncs(optimizedFileName, namePrefix)
-   File "/home/vicharak/hdd/openasip/openasip/llvm/bin/oacc", line 780, in linkEmulationFuncs
-     " -f " + internalizeCmd + " -o " + outputName +
- NameError: name 'internalizeCmd' is not defined

======================================================================
FAIL: test_proge_and_pig (__main__.TestProGeAndPIG)
Test the processor generator and the program image generator (with
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/vicharak/hdd/openasip/openasip/llvm/bin/oa-selftest", line 597, in test_proge_and_pig
    self.assertEqual(stderr, '')
AssertionError: "Unable to copy file /tmp/tmp5q9xoier/pro[168 chars]l'\n" != ''
- Unable to copy file /tmp/tmp5q9xoier/proc_vhdl/vhdl/lsu_le.vhdl:Unable to copy '/home/vicharak/hdd/openasip/openasip/llvm/share/openasip/hdb/vhdl/fu/lsu_le.vhdl' to '/tmp/tmp5q9xoier/proc_vhdl/vhdl/lsu_le.vhdl'

======================================================================
FAIL: test_compiled_simulation (__main__.TestSimulator)
Test the compiled engine of the simulator
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/vicharak/hdd/openasip/openasip/llvm/bin/oa-selftest", line 265, in test_compiled_simulation
    self._test_simulation("-q")
  File "/home/vicharak/hdd/openasip/openasip/llvm/bin/oa-selftest", line 255, in _test_simulation
    self.assertEqual(stdout.strip(), "Hello \nTTA World!")
AssertionError: 'Unable to load program. Cannot find suita[60 chars]zed.' != 'Hello \nTTA World!'
- Unable to load program. Cannot find suitable reader implementation for file.
- Simulation not initialized.+ Hello
+ TTA World!

======================================================================
FAIL: test_debugging_simulation (__main__.TestSimulator)
Test the debugging engine of the simulator
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/vicharak/hdd/openasip/openasip/llvm/bin/oa-selftest", line 261, in test_debugging_simulation
    self._test_simulation("")
  File "/home/vicharak/hdd/openasip/openasip/llvm/bin/oa-selftest", line 255, in _test_simulation
    self.assertEqual(stdout.strip(), "Hello \nTTA World!")
AssertionError: 'Unable to load program. Cannot find suita[60 chars]zed.' != 'Hello \nTTA World!'
- Unable to load program. Cannot find suitable reader implementation for file.
- Simulation not initialized.+ Hello
+ TTA World!

----------------------------------------------------------------------
Ran 8 tests in 25.471s

FAILED (failures=4)

It mostly ~ works ~ though. I was able to follow through the crc example. Let me know if you want me to try anything.

pjaaskel commented 1 year ago

Please retry with the 'main' branch. The llvm_16 branch is a WiP branch which might break.

bojle commented 1 year ago

Built with 'main' branch, install_llvm_15.sh, autogen -> configure -> make -> make install. 

$ oa-selftest -v
Testing OpenASIP installation, this can take up to two hours.
Time to go for a lunch? Use -v to see the progress.
test_noOpt (__main__.TestCompiler)
Test C compilation with -O0 ... ok
test_opt (__main__.TestCompiler)
Test C compilation with -O3 ... ok
test_custom_operation_from_c_code (__main__.TestCustomOps)
Test creating a simple custom operation with simulation code and ... ok
test_basic_plugins (__main__.TestExplorer)
Test the ImplementationSelector and Evaluate explorer plugins (this ... ok
test_default_plugins_found (__main__.TestExplorer)
Check that the default Explorer plugins are listed. ... ok
test_proge_and_pig (__main__.TestProGeAndPIG)
Test the processor generator and the program image generator (with ... FAIL
test_compiled_simulation (__main__.TestSimulator)
Test the compiled engine of the simulator ... ok
test_debugging_simulation (__main__.TestSimulator)
Test the debugging engine of the simulator ... ok

======================================================================
FAIL: test_proge_and_pig (__main__.TestProGeAndPIG)
Test the processor generator and the program image generator (with
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/vicharak/hdd/openasip/openasip/llvm/bin/oa-selftest", line 597, in test_proge_and_pig
    self.assertEqual(stderr, '')
AssertionError: "HDBManager.cc:4377: In function 'int HDB[261 chars]d)\n" != ''
- HDBManager.cc:4377: In function 'int HDB::HDBManager::addBooleanColumn(const std::string&, const std::string&)': DEBUG: SQL logic error
- generateprocessor: HDBManager.cc:4378: int HDB::HDBManager::addBooleanColumn(const std::string&, const std::string&): Assertion `false' failed.
- Aborted (core dumped)

----------------------------------------------------------------------
Ran 8 tests in 49.963s

FAILED (failures=1)
pjaaskel commented 1 year ago

@karihepola that HDB zero_registers change was done with the RISC-V features. Do you have ideas?

@bojle is it possible that you have installed an older version of the tools and it somehow finds the older one's HDB (without that column) somewhere?

karihepola commented 1 year ago

I tried triggering this issue myself but was not successful. OpenASIP should be able to migrate older versions of the HDBs without that zero_register column to the newer version where that column is present. (I reckon that is why it was called in this case). However, the repository should have updated all the included HDBs to the newest version with that column added.

@bojle Is it possible that you have somehow corrupted HDBs? I recommend you try cloning the repository again if you haven't yet.

bojle commented 1 year ago

I am in fact doing that. All the tests on different branches that I did were fresh installation from fresh clones. How can I check whether HDBs are corrupted or not? where are they stored, I couldn't find any .db or .hdb files?

I'll rebuild again. These are the steps:

$ make unistall
$ make clean
$ rm -rf .
$ git clone github.com/cpc/openasip --depth 1
$ cd openasip/openasip
$ ./tools/scripts/install_llvm_15.sh $prefix_var
$ ./autogen && ./configure --prefix=$prefix_var && make && sudo make install
karihepola commented 1 year ago

Then your HDBs are most likely fine. You can find most of the included HDBs (*.hdb files) in the openasip/openasip/hdb directory. Were you able to find out which HDB causes the crash? All of them should have that rf_architecture table included.

karihepola commented 1 year ago

That test case that is failing should try to open these HDBs from the install directory:

$prefix_var/share/openasip/hdb/generate_base32.hdb
$prefix_var/share/openasip/hdb/generate_lsu_32.hdb
$prefix_var/share/openasip/hdb/generate_rf_iu.hdb
$prefix_var/share/openasip/hdb/asic_130nm_1.5V.hdb

You can try to open them with hdbeditor and check if you get one of them to crash. You can also inspect them with an external tool such as sqlitebrowser to see the included tables and columns.

bojle commented 1 year ago

Alright, well I was not make unistalling up till now. I think you're right, cruft from previous builds weren't being removed. Current build returns this:

Testing OpenASIP installation, this can take up to two hours.
Time to go for a lunch? Use -v to see the progress.
test_noOpt (__main__.TestCompiler)
Test C compilation with -O0 ... ok
test_opt (__main__.TestCompiler)
Test C compilation with -O3 ... ok
test_custom_operation_from_c_code (__main__.TestCustomOps)
Test creating a simple custom operation with simulation code and ... ok
test_basic_plugins (__main__.TestExplorer)
Test the ImplementationSelector and Evaluate explorer plugins (this ... ok
test_default_plugins_found (__main__.TestExplorer)
Check that the default Explorer plugins are listed. ... ok
test_proge_and_pig (__main__.TestProGeAndPIG)
Test the processor generator and the program image generator (with ... FAIL
test_compiled_simulation (__main__.TestSimulator)
Test the compiled engine of the simulator ... ok
test_debugging_simulation (__main__.TestSimulator)
Test the debugging engine of the simulator ... ok

======================================================================
FAIL: test_proge_and_pig (__main__.TestProGeAndPIG)
Test the processor generator and the program image generator (with
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/vicharak/hdd/openasip/openasip/llvm/bin/oa-selftest", line 597, in test_proge_and_pig
    self.assertEqual(stderr, '')
AssertionError: "Unable to copy file /tmp/tmpx6f1t0xw/pro[168 chars]l'\n" != ''
- Unable to copy file /tmp/tmpx6f1t0xw/proc_vhdl/vhdl/lsu_le.vhdl:Unable to copy '/home/vicharak/hdd/openasip/openasip/llvm/share/openasip/hdb/vhdl/fu/lsu_le.vhdl' to '/tmp/tmpx6f1t0xw/proc_vhdl/vhdl/lsu_le.vhdl'

----------------------------------------------------------------------
Ran 8 tests in 50.508s

FAILED (failures=1)
karihepola commented 1 year ago

For some reason it is not finding that lsu_le.vhdlsource file in your install directory. If you manually check can you see it there in the mentioned path? You can find the source file here and it should be moved to your install directory during the installation phase.

bojle commented 1 year ago

that fixed the issue, thanks

SangYukekeaiai commented 2 months ago

that fixed the issue, thanks

Hello There! Could you please tell me how to deal with this issue? I have the same issue:

Traceback (most recent call last):
  File "/home/yy/local/bin/oa-selftest", line 597, in test_proge_and_pig
    self.assertEqual(stderr, '')
AssertionError: "Unable to copy file /tmp/tmp4ug8ilgx/pro[141 chars]l'\n" != ''
- Unable to copy file /tmp/tmp4ug8ilgx/proc_vhdl/vhdl/lsu_le.vhdl:Unable to copy '/home/yy/local/share/openasip/hdb/vhdl/fu/lsu_le.vhdl' to '/tmp/tmp4ug8ilgx/proc_vhdl/vhdl/lsu_le.vhdl'

I checked my install directory, the lsu_le.vhdl files are there in both openasip folder and $HOME/local/share/openasip/hdb/vhdl/fu/ folder. The other tests are all correct.