CoreIR Passing Module As Generator Argument issue

David-Durst commented 6 years ago

Not sure that this works. My code is at https://github.com/David-Durst/coreir/blob/aetherling/src/libs/aetherlinglib.cpp#L50. When I use it, I get the following error:

durst@DN800caf06:~/dev/W17-8/coreir/coreir$ make test -j
/Library/Developer/CommandLineTools/usr/bin/make -C src dylib
/Library/Developer/CommandLineTools/usr/bin/make -C simulator
make[2]: Nothing to be done for `all'.
/Library/Developer/CommandLineTools/usr/bin/make -C passes
/Library/Developer/CommandLineTools/usr/bin/make -C analysis
make[3]: Nothing to be done for `all'.
/Library/Developer/CommandLineTools/usr/bin/make -C transform
make[3]: Nothing to be done for `all'.
/Library/Developer/CommandLineTools/usr/bin/make -C ir dylib
make[2]: Nothing to be done for `dylib'.
/Library/Developer/CommandLineTools/usr/bin/make -C coreir-c dylib
make[2]: Nothing to be done for `dylib'.
/Library/Developer/CommandLineTools/usr/bin/make -C libs dylib
g++ -std=c++11 -Wall -fPIC -Werror -I../../include -I. -c -o build/aetherlinglib.o aetherlinglib.cpp
g++ -std=c++11 -Wall -fPIC -Werror -I../../include -I. -c -o build/commonlib.o commonlib.cpp
g++ -std=c++11 -Wall -fPIC -Werror -I../../include -I. -c -o build/rtlil.o rtlil.cpp
In file included from aetherlinglib.cpp:1:
In file included from ../../include/coreir/libs/aetherlinglib.h:8:
In file included from ../../include/coreir.h:9:
../../include/coreir/ir/value.h:56:28: error: implicit instantiation of undefined template 'CoreIR::Underlying2ValueType<CoreIR::Module *>'
      return cast<typename Underlying2ValueType<T>::type>(this)->get();
                           ^
aetherlinglib.cpp:50:56: note: in instantiation of function template specialization 'CoreIR::Value::get<CoreIR::Module *>' requested here
            Module* opModule = genargs.at("operator")->get<Module*>();
                                                       ^
../../include/coreir/ir/value.h:14:8: note: template is declared here
struct Underlying2ValueType;
       ^
1 error generated.
make[2]: *** [build/aetherlinglib.o] Error 1
make[2]: *** Waiting for unfinished jobs....
rm build/rtlil.o build/commonlib.o
make[1]: *** [dylib] Error 2
make: *** [build] Error 2
durst@DN800caf06:~/dev/W17-8/coreir/coreir$

David-Durst commented 6 years ago

Fixed by https://github.com/David-Durst/coreir/commit/78ac275e19c9b2025d4b21e3215886a0a26cfc1c . Will close this when I merge in the changes.

David-Durst commented 6 years ago

Another bug, see https://github.com/David-Durst/coreir/blob/aetherling/tests/unit/simpleAetherling.cpp for the test,

for file in 'build/*'
build/simpleAetherling ERROR: Param type mismatch for: operator (global.mulBy2 vs Module)

0 libcoreir.dylib 0x00000001094f31c4 _ZN6CoreIR20checkValuesAreParamsENSt313mapINS0_12basic_stringIcNS0_11char_traitsIcEENS0_9allocatorIcEEEEPNS_5ValueENS0_4lessIS7_EENS5_INS0_4pairIKS7_S9_EEEEEENS1_IS7_PNS_9ValueTypeESB_NS5_INSC_ISD_SI_EEEEEE + 14484 1 libcoreir.dylib 0x000000010963fa64 _ZN6CoreIR9Generator9getModuleENSt3__13mapINS1_12basic_stringIcNS1_11char_traitsIcEENS1_9allocatorIcEEEEPNS_5ValueENS1_4lessIS8_EENS6_INS1_4pairIKS8_SA_EEEEEE + 5780 2 libcoreir.dylib 0x00000001096a2572 _ZN6CoreIR9ModuleDef11addInstanceENSt3112basic_stringIcNS1_11char_traitsIcEENS1_9allocatorIcEEEEPNS_9GeneratorENS1_3mapIS7_PNS_5ValueENS1_4lessIS7_EENS5_INS1_4pairIKS7_SCEEEEEESJ + 2418 3 libcoreir.dylib 0x00000001096a497d _ZN6CoreIR9ModuleDef11addInstanceENSt3__112basic_stringIcNS1_11char_traitsIcEENS1_9allocatorIcEEEES7_NS1_3mapIS7_PNS_5ValueENS1_4lessIS7_EENS5_INS1_4pairIKS7_SAEEEEEESH + 4109 4 simpleAetherling 0x00000001094be29b main + 10619 5 libdyld.dylib 0x00007fff6b6c7115 start + 1 make: *** [test] Error 1

rdaly525 commented 6 years ago

Just fixed with latest push to valuemodule branch

David-Durst commented 6 years ago

I cherry-picked in the commit (see https://github.com/David-Durst/coreir/commits/aetherling) and now I get a segfault when running my tests. The error from the log is:

for file in 'build/*'
build/simpleAetherling In Run Generators ERROR: ./run: line 12: 31451 Segmentation fault: 11 $file make: *** [test] Error 139

rdaly525 commented 6 years ago

@David-Durst Not sure where you are getting a segfault. I added a test (test/unit/moduleparam.cpp) which is an example of a map where you pass in a module. See if there is some difference between your test and mine and if there is, can you send the full test that segfaults so that I can reproduce it?

David-Durst commented 6 years ago

The segfault disappears. It may have been old binaries. I'm not sure how that happened, as I was sure to make clean before asking about the segault. My unit test now works.

However, I still can't seem to get my simulator test to work. I now have your test running through a simulator and it works https://github.com/David-Durst/coreir/blob/aetherling/tests/simulator/moduleparamsim.cpp . However, when I try to do the same thing, mine fails: https://github.com/David-Durst/coreir/blob/aetherling/tests/simulator/aetherlingSim.cpp. Is there any chance this is because the module you are using is in coreIR and the one I'm using is in the global namespace? Maybe this causes the code to be unable to find my module when it tries to make an instance of it?

I'm also getting a weird issue where sometimes the getModuleRef()->print() works and sometimes it doesn't. See the error message below, which doesn't include mapN definition even though it should by https://github.com/David-Durst/coreir/blob/aetherling/tests/simulator/aetherlingSim.cpp#L81. (note: sometimes the line prints out. I've been unable to identify what causes it to work or not).

My error message is now:

+ [[ -n '' ]]
+ ./build/simtests
ALREADY ADDED CONNECTION!
ALREADY ADDED CONNECTION!
ALREADY ADDED CONNECTION!
In Run Generators
Done running generators
Module: mainMapNMulTest
  Type: {'out_0':Bit[16], 'out_1':Bit[16], 'out_2':Bit[16], 'out_3':Bit[16]}
  Def? Yes
  Def:
    Instances:
      constInput0 : const(width:16)
      constInput1 : const(width:16)
      constInput2 : const(width:16)
      constInput3 : const(width:16)
      map4$op_0 : mulBy2
      map4$op_1 : mulBy2
      map4$op_2 : mulBy2
      map4$op_3 : mulBy2
    Connections:
      constInput0.out <=> map4$op_0.in
      constInput1.out <=> map4$op_1.in
      constInput2.out <=> map4$op_2.in
      constInput3.out <=> map4$op_3.in
      map4$op_0.out <=> self.out_0
      map4$op_1.out <=> self.out_1
      map4$op_2.out <=> self.out_2
      map4$op_3.out <=> self.out_3

Starting topological sort
topo_order.size() = 12
numVertices(g)    = 12
Unsupported node: map4$op_3 has operation name: global.mulBy2
Assertion failed: (false), function updateNodeValues, file interpret.cpp, line 1125.

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
simtests is a Catch v1.3.1 host application.
Run with -? for options

-------------------------------------------------------------------------------
Simulate mapN from aetherlinglib
  aetherlinglib mapN with 4 mul, 3 as constant, 16 bit width
-------------------------------------------------------------------------------
aetherlingSim.cpp:19
...............................................................................

aetherlingSim.cpp:24: FAILED:
due to a fatal error condition:
  SIGABRT - Abort (abnormal termination) signal

===============================================================================
test cases: 1 | 1 failed
assertions: 1 | 1 failed

rdaly525 commented 6 years ago

the Module.newModuleDef() function only creates a new module def object based on the module declaration. It does not actually set that definition as the module's definition.

You need to use Module.setDef(ModuleDef*) function to actually get the function set.

David-Durst commented 6 years ago

For some reason, once I add the setDef, i get the below message when I run the executable straight, but I get a different failure when I run it in lldb. I'm investigating further.

set -e
[[ -n '' ]]
./build/simtests ALREADY ADDED CONNECTION! ALREADY ADDED CONNECTION! ALREADY ADDED CONNECTION! Module: mapN(operator:global.mulBy2, parallelOperators:4, width:16) Type: {'in':BitIn[16][4], 'out':Bit[16][4]} Def? No In Run Generators Done running generators Module: mainMapNMulTest Type: {'out_0':Bit[16], 'out_1':Bit[16], 'out_2':Bit[16], 'out_3':Bit[16]} Def? Yes Def: Instances: constInput0 : const(width:16) constInput1 : const(width:16) constInput2 : const(width:16) constInput3 : const(width:16) map4$op_0$constInput_4 : const(width:16) map4$op_0$mul : mul(width:16) map4$op_1$constInput_4 : const(width:16) map4$op_1$mul : mul(width:16) map4$op_2$constInput_4 : const(width:16) map4$op_2$mul : mul(width:16) map4$op_3$constInput_4 : const(width:16) map4$op_3$mul : mul(width:16) Connections: constInput0.out <=> map4$op_0$mul.in0 constInput1.out <=> map4$op_1$mul.in0 constInput2.out <=> map4$op_2$mul.in0 constInput3.out <=> map4$op_3$mul.in0 map4$op_0$constInput_4.out <=> map4$op_0$mul.in1 map4$op_0$mul.out <=> self.out_0 map4$op_1$constInput_4.out <=> map4$op_1$mul.in1 map4$op_1$mul.out <=> self.out_1 map4$op_2$constInput_4.out <=> map4$op_2$mul.in1 map4$op_2$mul.out <=> self.out_2 map4$op_3$constInput_4.out <=> map4$op_3$mul.in1 map4$op_3$mul.out <=> self.out_3

hi


simtests is a Catch v1.3.1 host application.
Run with -? for options

-------------------------------------------------------------------------------
Simulate mapN from aetherlinglib
  aetherlinglib mapN with 4 mul, 3 as constant, 16 bit width
-------------------------------------------------------------------------------
aetherlingSim.cpp:20
...............................................................................

aetherlingSim.cpp:25: FAILED:
due to a fatal error condition:
  SIGSEGV - Segmentation violation signal

===============================================================================
test cases: 1 | 1 failed
assertions: 1 | 1 failed

David-Durst commented 6 years ago

Fixing some of the issues. For some reason, some change I'm making is causing lots of other tests to fail. Do you have any idea why enabling debugging flags or just running make from within the tests/simulator folder would cause many other tests to start failing?

David-Durst commented 6 years ago

Ok, so weird stuff. I cloned a new copy of it and now everything works (after doing the setDef and a few other things). Two questions:

How do I avoid doing this in the future? Its annoying that repeated git checkout and make clean didn't work.
https://github.com/David-Durst/coreir/blob/aetherling/tests/simulator/aetherlingSim.cpp#L83 is causing a segfault in the tests. Any idea why? I don't think its my code, as the same code is ok a few lines earlier. I think this plays into the issue I was seeing earlier with it only printing sometimes. I believe this line causes the segfaults as: (a) segfaults happen when its uncommented, don't when its commented, and (b) lldb shows an invalid memory access (I think that's the error message, I forget exactly what the message is) when it hits this line.

David-Durst commented 6 years ago

One last error message. If this passes, should be in the clear and then I'll pull request this in.

I'm wiring a bitin to a muxN's selector and but it doesn't appear that the wiring is actually occurring. The below error message shows that self.mergeCur isn't getting wired to mergeMux.in.sel even though I'm doing so on https://github.com/David-Durst/coreir/blob/aetherling/src/libs/aetherlinglib/aeReduceN.h#L113. Also, the failure message appears to be about selectors not being wired up.

Module: add(width:16) Type: {'in0':BitIn[16], 'in1':BitIn[16], 'out':Bit[16]} Def? No Module: reduceNSerializable(numLayers:4, operator:coreir.add, width:16) Type: {'in':BitIn[16][16], 'out':Bit[16], 'mergeCur':BitIn} Def? No In Run Generators Done running generators Module: mainMapNMulTest Type: {'out':Bit[16]} Def? Yes Def: Instances: constInput0 : const(width:16) constInput1 : const(width:16) constInput10 : const(width:16) constInput11 : const(width:16) constInput12 : const(width:16) constInput13 : const(width:16) constInput14 : const(width:16) constInput15 : const(width:16) constInput2 : const(width:16) constInput3 : const(width:16) constInput4 : const(width:16) constInput5 : const(width:16) constInput6 : const(width:16) constInput7 : const(width:16) constInput8 : const(width:16) constInput9 : const(width:16) reduce4$lastOutputReg : reg(width:16) reduce4$mergeMux$join : mux(width:16) reduce4$mergeOp : add(width:16) reduce4$reducer$op_0_0 : add(width:16) reduce4$reducer$op_1_0 : add(width:16) reduce4$reducer$op_1_1 : add(width:16) reduce4$reducer$op_2_0 : add(width:16) reduce4$reducer$op_2_1 : add(width:16) reduce4$reducer$op_2_2 : add(width:16) reduce4$reducer$op_2_3 : add(width:16) reduce4$reducer$op_3_0 : add(width:16) reduce4$reducer$op_3_1 : add(width:16) reduce4$reducer$op_3_2 : add(width:16) reduce4$reducer$op_3_3 : add(width:16) reduce4$reducer$op_3_4 : add(width:16) reduce4$reducer$op_3_5 : add(width:16) reduce4$reducer$op_3_6 : add(width:16) reduce4$reducer$op_3_7 : add(width:16) Connections: constInput0.out <=> reduce4$reducer$op_3_0.in0 constInput1.out <=> reduce4$reducer$op_3_0.in1 constInput10.out <=> reduce4$reducer$op_3_5.in0 constInput11.out <=> reduce4$reducer$op_3_5.in1 constInput12.out <=> reduce4$reducer$op_3_6.in0 constInput13.out <=> reduce4$reducer$op_3_6.in1 constInput14.out <=> reduce4$reducer$op_3_7.in0 constInput15.out <=> reduce4$reducer$op_3_7.in1 constInput2.out <=> reduce4$reducer$op_3_1.in0 constInput3.out <=> reduce4$reducer$op_3_1.in1 constInput4.out <=> reduce4$reducer$op_3_2.in0 constInput5.out <=> reduce4$reducer$op_3_2.in1 constInput6.out <=> reduce4$reducer$op_3_3.in0 constInput7.out <=> reduce4$reducer$op_3_3.in1 constInput8.out <=> reduce4$reducer$op_3_4.in0 constInput9.out <=> reduce4$reducer$op_3_4.in1 reduce4$lastOutputReg.in <=> reduce4$mergeMux$join.out reduce4$lastOutputReg.out <=> reduce4$mergeOp.in1 reduce4$mergeMux$join.in0 <=> reduce4$reducer$op_0_0.out reduce4$mergeMux$join.in1 <=> reduce4$mergeOp.out reduce4$mergeMux$join.out <=> self.out reduce4$mergeOp.in0 <=> reduce4$reducer$op_0_0.out reduce4$reducer$op_1_0.in0 <=> reduce4$reducer$op_2_0.out reduce4$reducer$op_1_0.in1 <=> reduce4$reducer$op_2_1.out reduce4$reducer$op_1_1.in0 <=> reduce4$reducer$op_2_2.out reduce4$reducer$op_1_1.in1 <=> reduce4$reducer$op_2_3.out reduce4$reducer$op_2_0.in0 <=> reduce4$reducer$op_3_0.out reduce4$reducer$op_2_0.in1 <=> reduce4$reducer$op_3_1.out reduce4$reducer$op_2_1.in0 <=> reduce4$reducer$op_3_2.out reduce4$reducer$op_2_1.in1 <=> reduce4$reducer$op_3_3.out reduce4$reducer$op_2_2.in0 <=> reduce4$reducer$op_3_4.out reduce4$reducer$op_2_2.in1 <=> reduce4$reducer$op_3_5.out reduce4$reducer$op_2_3.in0 <=> reduce4$reducer$op_3_6.out reduce4$reducer$op_2_3.in1 <=> reduce4$reducer$op_3_7.out

Starting topological sort topo_order.size() = 36 numVertices(g) = 36 Assertion failed: (inSels.size() == 2), function updateBitVecBinop, file interpret.cpp, line 756.


simtests is a Catch v1.3.1 host application.
Run with -? for options

-------------------------------------------------------------------------------
Simulate reduceNSerializable from aetherlinglib
  aetherlinglib reduceN with 4 inputs, coreir.add as op, 16 bit width
-------------------------------------------------------------------------------
aetherlingSim.cpp:96
...............................................................................

aetherlingSim.cpp:101: FAILED:
due to a fatal error condition:
  SIGABRT - Abort (abnormal termination) signal

dillonhuff commented 6 years ago

David,

That is a problem that should be fixed in my version of the interpreter, but it is not fixed in the version currently in dev.

I will make a pull request that should fix it.

On Mon, Jan 29, 2018 at 11:49 AM, David Durst notifications@github.com wrote:

One last error message. If this passes, should be in the clear and then I'll pull request this in.

I'm wiring a bitin to a muxN's selector and but it doesn't appear that the wiring is actually occurring. The below error message shows that self.mergeCur isn't getting wired to mergeMux.in.sel even though I'm doing so on https://github.com/David-Durst/coreir/blob/aetherling/ src/libs/aetherlinglib/aeReduceN.h#L113. Also, the failure message appears to be about selectors not being wired up.

Module: add(width:16) Type: {'in0':BitIn[16], 'in1':BitIn[16], 'out':Bit[16]} Def? No Module: reduceNSerializable(numLayers:4, operator:coreir.add, width:16) Type: {'in':BitIn[16][16], 'out':Bit[16], 'mergeCur':BitIn} Def? No In Run Generators Done running generators Module: mainMapNMulTest Type: {'out':Bit[16]} Def? Yes Def: Instances: constInput0 : const(width:16) constInput1 : const(width:16) constInput10 : const(width:16) constInput11 : const(width:16) constInput12 : const(width:16) constInput13 : const(width:16) constInput14 : const(width:16) constInput15 : const(width:16) constInput2 : const(width:16) constInput3 : const(width:16) constInput4 : const(width:16) constInput5 : const(width:16) constInput6 : const(width:16) constInput7 : const(width:16) constInput8 : const(width:16) constInput9 : const(width:16) reduce4$lastOutputReg : reg(width:16) reduce4$mergeMux$join : mux(width:16) reduce4$mergeOp : add(width:16) reduce4$reducer$op_0_0 : add(width:16) reduce4$reducer$op_1_0 : add(width:16) reduce4$reducer$op_1_1 : add(width:16) reduce4$reducer$op_2_0 : add(width:16) reduce4$reducer$op_2_1 : add(width:16) reduce4$reducer$op_2_2 : add(width:16) reduce4$reducer$op_2_3 : add(width:16) reduce4$reducer$op_3_0 : add(width:16) reduce4$reducer$op_3_1 : add(width:16) reduce4$reducer$op_3_2 : add(width:16) reduce4$reducer$op_3_3 : add(width:16) reduce4$reducer$op_3_4 : add(width:16) reduce4$reducer$op_3_5 : add(width:16) reduce4$reducer$op_3_6 : add(width:16) reduce4$reducer$op_3_7 : add(width:16) Connections: constInput0.out <=> reduce4$reducer$op_3_0.in0 constInput1.out <=> reduce4$reducer$op_3_0.in1 constInput10.out <=> reduce4$reducer$op_3_5.in0 constInput11.out <=> reduce4$reducer$op_3_5.in1 constInput12.out <=> reduce4$reducer$op_3_6.in0 constInput13.out <=> reduce4$reducer$op_3_6.in1 constInput14.out <=> reduce4$reducer$op_3_7.in0 constInput15.out <=> reduce4$reducer$op_3_7.in1 constInput2.out <=> reduce4$reducer$op_3_1.in0 constInput3.out <=> reduce4$reducer$op_3_1.in1 constInput4.out <=> reduce4$reducer$op_3_2.in0 constInput5.out <=> reduce4$reducer$op_3_2.in1 constInput6.out <=> reduce4$reducer$op_3_3.in0 constInput7.out <=> reduce4$reducer$op_3_3.in1 constInput8.out <=> reduce4$reducer$op_3_4.in0 constInput9.out <=> reduce4$reducer$op_3_4.in1 reduce4$lastOutputReg.in <=> reduce4$mergeMux$join.out reduce4$lastOutputReg.out <=> reduce4$mergeOp.in1 reduce4$mergeMux$join.in0 <=> reduce4$reducer$op_0_0.out reduce4$mergeMux$join.in1 <=> reduce4$mergeOp.out reduce4$mergeMux$join.out <=> self.out reduce4$mergeOp.in0 <=> reduce4$reducer$op_0_0.out reduce4$reducer$op_1_0.in0 <=> reduce4$reducer$op_2_0.out reduce4$reducer$op_1_0.in1 <=> reduce4$reducer$op_2_1.out reduce4$reducer$op_1_1.in0 <=> reduce4$reducer$op_2_2.out reduce4$reducer$op_1_1.in1 <=> reduce4$reducer$op_2_3.out reduce4$reducer$op_2_0.in0 <=> reduce4$reducer$op_3_0.out reduce4$reducer$op_2_0.in1 <=> reduce4$reducer$op_3_1.out reduce4$reducer$op_2_1.in0 <=> reduce4$reducer$op_3_2.out reduce4$reducer$op_2_1.in1 <=> reduce4$reducer$op_3_3.out reduce4$reducer$op_2_2.in0 <=> reduce4$reducer$op_3_4.out reduce4$reducer$op_2_2.in1 <=> reduce4$reducer$op_3_5.out reduce4$reducer$op_2_3.in0 <=> reduce4$reducer$op_3_6.out reduce4$reducer$op_2_3.in1 <=> reduce4$reducer$op_3_7.out

Starting topological sort topo_order.size() = 36 numVertices(g) = 36 Assertion failed: (inSels.size() == 2), function updateBitVecBinop, file interpret.cpp, line 756.

simtests is a Catch v1.3.1 host application. Run with -? for options

Simulate reduceNSerializable from aetherlinglib aetherlinglib reduceN with 4 inputs, coreir.add as op, 16 bit width

aetherlingSim.cpp:96 ...............................................................................

aetherlingSim.cpp:101: FAILED: due to a fatal error condition: SIGABRT - Abort (abnormal termination) signal

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/rdaly525/coreir/issues/364#issuecomment-361364648, or mute the thread https://github.com/notifications/unsubscribe-auth/ADOwsFvOJfvYnAPWSXoJzxiQT0V0prRzks5tPiDLgaJpZM4Ru4jw .

David-Durst commented 6 years ago

My code that uses this is at pull request #369

rdaly525 commented 6 years ago

@David-Durst Can you confirm that this can be closed?

rdaly525 / coreir

CoreIR Passing Module As Generator Argument issue #364

Simulate reduceNSerializable from aetherlinglib aetherlinglib reduceN with 4 inputs, coreir.add as op, 16 bit width