error using python interface pass

timhae commented 5 years ago

Steps to reproduce the issue

cd examples/python-api/
yosys -m pass.py
read_verilog ../../tests/simple/fiedler-cooley.v

Expected behavior

output of the frequency of modules as defined in pass.py

Actual behavior

crash with the following log:

terminate called after throwing an instance of 'std::out_of_range'
  what():  vector::_M_range_check: __n (which is 171) >= this->size() (which is 1)
Aborted (core dumped)

Please describe how the behavior you see differs from the expected behavior.

the comands produce an error instead of a nice graph. This might be simply because I do not know how to create a yosys pass from python but the paper mentions that this should work as well. If this is only because I did not create an shared object file, please let me know how to do so, I could not find any documentation about that.

cliffordwolf commented 5 years ago

cc @btut

btut commented 5 years ago

Sorry for the late reply. I am able to reproduce this issue. What's interesting, this does only happen if you use the installed pyosys library. I will look into it, although I am clueless about why that would make a difference! The only difference between the two ways to import is that the regular way (using the installed copy) calls init.py, but I don't see why that would have such an effect.

A workaround seems to be to use the libyosys.so file directly. You can copy it to the directory your pass sits in and change the import from from pyosys import libyosys as ys to import libyosys as ys This will not use the installed version of the library, but the one you copied.

btut commented 5 years ago

The issue seems to lie with the static variable global_refcountstorage in IdString. The size of that vector grows to 179, but at the point it is queried (at position 171) the size is back at 0. This confuses me. I cannot find any place where anything is removed from the vector. The only explanation I can come up with is that the C++ code and the Python code see two different vectors for some reason, but that does not make a lot of sense considering the workaround. Why would this work when using the local copy?

timhae commented 5 years ago

The workaround works for me as well, thanks for getting back to me :)

btut commented 5 years ago

I still have no solution for this and am running out of places to ask about it (actually I only tried the Cplusplus-sig mailing list, but that one should be very well fitting for such a question / https://mail.python.org/pipermail/cplusplus-sig/2019-July/017551.html). @daveshah1 you worked with boost::python on nextpnr, right? Do you have any idea what might cause this?

daveshah1 commented 5 years ago

I didn't see this in nextpnr because we take a different approach there, with Python more deeply embedded and automatically added to the Inittab: https://github.com/YosysHQ/nextpnr/blob/master/common/pybindings.cc#L235

The problem here is that global constructors are being called twice. I would need to dig into the dynamic linking going on a bit more to work out exactly why. I added the following to rtlil.cc:

class GlobalInitCanary {
public:
    GlobalInitCanary() {
        printf("***** GlobalInitCanary() *****\n");
    };
    ~GlobalInitCanary() {
        printf("***** ~GlobalInitCanary() *****\n");
    };
};

GlobalInitCanary global_init_canary;

With pass.py unmodified, Yosys startup prints:

***** GlobalInitCanary() *****

 /----------------------------------------------------------------------------\
 |                                                                            |
 |  yosys -- Yosys Open SYnthesis Suite                                       |
 |                                                                            |
 |  Copyright (C) 2012 - 2018  Clifford Wolf <clifford@clifford.at>           |
 |                                                                            |
 |  Permission to use, copy, modify, and/or distribute this software for any  |
 |  purpose with or without fee is hereby granted, provided that the above    |
 |  copyright notice and this permission notice appear in all copies.         |
 |                                                                            |
 |  THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES  |
 |  WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF          |
 |  MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR   |
 |  ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES    |
 |  WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN     |
 |  ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF   |
 |  OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.            |
 |                                                                            |
 \----------------------------------------------------------------------------/

 Yosys 0.8+612 (git sha1 d6a289d3, clang 8.0.0 -fPIC -Os)

***** GlobalInitCanary() *****

With the modified pass.py to use import libyosys, startup now prints:

***** GlobalInitCanary() *****

 /----------------------------------------------------------------------------\
 |                                                                            |
 |  yosys -- Yosys Open SYnthesis Suite                                       |
 |                                                                            |
 |  Copyright (C) 2012 - 2018  Clifford Wolf <clifford@clifford.at>           |
 |                                                                            |
 |  Permission to use, copy, modify, and/or distribute this software for any  |
 |  purpose with or without fee is hereby granted, provided that the above    |
 |  copyright notice and this permission notice appear in all copies.         |
 |                                                                            |
 |  THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES  |
 |  WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF          |
 |  MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR   |
 |  ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES    |
 |  WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN     |
 |  ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF   |
 |  OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.            |
 |                                                                            |
 \----------------------------------------------------------------------------/

 Yosys 0.8+612 (git sha1 d6a289d3, clang 8.0.0 -fPIC -Os)

Showing that global constructors are only called once.

daveshah1 commented 5 years ago

It seems like import libyosys is using the built-in libyosys Python module rather than doing any dynamic linking, hence not re-invoking constructors (similar to how Python scripting in nextpnr works). It seems to work correctly even if I delete every possible location of libyosys.so.

So I would suggest using just import libyosys in plugins going forward.

arcturusannamalai commented 1 year ago

I also want to add 'pass' being a Python keyword we may rename the file to something like cellstatspass.py for better reuse

YosysHQ / yosys