tango-controls / pytango

This project was moved to gitlab.com
http://pytango.rtfd.io
54 stars 44 forks source link

DS hangs when concurrently subscribing to events and destructing DeviceProxy #315

Closed reszelaz closed 2 years ago

reszelaz commented 4 years ago

Hi PyTango experts,

I would like to report an issue in PyTango (or maybe cppTango?) which affects Sardana project and is actually a big show stopper for us.

In summary: while subscribing to events in one device and destroying DeviceProxy in another one in parallel the device server hangs forever.

I think that all the necessary information and the steps to reproduce it are in https://github.com/reszelaz/test-tango. Note that this may be a similar/related issue to https://github.com/tango-controls/pytango/issues/292 so I mention here @bourtemb and @mliszcz who were involved in this investigation. Thanks in advance for looking into this as well!

If you have any questions, don't hesitate to ask.

Cheers, Zibi

mliszcz commented 4 years ago

@reszelaz Hi Zibi. Last week I've tried to contact you via email but with no success. My question is: have you tried to apply ensure_self workaround on Sardana (for both #292 and #315)? Thanks!

reszelaz commented 4 years ago

Sorry Michal, I was on a training the whole week and must have missed your email. I will take a look on it next week. Again many thanks for your help!

reszelaz commented 4 years ago

Hi @mliszcz, As @ajoubertza says in #318 and #327 the ensure_self workaround does not solve this issue.

mliszcz commented 4 years ago

Hi @reszelaz . Do you know how to reproduce the issue using Sardana test suite?

reszelaz commented 4 years ago

We don't have an automatic test that triggers this issue. It was observed at the beamline and we were able to reproduce it manually. However the example from this issue description is enough to reproduce it.

ajoubertza commented 4 years ago

Hi @reszelaz.

I was looking at this again, because I want to make a new release, and if there is a possible fix it would be good to include it.

TL;DR I don't think there is anything to fix in PyTango.

I can prevent the deadlock in the example by using a shared threading.Lock to protect the two read attribute functions in the different devices.

I guess the reason for the deadlock is similar to what we see in #292. The server handles the request from each client in a different thread, so the Python code being executed to handle each read attribute function can get interrupted after any Python bytecode instruction. In this case we don't have cyclic references, so the garbage collector isn't involved, but the DeviceProxy C++ destructors are being called as soon as the objects go out of scope. In Device1, this will cause it to unsubscribe from all events at the end of read_attr1. In Device2, we aren't subscribing in read_attr2, but the destructor will still try and unsubscribe from all events. In the C++ code unsubscribing calls ZmqEventConsumer::get_subscribed_event_ids which takes a ReaderLock.

From your backtrace, when we get the deadlock it looks like Device2 is busing with the DeviceProxy destructor, while Device1 is busy handling the callback from the event subscription. I also see the EventConsumerKeepAliveThread is trying to get a lock.

Note: In Device1.read_attr1, the subscription to the callback seem unnecessary, as we unsubscribe as soon as we exit the method, and the DeviceProxy destructor is called. Does the Sardana code really do this?

Code was modified as below, with common a new module containing lock = threading.Lock():

In Device1.py:

import common
...
    def read_attr1(self):
        with common.lock:
            dev = tango.DeviceProxy("sys/tg_test/1")
            dev.subscribe_event("double_scalar",
                                tango.EventType.ATTR_CONF_EVENT,
                                cb)
            return dev.read_attribute("double_scalar").value

In Device2.py

import common
...
    def read_attr2(self):
        with common.lock:
            tango.DeviceProxy("sys/tg_test/1")
            return time.time()
jairomoldes commented 4 years ago

I think that the problem is actually with python GIL, so this would mean a problem with pytango.

Looks like there is a deadlock between 2 threads trying to do the following:

Together with @tiagocoutinho we are trying to reimplement DeviceProxy in order to force the release of the GIL while unsubscribing. We are still trying to make it work properly. We will let you know ASAP.

Of course we would like to get this fix included in the latest release. Could you please wait a little? We would also like to help you with the release process

andygotz commented 4 years ago

Hi @jairomoldes + @tiagocoutinho , thanks for this update and help on this issue and with the release! It is highly appreciated!

ajoubertza commented 4 years ago

@jairomoldes + @tiagocoutinho Well spotted. That's very interesting. I see how the two locks are causing this now that you have pointed it out 🙂

Sure, we can wait on the release (#342). But I hope it can be done this week.

If I understand correctly, the Python code is exiting the read_attr2 method, so the reference count on the Python DeviceProxy object goes to zero. Then Python interpreter (still holding the GIL) tries to free the Boost DeviceProxy wrapper object, which calls the libtango C++ DeviceProxy destructor, which is doing the unsubscribing (trying to grab the ReaderWriteLock).

Are you planning to add a destructor to the Boost DeviceProxy C++ wrapper class and release the GIL there while calling the libtango destructor?

ajoubertza commented 4 years ago

Note that ALBA agreed not to delay the v9.3.2 release any longer for this issue.

tiagocoutinho commented 3 years ago

Hi, We (me and @jairomoldes), have tried a few options without any success. We are out of ideas. it anyone is willing, please feel free to try any solutions.

bourtemb commented 3 years ago

DeviceProxy::subscribe_event documentation says the following:

The subscribe_event() call returns an event id which has to be specified when unsubscribing from this event. Please, note that the cb parameter is a pointer. The lifetime of the pointed to object must at least be equal to the time when events are requested because only the pointer is stored into the event machinery. The same thing is true for the DeviceProxy instance on which the subscribe_event() method is called.

This means the DeviceProxy object must stay alive as long as we want to receive events. As soon as the DeviceProxy object is destroyed, the events which have been subscribed will be unsubscribed. It does not make sense to create a DeviceProxy object just for the time to subscribe to events and to destroy it immediately as in the example. I can understand this could eventually happen in case an error occurs and if the device server programmers decides that he is no longer interested in receiving events. This is the only case I can imagine where this kind of code would make sense.

bourtemb commented 3 years ago

Please also note that @mliszcz has done some work on the topic to avoid unintended unsubscriptions when using several proxies pointing to the same device: https://github.com/tango-controls/cppTango/pull/746 which should fix https://github.com/tango-controls/cppTango/issues/353 Please feel free to test it and give some feedback.

jairomoldes commented 3 years ago

Please feel free to test it and give some feedback.

I tried this but the problem is still there.

reszelaz commented 3 years ago

Thanks @tiagocoutinho, @bourtemb and @jairomoldes for investigating this issue!

It does not make sense to create a DeviceProxy object just for the time to subscribe to events and to destroy it immediately as in the example. I can understand this could eventually happen in case an error occurs and if the device server programmers decides that he is no longer interested in receiving events. This is the only case I can imagine where this kind of code would make sense.

I understand that the above example does look not very realistic when one think about programming with PyTango. In case of using disposable Taurus attributes it is however a very probable scenario - Taurus subscribes to the configuration events when constructing an attribute.

I explain you another scenario which we also suffer at ALBA. In case of the MacroServer Tango DS (part of the Sardana project) we run macros (procedures written in Python) in parallel. Within the MacroServer server we define multiple Door Tango devices and each Door can run one macro at the same time. Macros may last for short or long time, may subscribe to events and DeviceProxies may get destroyed when not necessary. To demonstrate this scenario I have changed the originally posted example - see Demo with commands and background jobs. The macros that hang at ALBA does not involve Taurus and just use PyTango.

The backtraces of the hung threads point to the same issue but better if someone more experienced confirm that.

reszelaz commented 3 years ago

To demonstrate this scenario I have changed the originally posted example - see Demo with commands and background jobs.

I have just realized that in this example I was not using the tango.EnsureOmniThread() context manager. Now I have added it in https://github.com/reszelaz/test-tango/commit/c12320775963614fec0142aaa0ddc8a7f0c763ed but the problem persist.

ajoubertza commented 3 years ago

I have just realized that in this example I was not using the tango.EnsureOmniThread() context manager. Now I have added it in reszelaz/test-tango@c123207 but the problem persist.

Thanks, I was wondering if that would make a difference. I will investigate further.

ajoubertza commented 3 years ago

I tried adding a function to be called when the C++ DeviceProxy object is to be released by the Boost extension. In that new function, destroy_proxy, I release the GIL before deleting (so calling Tango::DeviceProxy::~DeviceProxy). I still get a deadlock, but I'm not sure of the cause now. It doesn't look like GIL + map_modification_lock ReadersWritersLock. More like Tango::TangoMonitor + map_modification_lock? We also have the EventConsumerKeepAliveThread waiting for map_modification_lock, but that was in the original backtraces.

I only tried the clients that read the attributes, not the clients that use the commands.

Git diff of the relevant changes in the device_proxy.cpp file:

$ git diff ext/device_proxy.cpp 
...
@@ -594,6 +614,7 @@ namespace PyDeviceProxy
             bool stateless,
             PyTango::ExtractAs extract_as )
     {
+        std::cout << "subscribe_event_attrib" << std::endl;
         Tango::DeviceProxy& self = bopy::extract<Tango::DeviceProxy&>(py_self);
         CSequenceFromPython<StdStringVector> filters(py_filters);

@@ -605,13 +626,29 @@ namespace PyDeviceProxy
             cb->set_device(py_self);
             cb->set_extract_as(extract_as);

+            int x;
+            {
+            // 
+            // 
             AutoPythonAllowThreads guard;
-            return self.subscribe_event(attr_name, event, cb, *filters, stateless);
+            std::cout << "subscribe_event_attrib - GIL released 1" << std::endl;
+            x = self.subscribe_event(attr_name, event, cb, *filters, stateless);
+            }
+            std::cout << "subscribe_event_attrib - GIL reacquired 1" << std::endl;
+            return x;
         } else {
             event_queue_size = bopy::extract<int>(py_cb_or_queuesize);
+            int x;
+            {
+            // 
+            // 
             AutoPythonAllowThreads guard;
-            return self.subscribe_event(attr_name, event, event_queue_size,
+            std::cout << "subscribe_event_attrib - GIL released 2" << std::endl;
+            x = self.subscribe_event(attr_name, event, event_queue_size,
                                         *filters, stateless);
+            }
+            std::cout << "subscribe_event_attrib - GIL reacquired 2" << std::endl;
+            return x;
         }
     }

@@ -619,8 +656,15 @@ namespace PyDeviceProxy
     {
         // If the callback is running, unsubscribe_event will lock
         // until it finishes. So we MUST release GIL to avoid a deadlock
+        std::cout << "unsubscribe_event - proxy" << std::endl;
+        {
+        //
+        //
         AutoPythonAllowThreads guard;
+        std::cout << "unsubscribe_event - GIL released" << std::endl;
         self.unsubscribe_event(event);
+        }
+        std::cout << "unsubscribe_event - done (have GIL)" << std::endl;
     }

     template<typename ED, typename EDList>
@@ -699,6 +743,19 @@ namespace PyDeviceProxy
         return get_events__aux<Tango::DevIntrChangeEventData, Tango::DevIntrChangeEventDataList>(py_self, event_id, extract_as);
     }

+    void destroy_proxy(Tango::DeviceProxy* proxy)
+    {
+    std::cout << "destroy_proxy() " << proxy << std::endl;
+        {
+        // 
+        //
+        AutoPythonAllowThreads guard;
+        std::cout << "destroy_proxy - GIL released" << std::endl;
+        delete proxy;
+        }
+    std::cout << "destroy_proxy() done." << std::endl;
+    }
+
     static boost::shared_ptr<Tango::DeviceProxy> makeDeviceProxy1(const std::string& name)
     {
     Tango::DeviceProxy* dp = NULL;
@@ -706,7 +763,10 @@ namespace PyDeviceProxy
         AutoPythonAllowThreads guard;
         dp = new Tango::DeviceProxy(name.c_str());
     }
-        return boost::shared_ptr<Tango::DeviceProxy>(dp);
+    auto x = boost::shared_ptr<Tango::DeviceProxy>(dp, &PyDeviceProxy::destroy_proxy);
+    std::cout << "makeDeviceProxy1 - made new Tango::DeviceProxy, id " << x << std::endl;
+    return x;
+//        return boost::shared_ptr<Tango::DeviceProxy>(dp);
     }

     static boost::shared_ptr<Tango::DeviceProxy> makeDeviceProxy2(const std::string& name, bool b)

Output from server, showing some new output. It was busy with a subscribe event, and the garbage collection triggered the clean up of many old device proxy instances. Both the subscription and device proxy cleanup threads had released the GIL. (That makes me wonder if it is safe to release the GIL while Python is doing a garbage collection cycle...)

$ python3 DeviceServer.py test
...
subscribe_event_attrib
subscribe_event_attrib - GIL released 1
subscribe_event_attrib - GIL reacquired 1
makeDeviceProxy1 - made new Tango::DeviceProxy, id0x7f909801e080
makeDeviceProxy1 - made new Tango::DeviceProxy, id0x7f90a4085440
subscribe_event_attrib
subscribe_event_attrib - GIL released 1
subscribe_event_attrib - GIL reacquired 1
makeDeviceProxy1 - made new Tango::DeviceProxy, id0x7f909801cf90
makeDeviceProxy1 - made new Tango::DeviceProxy, id0x7f90a4085c00
subscribe_event_attrib
subscribe_event_attrib - GIL released 1
destroy_proxy() 0x7f90a40627a0
destroy_proxy - GIL released
destroy_proxy() done.
...
destroy_proxy() 0x7f909801cf90
destroy_proxy - GIL released
destroy_proxy() done.
makeDeviceProxy1 - made new Tango::DeviceProxy, id0x7f90980080e0
...
makeDeviceProxy1 - made new Tango::DeviceProxy, id0x7f909801faa0
destroy_proxy() 0x7f90980080e0
destroy_proxy - GIL released

Sorry for super-long comment, but I didn't want to cut the back traces:

(env-py3.8-tango9.3.4) root@aef7cf5ec9b3:/opt/project# gdb -p 2501
GNU gdb (Debian 8.2.1-2+b3) 8.2.1
Copyright (C) 2018 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Type "show copying" and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
    <http://www.gnu.org/software/gdb/documentation/>.

For help, type "help".
Type "apropos word" to search for commands related to "word".
Attaching to process 2501
[New LWP 2503]
[New LWP 2504]
[New LWP 2505]
[New LWP 2506]
[New LWP 2507]
[New LWP 2508]
[New LWP 2509]
[New LWP 2513]
[New LWP 2515]
[New LWP 2516]
[New LWP 2517]
[New LWP 2518]
[New LWP 2522]
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
futex_wait_cancelable (private=0, expected=0, futex_word=0x55783e3343c0) at ../sysdeps/unix/sysv/linux/futex-internal.h:88
88  ../sysdeps/unix/sysv/linux/futex-internal.h: No such file or directory.
(gdb) thread apply all bt

Thread 14 (Thread 0x7f90ad7fa700 (LWP 2522)):
#0  futex_wait_cancelable (private=0, expected=0, futex_word=0x7f90cf5b0198 <Tango::EventConsumer::map_modification_lock+88>)
    at ../sysdeps/unix/sysv/linux/futex-internal.h:88
#1  __pthread_cond_wait_common (abstime=0x0, mutex=0x7f90cf5b0140 <Tango::EventConsumer::map_modification_lock>,
    cond=0x7f90cf5b0170 <Tango::EventConsumer::map_modification_lock+48>) at pthread_cond_wait.c:502
#2  __pthread_cond_wait (cond=0x7f90cf5b0170 <Tango::EventConsumer::map_modification_lock+48>,
    mutex=0x7f90cf5b0140 <Tango::EventConsumer::map_modification_lock>) at pthread_cond_wait.c:655
#3  0x00007f90ce77e3d1 in omni_condition::wait() () from /opt/conda/envs/env-py3.8-tango9.3.4/lib/./libomnithread.so.4
#4  0x00007f90cef15065 in ReadersWritersLock::readerIn (this=0x7f90cf5b0140 <Tango::EventConsumer::map_modification_lock>)
    at /usr/local/src/conda/cpptango-9.3.4/cppapi/server/readers_writers_lock.h:66
#5  0x00007f90cef152d4 in ReaderLock::ReaderLock (this=0x7f90ad7f78b8, l=...)
    at /usr/local/src/conda/cpptango-9.3.4/cppapi/server/readers_writers_lock.h:176
#6  0x00007f90cf0ea8de in Tango::ZmqEventConsumer::get_subscribed_event_ids (this=0x7f90a400c290, _dev=0x7f90980080e0, _ids=...)
    at /usr/local/src/conda/cpptango-9.3.4/cppapi/client/zmqeventconsumer.cpp:3461
#7  0x00007f90cef90449 in Tango::DeviceProxy::unsubscribe_all_events (this=0x7f90980080e0)
    at /usr/local/src/conda/cpptango-9.3.4/cppapi/client/devapi_base.cpp:2679
#8  0x00007f90cef90252 in Tango::DeviceProxy::~DeviceProxy (this=0x7f90980080e0, __in_chrg=<optimized out>)
    at /usr/local/src/conda/cpptango-9.3.4/cppapi/client/devapi_base.cpp:2642
#9  0x00007f90cef9036d in Tango::DeviceProxy::~DeviceProxy (this=0x7f90980080e0, __in_chrg=<optimized out>)
    at /usr/local/src/conda/cpptango-9.3.4/cppapi/client/devapi_base.cpp:2670
#10 0x00007f90cfc74797 in PyDeviceProxy::destroy_proxy (proxy=0x7f90980080e0) at /opt/project/ext/device_proxy.cpp:754
#11 0x00007f90cfcbb1dd in boost::detail::sp_counted_impl_pd<Tango::DeviceProxy*, void (*)(Tango::DeviceProxy*)>::dispose (this=0x7f9098017510)
    at /opt/conda/envs/env-py3.8-tango9.3.4/include/boost/smart_ptr/detail/sp_counted_impl.hpp:173
#12 0x00007f90cfb2694a in boost::detail::sp_counted_base::release (this=0x7f90cfb2694a <boost::detail::sp_counted_base::release()+26>)
    at /opt/conda/envs/env-py3.8-tango9.3.4/include/boost/smart_ptr/detail/sp_counted_base_std_atomic.hpp:110
#13 0x00007f90cfc6d6c6 in boost::detail::shared_count::~shared_count (this=0x7f90bfd5ffa8, __in_chrg=<optimized out>)
    at /opt/conda/envs/env-py3.8-tango9.3.4/include/boost/smart_ptr/detail/shared_count.hpp:427
#14 0x00007f90cfc74829 in boost::shared_ptr<Tango::DeviceProxy>::~shared_ptr (this=0x7f90bfd5ffa0, __in_chrg=<optimized out>)
    at /opt/conda/envs/env-py3.8-tango9.3.4/include/boost/smart_ptr/shared_ptr.hpp:341
#15 0x00007f90cfcde027 in boost::python::objects::pointer_holder<boost::shared_ptr<Tango::DeviceProxy>, Tango::DeviceProxy>::~pointer_holder (
    this=0x7f90bfd5ff90, __in_chrg=<optimized out>) at /opt/conda/envs/env-py3.8-tango9.3.4/include/boost/python/object/pointer_holder.hpp:51
#16 0x00007f90ce9c485c in instance_dealloc () from /opt/conda/envs/env-py3.8-tango9.3.4/lib/libboost_python38.so.1.73.0
#17 0x000055783cbed35d in subtype_dealloc (self=<optimized out>) at /tmp/build/80754af9/python_1599203911753/work/Objects/typeobject.c:1192
#18 0x000055783cb49746 in _Py_DECREF () at /tmp/build/80754af9/python_1599203911753/work/Include/object.h:478
#19 frame_dealloc (f=0x7f90bc4f4580) at /tmp/build/80754af9/python_1599203911753/work/Objects/frameobject.c:430
#20 0x000055783cb4991b in _Py_DECREF () at /tmp/build/80754af9/python_1599203911753/work/Include/object.h:478
#21 _Py_XDECREF () at /tmp/build/80754af9/python_1599203911753/work/Include/object.h:541
#22 frame_dealloc (f=0x7f90bc4f6040) at /tmp/build/80754af9/python_1599203911753/work/Objects/frameobject.c:438
#23 0x000055783cc1176a in _Py_DECREF () at /tmp/build/80754af9/python_1599203911753/work/Include/object.h:478
--Type <RET> for more, q to quit, c to continue without paging--c
#24 tb_clear (tb=0x7f90bfd52a80) at /tmp/build/80754af9/python_1599203911753/work/Python/traceback.c:185
#25 0x000055783cb69255 in delete_garbage.isra.26 (old=0x55783cda0528 <_PyRuntime+392>, collectable=0x7f90ad7f7c10) at /tmp/build/80754af9/python_1599203911753/work/Modules/gcmodule.c:948
#26 collect (generation=<optimized out>, n_collected=0x7f90ad7f7cf8, n_uncollectable=0x7f90ad7f7d00, nofail=0, state=0x55783cda04f8 <_PyRuntime+344>) at /tmp/build/80754af9/python_1599203911753/work/Modules/gcmodule.c:1123
#27 0x000055783cc1cd35 in collect_with_callback (generation=0, state=0x55783cda04f8 <_PyRuntime+344>) at /tmp/build/80754af9/python_1599203911753/work/Modules/gcmodule.c:1240
#28 0x000055783cb69998 in _PyObject_GC_Alloc (basicsize=<optimized out>, use_calloc=0) at /tmp/build/80754af9/python_1599203911753/work/Modules/gcmodule.c:1977
#29 _PyObject_GC_Malloc (basicsize=<optimized out>) at /tmp/build/80754af9/python_1599203911753/work/Modules/gcmodule.c:1987
#30 0x000055783cbda6fd in PyType_GenericAlloc (nitems=0, type=0x55783cd8c940 <_PyExc_KeyError>) at /tmp/build/80754af9/python_1599203911753/work/Objects/typeobject.c:1015
#31 BaseException_new (type=0x55783cd8c940 <_PyExc_KeyError>, args=0x7f90bc4cf670, kwds=<optimized out>) at /tmp/build/80754af9/python_1599203911753/work/Objects/exceptions.c:40
#32 0x000055783cb6a585 in type_call (type=0x55783cd8c940 <_PyExc_KeyError>, args=0x7f90bc4cf670, kwds=0x0) at /tmp/build/80754af9/python_1599203911753/work/Objects/typeobject.c:974
#33 0x000055783cb6a200 in PyObject_Call (callable=0x55783cd8c940 <_PyExc_KeyError>, args=<optimized out>, kwargs=<optimized out>) at /tmp/build/80754af9/python_1599203911753/work/Objects/call.c:245
#34 0x000055783cb6a456 in _PyErr_CreateException (value=0x7f90bc4cf670, exception=0x55783cd8c940 <_PyExc_KeyError>) at /tmp/build/80754af9/python_1599203911753/work/Python/errors.c:93
#35 _PyErr_NormalizeException (tstate=0x7f9098005620, exc=0x7f90ad7f7eb8, val=0x7f90ad7f7ec0, tb=0x7f90ad7f7ec8) at /tmp/build/80754af9/python_1599203911753/work/Python/errors.c:327
#36 0x000055783cbf276a in _PyEval_EvalFrameDefault (f=<optimized out>, throwflag=<optimized out>) at /tmp/build/80754af9/python_1599203911753/work/Python/ceval.c:3756
#37 0x000055783cbb8c0b in function_code_fastcall (globals=<optimized out>, nargs=1, args=<optimized out>, co=<optimized out>) at /tmp/build/80754af9/python_1599203911753/work/Objects/call.c:283
#38 _PyFunction_Vectorcall (kwnames=<optimized out>, nargsf=<optimized out>, stack=0x7f90bc4851e8, func=0x7f90bfe37940) at /tmp/build/80754af9/python_1599203911753/work/Objects/call.c:410
#39 _PyObject_Vectorcall (kwnames=<optimized out>, nargsf=<optimized out>, args=0x7f90bc4851e8, callable=0x7f90bfe37940) at /tmp/build/80754af9/python_1599203911753/work/Include/cpython/abstract.h:127
#40 method_vectorcall (method=<optimized out>, args=0x7f90bc4851f0, nargsf=<optimized out>, kwnames=<optimized out>) at /tmp/build/80754af9/python_1599203911753/work/Objects/classobject.c:60
#41 0x000055783cb2d77f in _PyObject_Vectorcall (kwnames=0x0, nargsf=<optimized out>, args=0x7f90bc4851f0, callable=0x7f90d01b2400) at /tmp/build/80754af9/python_1599203911753/work/Include/cpython/abstract.h:127
#42 call_function (kwnames=0x0, oparg=<optimized out>, pp_stack=<synthetic pointer>, tstate=0x7f9098005620) at /tmp/build/80754af9/python_1599203911753/work/Python/ceval.c:4963
#43 _PyEval_EvalFrameDefault (f=<optimized out>, throwflag=<optimized out>) at /tmp/build/80754af9/python_1599203911753/work/Python/ceval.c:3469
#44 0x000055783cbb886b in function_code_fastcall (globals=<optimized out>, nargs=3, args=<optimized out>, co=<optimized out>) at /tmp/build/80754af9/python_1599203911753/work/Objects/call.c:283
#45 _PyFunction_Vectorcall (func=<optimized out>, stack=0x7f90ad7f8170, nargsf=<optimized out>, kwnames=<optimized out>) at /tmp/build/80754af9/python_1599203911753/work/Objects/call.c:410
#46 0x000055783cae70de in _PyObject_Vectorcall (kwnames=0x0, nargsf=3, args=0x7f90ad7f8170, callable=0x7f90bfe450d0) at /tmp/build/80754af9/python_1599203911753/work/Include/cpython/abstract.h:127
#47 _PyObject_FastCall () at /tmp/build/80754af9/python_1599203911753/work/Include/cpython/abstract.h:147
#48 _PyObject_FastCall_Prepend (callable=0x7f90bfe450d0, obj=<optimized out>, args=<optimized out>, nargs=3) at /tmp/build/80754af9/python_1599203911753/work/Objects/call.c:849
#49 0x000055783cbabd6c in call_unbound (nargs=2, args=0x7f90ad7f81e0, self=0x7f90bc4c7f70, func=0x7f90bfe450d0, unbound=1) at /tmp/build/80754af9/python_1599203911753/work/Objects/typeobject.c:1453
#50 call_method (nargs=2, args=0x7f90ad7f81e0, name=0x55783cd82be0 <PyId___setattr__.15136>, obj=0x7f90bc4c7f70) at /tmp/build/80754af9/python_1599203911753/work/Objects/typeobject.c:1485
#51 slot_tp_setattro (self=0x7f90bc4c7f70, name=<optimized out>, value=<optimized out>) at /tmp/build/80754af9/python_1599203911753/work/Objects/typeobject.c:6656
#52 0x000055783cba86a8 in PyObject_SetAttr (v=0x7f90bc4c7f70, name=<optimized out>, value=0x55783cd9d360 <_Py_NoneStruct>) at /tmp/build/80754af9/python_1599203911753/work/Objects/object.c:1045
#53 0x000055783cbee6b7 in _PyEval_EvalFrameDefault (f=<optimized out>, throwflag=<optimized out>) at /tmp/build/80754af9/python_1599203911753/work/Python/ceval.c:2372
#54 0x000055783cbb7a92 in _PyEval_EvalCodeWithName (_co=0x7f90bfe2f660, globals=<optimized out>, locals=<optimized out>, args=<optimized out>, argcount=<optimized out>, kwnames=0x0, kwargs=0x7f90ad7f8500, kwcount=<optimized out>, kwstep=1, defs=0x0, defcount=0, kwdefs=0x0, closure=0x0, name=0x7f90bfe2d670, qualname=0x7f90bfe2d670) at /tmp/build/80754af9/python_1599203911753/work/Python/ceval.c:4298
#55 0x000055783cbb948c in _PyFunction_Vectorcall (kwnames=0x0, nargsf=<optimized out>, stack=0x7f90ad7f84f0, func=0x7f90bfe37af0) at /tmp/build/80754af9/python_1599203911753/work/Objects/call.c:435
#56 _PyObject_FastCallDict (callable=0x7f90bfe37af0, args=0x7f90ad7f84f0, nargsf=<optimized out>, kwargs=<optimized out>) at /tmp/build/80754af9/python_1599203911753/work/Objects/call.c:96
#57 0x000055783cbb9733 in _PyObject_Call_Prepend (callable=0x7f90bfe37af0, obj=<optimized out>, args=0x7f90bc4cf8e0, kwargs=0x0) at /tmp/build/80754af9/python_1599203911753/work/Objects/call.c:887
#58 0x000055783cbb98ca in slot_tp_init (self=0x7f90bc4c7f70, args=0x7f90bc4cf8e0, kwds=0x0) at /tmp/build/80754af9/python_1599203911753/work/Objects/typeobject.c:6787
#59 0x000055783cb6a7d4 in type_call (kwds=0x0, args=0x7f90bc4cf8e0, type=<optimized out>) at /tmp/build/80754af9/python_1599203911753/work/Objects/typeobject.c:994
#60 _PyObject_MakeTpCall (callable=0x55783e2c2ef0, args=<optimized out>, nargs=<optimized out>, keywords=<optimized out>) at /tmp/build/80754af9/python_1599203911753/work/Objects/call.c:159
#61 0x000055783cbf1f56 in _PyObject_Vectorcall (kwnames=0x0, nargsf=<optimized out>, args=0x7f90bc483358, callable=0x55783e2c2ef0) at /tmp/build/80754af9/python_1599203911753/work/Include/cpython/abstract.h:125
#62 call_function (kwnames=0x0, oparg=<optimized out>, pp_stack=<synthetic pointer>, tstate=0x7f9098005620) at /tmp/build/80754af9/python_1599203911753/work/Python/ceval.c:4963
#63 _PyEval_EvalFrameDefault (f=<optimized out>, throwflag=<optimized out>) at /tmp/build/80754af9/python_1599203911753/work/Python/ceval.c:3469
#64 0x000055783cbb886b in function_code_fastcall (globals=<optimized out>, nargs=0, args=<optimized out>, co=<optimized out>) at /tmp/build/80754af9/python_1599203911753/work/Objects/call.c:283
#65 _PyFunction_Vectorcall (func=<optimized out>, stack=0x7f90bc481fb8, nargsf=<optimized out>, kwnames=<optimized out>) at /tmp/build/80754af9/python_1599203911753/work/Objects/call.c:410
#66 0x000055783cb2bb84 in _PyObject_Vectorcall (kwnames=0x0, nargsf=<optimized out>, args=0x7f90bc481fb8, callable=0x7f90bfd30700) at /tmp/build/80754af9/python_1599203911753/work/Include/cpython/abstract.h:127
#67 call_function (kwnames=0x0, oparg=<optimized out>, pp_stack=<synthetic pointer>, tstate=0x7f9098005620) at /tmp/build/80754af9/python_1599203911753/work/Python/ceval.c:4963
#68 _PyEval_EvalFrameDefault (f=<optimized out>, throwflag=<optimized out>) at /tmp/build/80754af9/python_1599203911753/work/Python/ceval.c:3500
#69 0x000055783cbb886b in function_code_fastcall (globals=<optimized out>, nargs=1, args=<optimized out>, co=<optimized out>) at /tmp/build/80754af9/python_1599203911753/work/Objects/call.c:283
#70 _PyFunction_Vectorcall (func=<optimized out>, stack=0x7f90bc4cf8c8, nargsf=<optimized out>, kwnames=<optimized out>) at /tmp/build/80754af9/python_1599203911753/work/Objects/call.c:410
#71 0x000055783cb6a041 in PyVectorcall_Call (callable=0x7f90bfd41160, tuple=<optimized out>, kwargs=<optimized out>) at /tmp/build/80754af9/python_1599203911753/work/Objects/call.c:199
#72 0x000055783cbef99b in do_call_core (kwdict=0x7f90bfd4a880, callargs=0x7f90bc4cf8b0, func=0x7f90bfd41160, tstate=<optimized out>) at /tmp/build/80754af9/python_1599203911753/work/Python/ceval.c:5010
#73 _PyEval_EvalFrameDefault (f=<optimized out>, throwflag=<optimized out>) at /tmp/build/80754af9/python_1599203911753/work/Python/ceval.c:3559
#74 0x000055783cbb7a92 in _PyEval_EvalCodeWithName (_co=0x7f90bfe2b450, globals=<optimized out>, locals=<optimized out>, args=<optimized out>, argcount=<optimized out>, kwnames=0x0, kwargs=0x7f90bc4823c0, kwcount=<optimized out>, kwstep=1, defs=0x0, defcount=0, kwdefs=0x0, closure=0x0, name=0x7f90ccd3d9f0, qualname=0x7f90bfe27ad0) at /tmp/build/80754af9/python_1599203911753/work/Python/ceval.c:4298
#75 0x000055783cbb8943 in _PyFunction_Vectorcall (func=<optimized out>, stack=0x7f90bc4823a8, nargsf=<optimized out>, kwnames=<optimized out>) at /tmp/build/80754af9/python_1599203911753/work/Objects/call.c:435
#76 0x000055783cb2d75e in _PyObject_Vectorcall (kwnames=0x0, nargsf=<optimized out>, args=0x7f90bc4823a8, callable=0x7f90bfe25a60) at /tmp/build/80754af9/python_1599203911753/work/Include/cpython/abstract.h:127
#77 call_function (kwnames=0x0, oparg=<optimized out>, pp_stack=<synthetic pointer>, tstate=0x7f9098005620) at /tmp/build/80754af9/python_1599203911753/work/Python/ceval.c:4963
#78 _PyEval_EvalFrameDefault (f=<optimized out>, throwflag=<optimized out>) at /tmp/build/80754af9/python_1599203911753/work/Python/ceval.c:3486
#79 0x000055783cbb7e19 in _PyEval_EvalCodeWithName (_co=0x7f90bfd60030, globals=<optimized out>, locals=<optimized out>, args=<optimized out>, argcount=<optimized out>, kwnames=0x0, kwargs=0x7f90ad7f8e20, kwcount=<optimized out>, kwstep=1, defs=0x0, defcount=0, kwdefs=0x0, closure=0x7f90bfd401f0, name=0x7f90bfd3e1b0, qualname=0x7f90bfd3f1c0) at /tmp/build/80754af9/python_1599203911753/work/Python/ceval.c:4298
#80 0x000055783cbb8943 in _PyFunction_Vectorcall (func=<optimized out>, stack=0x7f90ad7f8e10, nargsf=<optimized out>, kwnames=<optimized out>) at /tmp/build/80754af9/python_1599203911753/work/Objects/call.c:435
#81 0x000055783cbb8e79 in _PyObject_Vectorcall (kwnames=0x0, nargsf=2, args=0x7f90ad7f8e10, callable=0x7f90bfd413a0) at /tmp/build/80754af9/python_1599203911753/work/Include/cpython/abstract.h:127
#82 method_vectorcall (method=<optimized out>, args=<optimized out>, nargsf=<optimized out>, kwnames=<optimized out>) at /tmp/build/80754af9/python_1599203911753/work/Objects/classobject.c:89
#83 0x000055783caaaf36 in _PyObject_Vectorcall (callable=0x7f90d0116480, args=<optimized out>, nargsf=<optimized out>, kwnames=<optimized out>) at /tmp/build/80754af9/python_1599203911753/work/Include/cpython/abstract.h:127
#84 0x000055783cb02950 in _PyObject_FastCall () at /tmp/build/80754af9/python_1599203911753/work/Include/cpython/abstract.h:147
#85 _PyObject_CallFunctionVa (callable=0x7f90d0116480, format=<optimized out>, va=<optimized out>, is_size_t=<optimized out>) at /tmp/build/80754af9/python_1599203911753/work/Objects/call.c:935
#86 0x000055783cb02add in PyEval_CallMethod (obj=<optimized out>, name=<optimized out>, format=0x7f90cfe6275b "(O)") at /tmp/build/80754af9/python_1599203911753/work/Objects/call.c:1055
#87 0x00007f90cfd3d50a in PyAttr::read(Tango::DeviceImpl*, Tango::Attribute&) () from /opt/project/tango/_tango.cpython-38-x86_64-linux-gnu.so

#88 0x00007f90cf1c71cd in Tango::Device_3Impl::read_attributes_no_except (this=0x55783e3fb170, names=..., aid=..., second_try=true, idx=...) at /usr/local/src/conda/cpptango-9.3.4/cppapi/server/device_3.cpp:539
#89 0x00007f90cf1dbfc0 in Tango::Device_5Impl::read_attributes_5 (this=0x55783e3fb170, names=..., source=Tango::CACHE_DEV, cl_id=...) at /usr/local/src/conda/cpptango-9.3.4/cppapi/server/device_5.cpp:338
#90 0x00007f90cf108b12 in _0RL_lcfn_6fe2f94a21a10053_84000000 (cd=0x7f90ad7f9a40, svnt=0x55783e3fb9b8) at /usr/local/src/conda/cpptango-9.3.4/cppapi/server/idl/tangoSK.cpp:6494
#91 0x00007f90ceae0f44 in omniCallHandle::upcall(omniServant*, omniCallDescriptor&) () from /opt/conda/envs/env-py3.8-tango9.3.4/lib/libomniORB4.so.2
#92 0x00007f90cf10bafa in Tango::_impl_Device_5::_dispatch (this=0x55783e3fb9b0, _handle=...) at /usr/local/src/conda/cpptango-9.3.4/cppapi/server/idl/tangoSK.cpp:7416
#93 0x00007f90cead9c66 in omni::omniOrbPOA::dispatch(omniCallHandle&, omniLocalIdentity*) () from /opt/conda/envs/env-py3.8-tango9.3.4/lib/libomniORB4.so.2
#94 0x00007f90ceab0028 in omniLocalIdentity::dispatch(omniCallHandle&) () from /opt/conda/envs/env-py3.8-tango9.3.4/lib/libomniORB4.so.2
#95 0x00007f90ceb037b3 in omni::GIOP_S::handleRequest() () from /opt/conda/envs/env-py3.8-tango9.3.4/lib/libomniORB4.so.2
#96 0x00007f90ceb04819 in omni::GIOP_S::dispatcher() () from /opt/conda/envs/env-py3.8-tango9.3.4/lib/libomniORB4.so.2
#97 0x00007f90ceb0052c in omni::giopWorker::execute() () from /opt/conda/envs/env-py3.8-tango9.3.4/lib/libomniORB4.so.2
#98 0x00007f90ceaa0279 in omniAsyncWorker::real_run() () from /opt/conda/envs/env-py3.8-tango9.3.4/lib/libomniORB4.so.2
#99 0x00007f90ceaa06d0 in omniAsyncPoolServer::workerRun(omniAsyncWorker*) () from /opt/conda/envs/env-py3.8-tango9.3.4/lib/libomniORB4.so.2
#100 0x00007f90cea9fcc1 in omniAsyncWorker::mid_run() () from /opt/conda/envs/env-py3.8-tango9.3.4/lib/libomniORB4.so.2
#101 0x00007f90cf387044 in Tango::create_PyPerThData (info=...) at /usr/local/src/conda/cpptango-9.3.4/cppapi/server/utils.cpp:3255
#102 0x00007f90ceaa0176 in omniAsyncWorker::run(void*) () from /opt/conda/envs/env-py3.8-tango9.3.4/lib/libomniORB4.so.2
#103 0x00007f90ce77ef48 in omni_thread_wrapper () from /opt/conda/envs/env-py3.8-tango9.3.4/lib/./libomnithread.so.4
#104 0x00007f90d059efa3 in start_thread (arg=<optimized out>) at pthread_create.c:486
#105 0x00007f90d04cf4cf in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95

Thread 13 (Thread 0x7f90adffb700 (LWP 2518)):
#0  0x00007f90d04cf7ef in epoll_wait (epfd=29, events=0x7f90adffa1f0, maxevents=256, timeout=-1) at ../sysdeps/unix/sysv/linux/epoll_wait.c:30
#1  0x00007f90ce7abe7d in zmq::epoll_t::loop() () from /opt/conda/envs/env-py3.8-tango9.3.4/lib/./libzmq.so.5
#2  0x00007f90ce7e1bcf in thread_routine () from /opt/conda/envs/env-py3.8-tango9.3.4/lib/./libzmq.so.5
#3  0x00007f90d059efa3 in start_thread (arg=<optimized out>) at pthread_create.c:486
#4  0x00007f90d04cf4cf in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95

Thread 12 (Thread 0x7f90ae7fc700 (LWP 2517)):
#0  0x00007f90d04cf7ef in epoll_wait (epfd=27, events=0x7f90ae7fb1f0, maxevents=256, timeout=-1) at ../sysdeps/unix/sysv/linux/epoll_wait.c:30
#1  0x00007f90ce7abe7d in zmq::epoll_t::loop() () from /opt/conda/envs/env-py3.8-tango9.3.4/lib/./libzmq.so.5
#2  0x00007f90ce7e1bcf in thread_routine () from /opt/conda/envs/env-py3.8-tango9.3.4/lib/./libzmq.so.5
#3  0x00007f90d059efa3 in start_thread (arg=<optimized out>) at pthread_create.c:486
#4  0x00007f90d04cf4cf in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95

Thread 11 (Thread 0x7f90aeffd700 (LWP 2516)):
#0  0x00007f90d04c4819 in __GI___poll (fds=0x7f90aeffcae0, nfds=1, timeout=-1) at ../sysdeps/unix/sysv/linux/poll.c:29
#1  0x00007f90ce7f07da in zmq_poll () from /opt/conda/envs/env-py3.8-tango9.3.4/lib/./libzmq.so.5
#2  0x00007f90cf0dc31d in zmq::poll (items_=0x7f909c0039d0, nitems_=1, timeout_=-1) at /usr/local/src/conda-prefix/include/zmq.hpp:291
#3  0x00007f90cf0de070 in Tango::ZmqEventConsumer::run_undetached (this=0x7f90a400c290, arg=0x55783e033490) at /usr/local/src/conda/cpptango-9.3.4/cppapi/client/zmqeventconsumer.cpp:220
#4  0x00007f90ce77ef38 in omni_thread_wrapper () from /opt/conda/envs/env-py3.8-tango9.3.4/lib/./libomnithread.so.4
#5  0x00007f90d059efa3 in start_thread (arg=<optimized out>) at pthread_create.c:486
#6  0x00007f90d04cf4cf in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95

Thread 10 (Thread 0x7f90af7fe700 (LWP 2515)):
#0  futex_wait_cancelable (private=0, expected=0, futex_word=0x7f90cf5b0198 <Tango::EventConsumer::map_modification_lock+88>) at ../sysdeps/unix/sysv/linux/futex-internal.h:88
#1  __pthread_cond_wait_common (abstime=0x0, mutex=0x7f90cf5b0140 <Tango::EventConsumer::map_modification_lock>, cond=0x7f90cf5b0170 <Tango::EventConsumer::map_modification_lock+48>) at pthread_cond_wait.c:502
#2  __pthread_cond_wait (cond=0x7f90cf5b0170 <Tango::EventConsumer::map_modification_lock+48>, mutex=0x7f90cf5b0140 <Tango::EventConsumer::map_modification_lock>) at pthread_cond_wait.c:655
#3  0x00007f90ce77e3d1 in omni_condition::wait() () from /opt/conda/envs/env-py3.8-tango9.3.4/lib/./libomnithread.so.4
#4  0x00007f90cef15065 in ReadersWritersLock::readerIn (this=0x7f90cf5b0140 <Tango::EventConsumer::map_modification_lock>) at /usr/local/src/conda/cpptango-9.3.4/cppapi/server/readers_writers_lock.h:66
#5  0x00007f90cef152d4 in ReaderLock::ReaderLock (this=0x7f90af7fdca8, l=...) at /usr/local/src/conda/cpptango-9.3.4/cppapi/server/readers_writers_lock.h:176
#6  0x00007f90cf0c0057 in Tango::EventConsumerKeepAliveThread::run_undetached (this=0x7f90a400bd60, arg=0x0) at /usr/local/src/conda/cpptango-9.3.4/cppapi/client/eventkeepalive.cpp:590
#7  0x00007f90ce77ef38 in omni_thread_wrapper () from /opt/conda/envs/env-py3.8-tango9.3.4/lib/./libomnithread.so.4
#8  0x00007f90d059efa3 in start_thread (arg=<optimized out>) at pthread_create.c:486
#9  0x00007f90d04cf4cf in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95

Thread 9 (Thread 0x7f90affff700 (LWP 2513)):
#0  __lll_lock_wait () at ../sysdeps/unix/sysv/linux/x86_64/lowlevellock.S:103
#1  0x00007f90d05a1714 in __GI___pthread_mutex_lock (mutex=0x7f90a4065c00) at ../nptl/pthread_mutex_lock.c:80
#2  0x00007f90cef03581 in omni_mutex::lock (this=0x7f90a4065c00) at /usr/local/src/conda-prefix/include/omnithread.h:255
#3  0x00007f90cef035ca in omni_mutex_lock::omni_mutex_lock (this=0x7f90afffb1c0, m=...) at /usr/local/src/conda-prefix/include/omnithread.h:299
#4  0x00007f90cf0a2547 in Tango::TangoMonitor::rel_monitor (this=0x7f90a4065c00) at /usr/local/src/conda/cpptango-9.3.4/cppapi/server/tango_monitor.h:180
#5  0x00007f90cf0a286b in Tango::AutoTangoMonitor::~AutoTangoMonitor (this=0x7f90afffb350, __in_chrg=<optimized out>) at /usr/local/src/conda/cpptango-9.3.4/cppapi/server/auto_tango_monitor.h:115
#6  0x00007f90cf0af7cf in Tango::EventConsumer::get_fire_sync_event (this=0x7f90a400c290, device=0x7f90a4085c00, callback=0x7f90bc4c7440, ev_queue=0x0, event=Tango::ATTR_CONF_EVENT, event_name=..., obj_name=..., cb=..., callback_key=...) at /usr/local/src/conda/cpptango-9.3.4/cppapi/client/event.cpp:3264
#7  0x00007f90cf0a8d83 in Tango::EventConsumer::connect_event (this=0x7f90a400c290, device=0x7f90a4085c00, obj_name=..., event=Tango::ATTR_CONF_EVENT, callback=0x7f90bc4c7440, ev_queue=0x0, filters=..., event_name=..., event_id=0) at /usr/local/src/conda/cpptango-9.3.4/cppapi/client/event.cpp:1611
#8  0x00007f90cf0a6e6a in Tango::EventConsumer::subscribe_event (this=0x7f90a400c290, device=0x7f90a4085c00, attribute=..., event=Tango::ATTR_CONF_EVENT, callback=0x7f90bc4c7440, ev_queue=0x0, filters=..., stateless=false) at /usr/local/src/conda/cpptango-9.3.4/cppapi/client/event.cpp:1250
#9  0x00007f90cf0a6a7c in Tango::EventConsumer::subscribe_event (this=0x7f90a400c290, device=0x7f90a4085c00, attribute=..., event=Tango::ATTR_CONF_EVENT, callback=0x7f90bc4c7440, filters=..., stateless=false) at /usr/local/src/conda/cpptango-9.3.4/cppapi/client/event.cpp:1129
#10 0x00007f90cefd4a7d in Tango::DeviceProxy::subscribe_event (this=0x7f90a4085c00, attr_name=..., event=Tango::ATTR_CONF_EVENT, callback=0x7f90bc4c7440, filters=..., stateless=false) at /usr/local/src/conda/cpptango-9.3.4/cppapi/client/devapi_base.cpp:8175
#11 0x00007f90cfc7405d in PyDeviceProxy::subscribe_event_attrib (py_self=..., attr_name=..., event=Tango::ATTR_CONF_EVENT, py_cb_or_queuesize=..., py_filters=..., stateless=false, extract_as=PyTango::ExtractAsNumpy) at /opt/project/ext/device_proxy.cpp:635
#12 0x00007f90cfcc9426 in boost::python::detail::invoke<boost::python::to_python_value<int const&>, int (*)(boost::python::api::object, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, Tango::EventType, boost::python::api::object, boost::python::api::object&, bool, PyTango::ExtractAs), boost::python::arg_from_python<boost::python::api::object>, boost::python::arg_from_python<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&>, boost::python::arg_from_python<Tango::EventType>, boost::python::arg_from_python<boost::python::api::object>, boost::python::arg_from_python<boost::python::api::object&>, boost::python::arg_from_python<bool>, boost::python::arg_from_python<PyTango::ExtractAs> > (rc=..., f=@0x55783e2c6158: 0x7f90cfc73e9d <PyDeviceProxy::subscribe_event_attrib(boost::python::api::object, std::__cxx11::string const&, Tango::EventType, boost::python::api::object, boost::python::api::object&, bool, PyTango::ExtractAs)>, ac0=..., ac1=..., ac2=..., ac3=..., ac4=..., ac5=..., ac6=...) at /opt/conda/envs/env-py3.8-tango9.3.4/include/boost/python/detail/invoke.hpp:73
#13 0x00007f90cfcbdc74 in boost::python::detail::caller_arity<7u>::impl<int (*)(boost::python::api::object, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, Tango::EventType, boost::python::api::object, boost::python::api::object&, bool, PyTango::ExtractAs), boost::python::default_call_policies, boost::mpl::vector8<int, boost::python::api::object, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, Tango::EventType, boost::python::api::object, boost::python::api::object&, bool, PyTango::ExtractAs> >::operator() (this=0x55783e2c6158, args_=0x7f90bfd33a00) at /opt/conda/envs/env-py3.8-tango9.3.4/include/boost/python/detail/caller.hpp:233
#14 0x00007f90cfcb9452 in boost::python::objects::caller_py_function_impl<boost::python::detail::caller<int (*)(boost::python::api::object, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, Tango::EventType, boost::python::api::object, boost::python::api::object&, bool, PyTango::ExtractAs), boost::python::default_call_policies, boost::mpl::vector8<int, boost::python::api::object, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, Tango::EventType, boost::python::api::object, boost::python::api::object&, bool, PyTango::ExtractAs> > >::operator() (this=0x55783e2c6150, args=0x7f90bfd33a00, kw=0x0) at /opt/conda/envs/env-py3.8-tango9.3.4/include/boost/python/object/py_function.hpp:38
#15 0x00007f90ce9c90f5 in boost::python::objects::function::call(_object*, _object*) const () from /opt/conda/envs/env-py3.8-tango9.3.4/lib/libboost_python38.so.1.73.0
#16 0x00007f90ce9c9329 in boost::detail::function::void_function_ref_invoker0<boost::python::objects::(anonymous namespace)::bind_return, void>::invoke(boost::detail::function::function_buffer&) () from /opt/conda/envs/env-py3.8-tango9.3.4/lib/libboost_python38.so.1.73.0
#17 0x00007f90ce9cf9bb in boost::python::detail::exception_handler::operator()(boost::function0<void> const&) const () from /opt/conda/envs/env-py3.8-tango9.3.4/lib/libboost_python38.so.1.73.0
#18 0x00007f90cfcec0c4 in boost::detail::function::function_obj_invoker2<boost::_bi::bind_t<bool, boost::python::detail::translate_exception<Tango::NotAllowed, void (*)(Tango::NotAllowed const&)>, boost::_bi::list3<boost::arg<1>, boost::arg<2>, boost::_bi::value<void (*)(Tango::NotAllowed const&)> > >, bool, boost::python::detail::exception_handler const&, boost::function0<void> const&>::invoke(boost::detail::function::function_buffer&, boost::python::detail::exception_handler const&, boost::function0<void> const&) () from /opt/project/tango/_tango.cpython-38-x86_64-linux-gnu.so
#19 0x00007f90ce9cf98a in boost::python::detail::exception_handler::operator()(boost::function0<void> const&) const () from /opt/conda/envs/env-py3.8-tango9.3.4/lib/libboost_python38.so.1.73.0
#20 0x00007f90cfcec074 in boost::detail::function::function_obj_invoker2<boost::_bi::bind_t<bool, boost::python::detail::translate_exception<Tango::DeviceUnlocked, void (*)(Tango::DeviceUnlocked const&)>, boost::_bi::list3<boost::arg<1>, boost::arg<2>, boost::_bi::value<void (*)(Tango::DeviceUnlocked const&)> > >, bool, boost::python::detail::exception_handler const&, boost::function0<void> const&>::invoke(boost::detail::function::function_buffer&, boost::python::detail::exception_handler const&, boost::function0<void> const&) () from /opt/project/tango/_tango.cpython-38-x86_64-linux-gnu.so
#21 0x00007f90ce9cf98a in boost::python::detail::exception_handler::operator()(boost::function0<void> const&) const () from /opt/conda/envs/env-py3.8-tango9.3.4/lib/libboost_python38.so.1.73.0
#22 0x00007f90cfcec024 in boost::detail::function::function_obj_invoker2<boost::_bi::bind_t<bool, boost::python::detail::translate_exception<Tango::EventSystemFailed, void (*)(Tango::EventSystemFailed const&)>, boost::_bi::list3<boost::arg<1>, boost::arg<2>, boost::_bi::value<void (*)(Tango::EventSystemFailed const&)> > >, bool, boost::python::detail::exception_handler const&, boost::function0<void> const&>::invoke(boost::detail::function::function_buffer&, boost::python::detail::exception_handler const&, boost::function0<void> const&) () from /opt/project/tango/_tango.cpython-38-x86_64-linux-gnu.so
#23 0x00007f90ce9cf98a in boost::python::detail::exception_handler::operator()(boost::function0<void> const&) const () from /opt/conda/envs/env-py3.8-tango9.3.4/lib/libboost_python38.so.1.73.0
#24 0x00007f90cfcebfd4 in boost::detail::function::function_obj_invoker2<boost::_bi::bind_t<bool, boost::python::detail::translate_exception<Tango::AsynReplyNotArrived, void (*)(Tango::AsynReplyNotArrived const&)>, boost::_bi::list3<boost::arg<1>, boost::arg<2>, boost::_bi::value<void (*)(Tango::AsynReplyNotArrived const&)> > >, bool, boost::python::detail::exception_handler const&, boost::function0<void> const&>::invoke(boost::detail::function::function_buffer&, boost::python::detail::exception_handler const&, boost::function0<void> const&) () from /opt/project/tango/_tango.cpython-38-x86_64-linux-gnu.so
#25 0x00007f90ce9cf98a in boost::python::detail::exception_handler::operator()(boost::function0<void> const&) const () from /opt/conda/envs/env-py3.8-tango9.3.4/lib/libboost_python38.so.1.73.0
#26 0x00007f90cfcebf84 in boost::detail::function::function_obj_invoker2<boost::_bi::bind_t<bool, boost::python::detail::translate_exception<Tango::AsynCall, void (*)(Tango::AsynCall const&)>, boost::_bi::list3<boost::arg<1>, boost::arg<2>, boost::_bi::value<void (*)(Tango::AsynCall const&)> > >, bool, boost::python::detail::exception_handler const&, boost::function0<void> const&>::invoke(boost::detail::function::function_buffer&, boost::python::detail::exception_handler const&, boost::function0<void> const&) () from /opt/project/tango/_tango.cpython-38-x86_64-linux-gnu.so
#27 0x00007f90ce9cf98a in boost::python::detail::exception_handler::operator()(boost::function0<void> const&) const () from /opt/conda/envs/env-py3.8-tango9.3.4/lib/libboost_python38.so.1.73.0
#28 0x00007f90cfcebf34 in boost::detail::function::function_obj_invoker2<boost::_bi::bind_t<bool, boost::python::detail::translate_exception<Tango::NonSupportedFeature, void (*)(Tango::NonSupportedFeature const&)>, boost::_bi::list3<boost::arg<1>, boost::arg<2>, boost::_bi::value<void (*)(Tango::NonSupportedFeature const&)> > >, bool, boost::python::detail::exception_handler const&, boost::function0<void> const&>::invoke(boost::detail::function::function_buffer&, boost::python::detail::exception_handler const&, boost::function0<void> const&) () from /opt/project/tango/_tango.cpython-38-x86_64-linux-gnu.so
#29 0x00007f90ce9cf98a in boost::python::detail::exception_handler::operator()(boost::function0<void> const&) const () from /opt/conda/envs/env-py3.8-tango9.3.4/lib/libboost_python38.so.1.73.0
#30 0x00007f90cfcebee4 in boost::detail::function::function_obj_invoker2<boost::_bi::bind_t<bool, boost::python::detail::translate_exception<Tango::WrongData, void (*)(Tango::WrongData const&)>, boost::_bi::list3<boost::arg<1>, boost::arg<2>, boost::_bi::value<void (*)(Tango::WrongData const&)> > >, bool, boost::python::detail::exception_handler const&, boost::function0<void> const&>::invoke(boost::detail::function::function_buffer&, boost::python::detail::exception_handler const&, boost::function0<void> const&) () from /opt/project/tango/_tango.cpython-38-x86_64-linux-gnu.so
#31 0x00007f90ce9cf98a in boost::python::detail::exception_handler::operator()(boost::function0<void> const&) const () from /opt/conda/envs/env-py3.8-tango9.3.4/lib/libboost_python38.so.1.73.0
#32 0x00007f90cfcebe94 in boost::detail::function::function_obj_invoker2<boost::_bi::bind_t<bool, boost::python::detail::translate_exception<Tango::NonDbDevice, void (*)(Tango::NonDbDevice const&)>, boost::_bi::list3<boost::arg<1>, boost::arg<2>, boost::_bi::value<void (*)(Tango::NonDbDevice const&)> > >, bool, boost::python::detail::exception_handler const&, boost::function0<void> const&>::invoke(boost::detail::function::function_buffer&, boost::python::detail::exception_handler const&, boost::function0<void> const&) () from /opt/project/tango/_tango.cpython-38-x86_64-linux-gnu.so
#33 0x00007f90ce9cf98a in boost::python::detail::exception_handler::operator()(boost::function0<void> const&) const () from /opt/conda/envs/env-py3.8-tango9.3.4/lib/libboost_python38.so.1.73.0
#34 0x00007f90cfcebe44 in boost::detail::function::function_obj_invoker2<boost::_bi::bind_t<bool, boost::python::detail::translate_exception<Tango::WrongNameSyntax, void (*)(Tango::WrongNameSyntax const&)>, boost::_bi::list3<boost::arg<1>, boost::arg<2>, boost::_bi::value<void (*)(Tango::WrongNameSyntax const&)> > >, bool, boost::python::detail::exception_handler const&, boost::function0<void> const&>::invoke(boost::detail::function::function_buffer&, boost::python::detail::exception_handler const&, boost::function0<void> const&) () from /opt/project/tango/_tango.cpython-38-x86_64-linux-gnu.so
#35 0x00007f90ce9cf98a in boost::python::detail::exception_handler::operator()(boost::function0<void> const&) const () from /opt/conda/envs/env-py3.8-tango9.3.4/lib/libboost_python38.so.1.73.0
#36 0x00007f90cfcebdf4 in boost::detail::function::function_obj_invoker2<boost::_bi::bind_t<bool, boost::python::detail::translate_exception<Tango::CommunicationFailed, void (*)(Tango::CommunicationFailed const&)>, boost::_bi::list3<boost::arg<1>, boost::arg<2>, boost::_bi::value<void (*)(Tango::CommunicationFailed const&)> > >, bool, boost::python::detail::exception_handler const&, boost::function0<void> const&>::invoke(boost::detail::function::function_buffer&, boost::python::detail::exception_handler const&, boost::function0<void> const&) () from /opt/project/tango/_tango.cpython-38-x86_64-linux-gnu.so
#37 0x00007f90ce9cf98a in boost::python::detail::exception_handler::operator()(boost::function0<void> const&) const () from /opt/conda/envs/env-py3.8-tango9.3.4/lib/libboost_python38.so.1.73.0
#38 0x00007f90cfcebda4 in boost::detail::function::function_obj_invoker2<boost::_bi::bind_t<bool, boost::python::detail::translate_exception<Tango::ConnectionFailed, void (*)(Tango::ConnectionFailed const&)>, boost::_bi::list3<boost::arg<1>, boost::arg<2>, boost::_bi::value<void (*)(Tango::ConnectionFailed const&)> > >, bool, boost::python::detail::exception_handler const&, boost::function0<void> const&>::invoke(boost::detail::function::function_buffer&, boost::python::detail::exception_handler const&, boost::function0<void> const&) () from /opt/project/tango/_tango.cpython-38-x86_64-linux-gnu.so
#39 0x00007f90ce9cf98a in boost::python::detail::exception_handler::operator()(boost::function0<void> const&) const () from /opt/conda/envs/env-py3.8-tango9.3.4/lib/libboost_python38.so.1.73.0
#40 0x00007f90cfcebd54 in boost::detail::function::function_obj_invoker2<boost::_bi::bind_t<bool, boost::python::detail::translate_exception<Tango::DevFailed, void (*)(Tango::DevFailed const&)>, boost::_bi::list3<boost::arg<1>, boost::arg<2>, boost::_bi::value<void (*)(Tango::DevFailed const&)> > >, bool, boost::python::detail::exception_handler const&, boost::function0<void> const&>::invoke(boost::detail::function::function_buffer&, boost::python::detail::exception_handler const&, boost::function0<void> const&) () from /opt/project/tango/_tango.cpython-38-x86_64-linux-gnu.so
#41 0x00007f90ce9cf73f in boost::python::handle_exception_impl(boost::function0<void>) () from /opt/conda/envs/env-py3.8-tango9.3.4/lib/libboost_python38.so.1.73.0
#42 0x00007f90ce9c5f9a in function_call () from /opt/conda/envs/env-py3.8-tango9.3.4/lib/libboost_python38.so.1.73.0
#43 0x000055783cb6a85f in _PyObject_MakeTpCall (callable=0x55783e2c6170, args=<optimized out>, nargs=<optimized out>, keywords=<optimized out>) at /tmp/build/80754af9/python_1599203911753/work/Objects/call.c:159
#44 0x000055783cbb8fa1 in _PyObject_Vectorcall (kwnames=<optimized out>, nargsf=7, args=0x7f90a400c238, callable=0x55783e2c6170) at /tmp/build/80754af9/python_1599203911753/work/Include/cpython/abstract.h:125
#45 method_vectorcall (method=<optimized out>, args=0x7f90a400c240, nargsf=<optimized out>, kwnames=<optimized out>) at /tmp/build/80754af9/python_1599203911753/work/Objects/classobject.c:60
#46 0x000055783cb2d77f in _PyObject_Vectorcall (kwnames=0x0, nargsf=<optimized out>, args=0x7f90a400c240, callable=0x7f90cc8be800) at /tmp/build/80754af9/python_1599203911753/work/Include/cpython/abstract.h:127
#47 call_function (kwnames=0x0, oparg=<optimized out>, pp_stack=<synthetic pointer>, tstate=0x7f90a4005620) at /tmp/build/80754af9/python_1599203911753/work/Python/ceval.c:4963
#48 _PyEval_EvalFrameDefault (f=<optimized out>, throwflag=<optimized out>) at /tmp/build/80754af9/python_1599203911753/work/Python/ceval.c:3469
#49 0x000055783cbb7a92 in _PyEval_EvalCodeWithName (_co=0x7f90bfe36500, globals=<optimized out>, locals=<optimized out>, args=<optimized out>, argcount=<optimized out>, kwnames=0x0, kwargs=0x7f90bfd482f8, kwcount=<optimized out>, kwstep=1, defs=0x7f90bfe3e4b8, defcount=4, kwdefs=0x0, closure=0x0, name=0x7f90bfe30f30, qualname=0x7f90bfe30f30) at /tmp/build/80754af9/python_1599203911753/work/Python/ceval.c:4298
#50 0x000055783cbb8943 in _PyFunction_Vectorcall (func=<optimized out>, stack=0x7f90bfd482d8, nargsf=<optimized out>, kwnames=<optimized out>) at /tmp/build/80754af9/python_1599203911753/work/Objects/call.c:435
#51 0x000055783cb6a041 in PyVectorcall_Call (callable=0x7f90bfe46040, tuple=<optimized out>, kwargs=<optimized out>) at /tmp/build/80754af9/python_1599203911753/work/Objects/call.c:199
#52 0x000055783cbef99b in do_call_core (kwdict=0x7f90bc4c7340, callargs=0x7f90bfd482c0, func=0x7f90bfe46040, tstate=<optimized out>) at /tmp/build/80754af9/python_1599203911753/work/Python/ceval.c:5010
#53 _PyEval_EvalFrameDefault (f=<optimized out>, throwflag=<optimized out>) at /tmp/build/80754af9/python_1599203911753/work/Python/ceval.c:3559
#54 0x000055783cbb7a92 in _PyEval_EvalCodeWithName (_co=0x7f90bfe362f0, globals=<optimized out>, locals=<optimized out>, args=<optimized out>, argcount=<optimized out>, kwnames=0x0, kwargs=0x7f90bfd3f438, kwcount=<optimized out>, kwstep=1, defs=0x0, defcount=0, kwdefs=0x0, closure=0x0, name=0x7f90bfe346c0, qualname=0x7f90bfe346c0) at /tmp/build/80754af9/python_1599203911753/work/Python/ceval.c:4298
#55 0x000055783cbb8943 in _PyFunction_Vectorcall (func=<optimized out>, stack=0x7f90bfd3f418, nargsf=<optimized out>, kwnames=<optimized out>) at /tmp/build/80754af9/python_1599203911753/work/Objects/call.c:435
#56 0x000055783cb6a041 in PyVectorcall_Call (callable=0x7f90bfe45ee0, tuple=<optimized out>, kwargs=<optimized out>) at /tmp/build/80754af9/python_1599203911753/work/Objects/call.c:199
#57 0x000055783cbef99b in do_call_core (kwdict=0x7f90bc4c7400, callargs=0x7f90bfd3f400, func=0x7f90bfe45ee0, tstate=<optimized out>) at /tmp/build/80754af9/python_1599203911753/work/Python/ceval.c:5010
#58 _PyEval_EvalFrameDefault (f=<optimized out>, throwflag=<optimized out>) at /tmp/build/80754af9/python_1599203911753/work/Python/ceval.c:3559
#59 0x000055783cbb7a92 in _PyEval_EvalCodeWithName (_co=0x7f90bfe2b500, globals=<optimized out>, locals=<optimized out>, args=<optimized out>, argcount=<optimized out>, kwnames=0x7f90bfe2c1d8, kwargs=0x7f90a400c028, kwcount=<optimized out>, kwstep=1, defs=0x7f90bfe2d198, defcount=4, kwdefs=0x0, closure=0x0, name=0x7f90d0183670, qualname=0x7f90bfe27b20) at /tmp/build/80754af9/python_1599203911753/work/Python/ceval.c:4298
#60 0x000055783cbb8d20 in _PyFunction_Vectorcall (kwnames=<optimized out>, nargsf=<optimized out>, stack=0x7f90a400c008, func=0x7f90bfe25af0) at /tmp/build/80754af9/python_1599203911753/work/Objects/call.c:435
#61 _PyObject_Vectorcall (kwnames=<optimized out>, nargsf=<optimized out>, args=0x7f90a400c008, callable=0x7f90bfe25af0) at /tmp/build/80754af9/python_1599203911753/work/Include/cpython/abstract.h:127
#62 method_vectorcall (method=<optimized out>, args=0x7f90a400c010, nargsf=<optimized out>, kwnames=<optimized out>) at /tmp/build/80754af9/python_1599203911753/work/Objects/classobject.c:60
#63 0x000055783cb2d11a in _PyObject_Vectorcall (kwnames=0x7f90bfe2c1c0, nargsf=<optimized out>, args=<optimized out>, callable=0x7f90d01a72c0) at /tmp/build/80754af9/python_1599203911753/work/Include/cpython/abstract.h:127
#64 call_function (kwnames=0x7f90bfe2c1c0, oparg=<optimized out>, pp_stack=<synthetic pointer>, tstate=<optimized out>) at /tmp/build/80754af9/python_1599203911753/work/Python/ceval.c:4963
#65 _PyEval_EvalFrameDefault (f=<optimized out>, throwflag=<optimized out>) at /tmp/build/80754af9/python_1599203911753/work/Python/ceval.c:3515
#66 0x000055783cbb7f9f in _PyEval_EvalCodeWithName (_co=0x7f90bfe2ba80, globals=<optimized out>, locals=<optimized out>, args=<optimized out>, argcount=<optimized out>, kwnames=0x0, kwargs=0x7f90bfd45a48, kwcount=<optimized out>, kwstep=1, defs=0x0, defcount=0, kwdefs=0x0, closure=0x7f90bfdb2100, name=0x7f90bfe346c0, qualname=0x7f90bfe346c0) at /tmp/build/80754af9/python_1599203911753/work/Python/ceval.c:4298
#67 0x000055783cbb8d20 in _PyFunction_Vectorcall (kwnames=<optimized out>, nargsf=<optimized out>, stack=0x7f90bfd45a28, func=0x7f90bfdd11f0) at /tmp/build/80754af9/python_1599203911753/work/Objects/call.c:435
#68 _PyObject_Vectorcall (kwnames=<optimized out>, nargsf=<optimized out>, args=0x7f90bfd45a28, callable=0x7f90bfdd11f0) at /tmp/build/80754af9/python_1599203911753/work/Include/cpython/abstract.h:127
#69 method_vectorcall (method=<optimized out>, args=0x7f90bfd45a30, nargsf=<optimized out>, kwnames=<optimized out>) at /tmp/build/80754af9/python_1599203911753/work/Objects/classobject.c:60
#70 0x000055783cb2d77f in _PyObject_Vectorcall (kwnames=0x0, nargsf=<optimized out>, args=0x7f90bfd45a30, callable=0x7f90d01b23c0) at /tmp/build/80754af9/python_1599203911753/work/Include/cpython/abstract.h:127
#71 call_function (kwnames=0x0, oparg=<optimized out>, pp_stack=<synthetic pointer>, tstate=0x7f90a4005620) at /tmp/build/80754af9/python_1599203911753/work/Python/ceval.c:4963
#72 _PyEval_EvalFrameDefault (f=<optimized out>, throwflag=<optimized out>) at /tmp/build/80754af9/python_1599203911753/work/Python/ceval.c:3469
#73 0x000055783cbb886b in function_code_fastcall (globals=<optimized out>, nargs=0, args=<optimized out>, co=<optimized out>) at /tmp/build/80754af9/python_1599203911753/work/Objects/call.c:283
#74 _PyFunction_Vectorcall (func=<optimized out>, stack=0x7f90bc4cb8b8, nargsf=<optimized out>, kwnames=<optimized out>) at /tmp/build/80754af9/python_1599203911753/work/Objects/call.c:410
#75 0x000055783cb2bb84 in _PyObject_Vectorcall (kwnames=0x0, nargsf=<optimized out>, args=0x7f90bc4cb8b8, callable=0x7f90bfd30280) at /tmp/build/80754af9/python_1599203911753/work/Include/cpython/abstract.h:127
#76 call_function (kwnames=0x0, oparg=<optimized out>, pp_stack=<synthetic pointer>, tstate=0x7f90a4005620) at /tmp/build/80754af9/python_1599203911753/work/Python/ceval.c:4963
#77 _PyEval_EvalFrameDefault (f=<optimized out>, throwflag=<optimized out>) at /tmp/build/80754af9/python_1599203911753/work/Python/ceval.c:3500
#78 0x000055783cbb886b in function_code_fastcall (globals=<optimized out>, nargs=1, args=<optimized out>, co=<optimized out>) at /tmp/build/80754af9/python_1599203911753/work/Objects/call.c:283
#79 _PyFunction_Vectorcall (func=<optimized out>, stack=0x7f90bc4cff88, nargsf=<optimized out>, kwnames=<optimized out>) at /tmp/build/80754af9/python_1599203911753/work/Objects/call.c:410
#80 0x000055783cb6a041 in PyVectorcall_Call (callable=0x7f90bfd3d9d0, tuple=<optimized out>, kwargs=<optimized out>) at /tmp/build/80754af9/python_1599203911753/work/Objects/call.c:199
#81 0x000055783cbef99b in do_call_core (kwdict=0x7f90bc4c7740, callargs=0x7f90bc4cff70, func=0x7f90bfd3d9d0, tstate=<optimized out>) at /tmp/build/80754af9/python_1599203911753/work/Python/ceval.c:5010
#82 _PyEval_EvalFrameDefault (f=<optimized out>, throwflag=<optimized out>) at /tmp/build/80754af9/python_1599203911753/work/Python/ceval.c:3559
#83 0x000055783cbb7a92 in _PyEval_EvalCodeWithName (_co=0x7f90bfe2b450, globals=<optimized out>, locals=<optimized out>, args=<optimized out>, argcount=<optimized out>, kwnames=0x0, kwargs=0x7f90bc4c0930, kwcount=<optimized out>, kwstep=1, defs=0x0, defcount=0, kwdefs=0x0, closure=0x0, name=0x7f90ccd3d9f0, qualname=0x7f90bfe27ad0) at /tmp/build/80754af9/python_1599203911753/work/Python/ceval.c:4298
#84 0x000055783cbb8943 in _PyFunction_Vectorcall (func=<optimized out>, stack=0x7f90bc4c0918, nargsf=<optimized out>, kwnames=<optimized out>) at /tmp/build/80754af9/python_1599203911753/work/Objects/call.c:435
#85 0x000055783cb2d75e in _PyObject_Vectorcall (kwnames=0x0, nargsf=<optimized out>, args=0x7f90bc4c0918, callable=0x7f90bfe25a60) at /tmp/build/80754af9/python_1599203911753/work/Include/cpython/abstract.h:127
#86 call_function (kwnames=0x0, oparg=<optimized out>, pp_stack=<synthetic pointer>, tstate=0x7f90a4005620) at /tmp/build/80754af9/python_1599203911753/work/Python/ceval.c:4963
#87 _PyEval_EvalFrameDefault (f=<optimized out>, throwflag=<optimized out>) at /tmp/build/80754af9/python_1599203911753/work/Python/ceval.c:3486
#88 0x000055783cbb7e19 in _PyEval_EvalCodeWithName (_co=0x7f90bfd60030, globals=<optimized out>, locals=<optimized out>, args=<optimized out>, argcount=<optimized out>, kwnames=0x0, kwargs=0x7f90afffde20, kwcount=<optimized out>, kwstep=1, defs=0x0, defcount=0, kwdefs=0x0, closure=0x7f90d00fea30, name=0x7f90bfd61cf0, qualname=0x7f90bfd3aa80) at /tmp/build/80754af9/python_1599203911753/work/Python/ceval.c:4298
#89 0x000055783cbb8943 in _PyFunction_Vectorcall (func=<optimized out>, stack=0x7f90afffde10, nargsf=<optimized out>, kwnames=<optimized out>) at /tmp/build/80754af9/python_1599203911753/work/Objects/call.c:435
#90 0x000055783cbb8e79 in _PyObject_Vectorcall (kwnames=0x0, nargsf=2, args=0x7f90afffde10, callable=0x7f90bfd3dc10) at /tmp/build/80754af9/python_1599203911753/work/Include/cpython/abstract.h:127
#91 method_vectorcall (method=<optimized out>, args=<optimized out>, nargsf=<optimized out>, kwnames=<optimized out>) at /tmp/build/80754af9/python_1599203911753/work/Objects/classobject.c:89
#92 0x000055783caaaf36 in _PyObject_Vectorcall (callable=0x7f90d0037ec0, args=<optimized out>, nargsf=<optimized out>, kwnames=<optimized out>) at /tmp/build/80754af9/python_1599203911753/work/Include/cpython/abstract.h:127
#93 0x000055783cb02950 in _PyObject_FastCall () at /tmp/build/80754af9/python_1599203911753/work/Include/cpython/abstract.h:147
#94 _PyObject_CallFunctionVa (callable=0x7f90d0037ec0, format=<optimized out>, va=<optimized out>, is_size_t=<optimized out>) at /tmp/build/80754af9/python_1599203911753/work/Objects/call.c:935
#95 0x000055783cb02add in PyEval_CallMethod (obj=<optimized out>, name=<optimized out>, format=0x7f90cfe6275b "(O)") at /tmp/build/80754af9/python_1599203911753/work/Objects/call.c:1055
#96 0x00007f90cfd3d50a in PyAttr::read(Tango::DeviceImpl*, Tango::Attribute&) () from /opt/project/tango/_tango.cpython-38-x86_64-linux-gnu.so
#97 0x00007f90cf1c71cd in Tango::Device_3Impl::read_attributes_no_except (this=0x55783e3d7ae0, names=..., aid=..., second_try=true, idx=...) at /usr/local/src/conda/cpptango-9.3.4/cppapi/server/device_3.cpp:539
#98 0x00007f90cf1dbfc0 in Tango::Device_5Impl::read_attributes_5 (this=0x55783e3d7ae0, names=..., source=Tango::CACHE_DEV, cl_id=...) at /usr/local/src/conda/cpptango-9.3.4/cppapi/server/device_5.cpp:338
#99 0x00007f90cf108b12 in _0RL_lcfn_6fe2f94a21a10053_84000000 (cd=0x7f90afffea40, svnt=0x55783e3d8328) at /usr/local/src/conda/cpptango-9.3.4/cppapi/server/idl/tangoSK.cpp:6494
#100 0x00007f90ceae0f44 in omniCallHandle::upcall(omniServant*, omniCallDescriptor&) () from /opt/conda/envs/env-py3.8-tango9.3.4/lib/libomniORB4.so.2
#101 0x00007f90cf10bafa in Tango::_impl_Device_5::_dispatch (this=0x55783e3d8320, _handle=...) at /usr/local/src/conda/cpptango-9.3.4/cppapi/server/idl/tangoSK.cpp:7416
#102 0x00007f90cead9c66 in omni::omniOrbPOA::dispatch(omniCallHandle&, omniLocalIdentity*) () from /opt/conda/envs/env-py3.8-tango9.3.4/lib/libomniORB4.so.2
#103 0x00007f90ceab0028 in omniLocalIdentity::dispatch(omniCallHandle&) () from /opt/conda/envs/env-py3.8-tango9.3.4/lib/libomniORB4.so.2
#104 0x00007f90ceb037b3 in omni::GIOP_S::handleRequest() () from /opt/conda/envs/env-py3.8-tango9.3.4/lib/libomniORB4.so.2
#105 0x00007f90ceb04819 in omni::GIOP_S::dispatcher() () from /opt/conda/envs/env-py3.8-tango9.3.4/lib/libomniORB4.so.2
#106 0x00007f90ceb0052c in omni::giopWorker::execute() () from /opt/conda/envs/env-py3.8-tango9.3.4/lib/libomniORB4.so.2
#107 0x00007f90ceaa0279 in omniAsyncWorker::real_run() () from /opt/conda/envs/env-py3.8-tango9.3.4/lib/libomniORB4.so.2
#108 0x00007f90ceaa06d0 in omniAsyncPoolServer::workerRun(omniAsyncWorker*) () from /opt/conda/envs/env-py3.8-tango9.3.4/lib/libomniORB4.so.2
#109 0x00007f90cea9fcc1 in omniAsyncWorker::mid_run() () from /opt/conda/envs/env-py3.8-tango9.3.4/lib/libomniORB4.so.2
#110 0x00007f90cf387044 in Tango::create_PyPerThData (info=...) at /usr/local/src/conda/cpptango-9.3.4/cppapi/server/utils.cpp:3255
#111 0x00007f90ceaa0176 in omniAsyncWorker::run(void*) () from /opt/conda/envs/env-py3.8-tango9.3.4/lib/libomniORB4.so.2
#112 0x00007f90ce77ef48 in omni_thread_wrapper () from /opt/conda/envs/env-py3.8-tango9.3.4/lib/./libomnithread.so.4
#113 0x00007f90d059efa3 in start_thread (arg=<optimized out>) at pthread_create.c:486
#114 0x00007f90d04cf4cf in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95

Thread 8 (Thread 0x7f90bccfc700 (LWP 2509)):
#0  futex_abstimed_wait_cancelable (private=0, abstime=0x7f90bccfbd10, expected=0, futex_word=0x55783e3bb310) at ../sysdeps/unix/sysv/linux/futex-internal.h:205
#1  __pthread_cond_wait_common (abstime=0x7f90bccfbd10, mutex=0x55783e3bb2b0, cond=0x55783e3bb2e8) at pthread_cond_wait.c:539
#2  __pthread_cond_timedwait (cond=0x55783e3bb2e8, mutex=0x55783e3bb2b0, abstime=0x7f90bccfbd10) at pthread_cond_wait.c:667
#3  0x00007f90ce77e445 in omni_condition::timedwait(unsigned long, unsigned long) () from /opt/conda/envs/env-py3.8-tango9.3.4/lib/./libomnithread.so.4
#4  0x00007f90cf386ebd in Tango::TangoMonitor::wait (this=0x55783e3bb2b0, nb_millis=1799998) at /usr/local/src/conda/cpptango-9.3.4/cppapi/server/utils.cpp:3197
#5  0x00007f90cf281d1d in Tango::PollThread::get_command (this=0x55783e28fd30, tout=1799998) at /usr/local/src/conda/cpptango-9.3.4/cppapi/server/pollthread.cpp:285
#6  0x00007f90cf281850 in Tango::PollThread::run_undetached (this=0x55783e28fd30, ptr=0x0) at /usr/local/src/conda/cpptango-9.3.4/cppapi/server/pollthread.cpp:163
#7  0x00007f90ce77ef38 in omni_thread_wrapper () from /opt/conda/envs/env-py3.8-tango9.3.4/lib/./libomnithread.so.4
#8  0x00007f90d059efa3 in start_thread (arg=<optimized out>) at pthread_create.c:486
#9  0x00007f90d04cf4cf in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95

Thread 7 (Thread 0x7f90bd4fd700 (LWP 2508)):
#0  0x00007f90d04cf7ef in epoll_wait (epfd=20, events=0x7f90bd4fc1f0, maxevents=256, timeout=-1) at ../sysdeps/unix/sysv/linux/epoll_wait.c:30
#1  0x00007f90ce7abe7d in zmq::epoll_t::loop() () from /opt/conda/envs/env-py3.8-tango9.3.4/lib/./libzmq.so.5
#2  0x00007f90ce7e1bcf in thread_routine () from /opt/conda/envs/env-py3.8-tango9.3.4/lib/./libzmq.so.5
#3  0x00007f90d059efa3 in start_thread (arg=<optimized out>) at pthread_create.c:486
#4  0x00007f90d04cf4cf in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95

Thread 6 (Thread 0x7f90bdcfe700 (LWP 2507)):
#0  0x00007f90d04cf7ef in epoll_wait (epfd=18, events=0x7f90bdcfd1f0, maxevents=256, timeout=-1) at ../sysdeps/unix/sysv/linux/epoll_wait.c:30
#1  0x00007f90ce7abe7d in zmq::epoll_t::loop() () from /opt/conda/envs/env-py3.8-tango9.3.4/lib/./libzmq.so.5
#2  0x00007f90ce7e1bcf in thread_routine () from /opt/conda/envs/env-py3.8-tango9.3.4/lib/./libzmq.so.5
#3  0x00007f90d059efa3 in start_thread (arg=<optimized out>) at pthread_create.c:486
#4  0x00007f90d04cf4cf in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95

Thread 5 (Thread 0x7f90be4ff700 (LWP 2506)):
#0  0x00007f90d04c4819 in __GI___poll (fds=0x55783e3c2970, nfds=2, timeout=-1) at ../sysdeps/unix/sysv/linux/poll.c:29
#1  0x00007f90ceb1d3cf in omni::SocketCollection::Select() () from /opt/conda/envs/env-py3.8-tango9.3.4/lib/libomniORB4.so.2
#2  0x00007f90ceb47c62 in omni::unixEndpoint::AcceptAndMonitor(void (*)(void*, omni::giopConnection*), void*) () from /opt/conda/envs/env-py3.8-tango9.3.4/lib/libomniORB4.so.2
#3  0x00007f90ceb00742 in omni::giopRendezvouser::execute() () from /opt/conda/envs/env-py3.8-tango9.3.4/lib/libomniORB4.so.2
#4  0x00007f90ceaa0279 in omniAsyncWorker::real_run() () from /opt/conda/envs/env-py3.8-tango9.3.4/lib/libomniORB4.so.2
#5  0x00007f90cea9fcc1 in omniAsyncWorker::mid_run() () from /opt/conda/envs/env-py3.8-tango9.3.4/lib/libomniORB4.so.2
#6  0x00007f90cf387044 in Tango::create_PyPerThData (info=...) at /usr/local/src/conda/cpptango-9.3.4/cppapi/server/utils.cpp:3255
#7  0x00007f90ceaa0176 in omniAsyncWorker::run(void*) () from /opt/conda/envs/env-py3.8-tango9.3.4/lib/libomniORB4.so.2
#8  0x00007f90ce77ef48 in omni_thread_wrapper () from /opt/conda/envs/env-py3.8-tango9.3.4/lib/./libomnithread.so.4
#9  0x00007f90d059efa3 in start_thread (arg=<optimized out>) at pthread_create.c:486
#10 0x00007f90d04cf4cf in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95

Thread 4 (Thread 0x7f90bed00700 (LWP 2505)):
#0  0x00007f90d04c4819 in __GI___poll (fds=0x55783e3c2140, nfds=2, timeout=-1) at ../sysdeps/unix/sysv/linux/poll.c:29
#1  0x00007f90ceb1d3cf in omni::SocketCollection::Select() () from /opt/conda/envs/env-py3.8-tango9.3.4/lib/libomniORB4.so.2
#2  0x00007f90ceb44c42 in omni::tcpEndpoint::AcceptAndMonitor(void (*)(void*, omni::giopConnection*), void*) () from /opt/conda/envs/env-py3.8-tango9.3.4/lib/libomniORB4.so.2
#3  0x00007f90ceb00742 in omni::giopRendezvouser::execute() () from /opt/conda/envs/env-py3.8-tango9.3.4/lib/libomniORB4.so.2
#4  0x00007f90ceaa0279 in omniAsyncWorker::real_run() () from /opt/conda/envs/env-py3.8-tango9.3.4/lib/libomniORB4.so.2
#5  0x00007f90cea9fcc1 in omniAsyncWorker::mid_run() () from /opt/conda/envs/env-py3.8-tango9.3.4/lib/libomniORB4.so.2
#6  0x00007f90cf387044 in Tango::create_PyPerThData (info=...) at /usr/local/src/conda/cpptango-9.3.4/cppapi/server/utils.cpp:3255
#7  0x00007f90ceaa0176 in omniAsyncWorker::run(void*) () from /opt/conda/envs/env-py3.8-tango9.3.4/lib/libomniORB4.so.2
#8  0x00007f90ce77ef48 in omni_thread_wrapper () from /opt/conda/envs/env-py3.8-tango9.3.4/lib/./libomnithread.so.4
#9  0x00007f90d059efa3 in start_thread (arg=<optimized out>) at pthread_create.c:486
#10 0x00007f90d04cf4cf in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95

Thread 3 (Thread 0x7f90bf501700 (LWP 2504)):
#0  futex_abstimed_wait_cancelable (private=0, abstime=0x7f90bf500c10, expected=0, futex_word=0x55783e38efe0) at ../sysdeps/unix/sysv/linux/futex-internal.h:205
#1  __pthread_cond_wait_common (abstime=0x7f90bf500c10, mutex=0x55783e36db80, cond=0x55783e38efb8) at pthread_cond_wait.c:539
#2  __pthread_cond_timedwait (cond=0x55783e38efb8, mutex=0x55783e36db80, abstime=0x7f90bf500c10) at pthread_cond_wait.c:667
#3  0x00007f90ce77e445 in omni_condition::timedwait(unsigned long, unsigned long) () from /opt/conda/envs/env-py3.8-tango9.3.4/lib/./libomnithread.so.4
#4  0x00007f90ceaf7a70 in omni::Scavenger::execute() () from /opt/conda/envs/env-py3.8-tango9.3.4/lib/libomniORB4.so.2
#5  0x00007f90ceaa0279 in omniAsyncWorker::real_run() () from /opt/conda/envs/env-py3.8-tango9.3.4/lib/libomniORB4.so.2
#6  0x00007f90cea9fcc1 in omniAsyncWorker::mid_run() () from /opt/conda/envs/env-py3.8-tango9.3.4/lib/libomniORB4.so.2
#7  0x00007f90ceaa0176 in omniAsyncWorker::run(void*) () from /opt/conda/envs/env-py3.8-tango9.3.4/lib/libomniORB4.so.2
#8  0x00007f90ce77ef48 in omni_thread_wrapper () from /opt/conda/envs/env-py3.8-tango9.3.4/lib/./libomnithread.so.4
#9  0x00007f90d059efa3 in start_thread (arg=<optimized out>) at pthread_create.c:486
#10 0x00007f90d04cf4cf in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95

Thread 2 (Thread 0x7f90bfd1c700 (LWP 2503)):
#0  0x00007f90d040e4cc in __GI___sigtimedwait (set=set@entry=0x7f90bfd1be30, info=info@entry=0x7f90bfd1bd00, timeout=timeout@entry=0x0) at ../sysdeps/unix/sysv/linux/sigtimedwait.c:29
#1  0x00007f90d05a92bc in __sigwait (set=0x7f90bfd1be30, sig=0x7f90bfd1bdd0) at ../sysdeps/unix/sysv/linux/sigwait.c:28
#2  0x00007f90cf37c912 in Tango::DServerSignal::ThSig::run_undetached (this=0x55783e0327d0, ptr=0x0) at /usr/local/src/conda/cpptango-9.3.4/cppapi/server/thsig.cpp:89
#3  0x00007f90ce77ef38 in omni_thread_wrapper () from /opt/conda/envs/env-py3.8-tango9.3.4/lib/./libomnithread.so.4
#4  0x00007f90d059efa3 in start_thread (arg=<optimized out>) at pthread_create.c:486
#5  0x00007f90d04cf4cf in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95

Thread 1 (Thread 0x7f90d023a740 (LWP 2501)):
#0  futex_wait_cancelable (private=0, expected=0, futex_word=0x55783e3343c0) at ../sysdeps/unix/sysv/linux/futex-internal.h:88
#1  __pthread_cond_wait_common (abstime=0x0, mutex=0x55783e355b68, cond=0x55783e334398) at pthread_cond_wait.c:502
#2  __pthread_cond_wait (cond=0x55783e334398, mutex=0x55783e355b68) at pthread_cond_wait.c:655
#3  0x00007f90ce77e3d1 in omni_condition::wait() () from /opt/conda/envs/env-py3.8-tango9.3.4/lib/./libomnithread.so.4
#4  0x00007f90cea9fb79 in omniAsyncDedicated::perform(unsigned long, unsigned long) () from /opt/conda/envs/env-py3.8-tango9.3.4/lib/libomniORB4.so.2
#5  0x00007f90cf383e15 in Tango::Util::server_perform_work (this=0x55783e3bb0e0) at /usr/local/src/conda/cpptango-9.3.4/cppapi/server/utils.cpp:1963
#6  0x00007f90cf383ee7 in Tango::Util::server_run (this=0x55783e3bb0e0) at /usr/local/src/conda/cpptango-9.3.4/cppapi/server/utils.cpp:2081
#7  0x00007f90cfe2c7cb in PyUtil::server_run(Tango::Util&) () from /opt/project/tango/_tango.cpython-38-x86_64-linux-gnu.so
#8  0x00007f90cfe2f793 in boost::python::objects::caller_py_function_impl<boost::python::detail::caller<void (*)(Tango::Util&), boost::python::default_call_policies, boost::mpl::vector2<void, Tango::Util&> > >::operator()(_object*, _object*) () from /opt/project/tango/_tango.cpython-38-x86_64-linux-gnu.so
#9  0x00007f90ce9c90f5 in boost::python::objects::function::call(_object*, _object*) const () from /opt/conda/envs/env-py3.8-tango9.3.4/lib/libboost_python38.so.1.73.0
#10 0x00007f90ce9c9329 in boost::detail::function::void_function_ref_invoker0<boost::python::objects::(anonymous namespace)::bind_return, void>::invoke(boost::detail::function::function_buffer&) () from /opt/conda/envs/env-py3.8-tango9.3.4/lib/libboost_python38.so.1.73.0
#11 0x00007f90ce9cf9bb in boost::python::detail::exception_handler::operator()(boost::function0<void> const&) const () from /opt/conda/envs/env-py3.8-tango9.3.4/lib/libboost_python38.so.1.73.0
#12 0x00007f90cfcec0c4 in boost::detail::function::function_obj_invoker2<boost::_bi::bind_t<bool, boost::python::detail::translate_exception<Tango::NotAllowed, void (*)(Tango::NotAllowed const&)>, boost::_bi::list3<boost::arg<1>, boost::arg<2>, boost::_bi::value<void (*)(Tango::NotAllowed const&)> > >, bool, boost::python::detail::exception_handler const&, boost::function0<void> const&>::invoke(boost::detail::function::function_buffer&, boost::python::detail::exception_handler const&, boost::function0<void> const&) () from /opt/project/tango/_tango.cpython-38-x86_64-linux-gnu.so
#13 0x00007f90ce9cf98a in boost::python::detail::exception_handler::operator()(boost::function0<void> const&) const () from /opt/conda/envs/env-py3.8-tango9.3.4/lib/libboost_python38.so.1.73.0
#14 0x00007f90cfcec074 in boost::detail::function::function_obj_invoker2<boost::_bi::bind_t<bool, boost::python::detail::translate_exception<Tango::DeviceUnlocked, void (*)(Tango::DeviceUnlocked const&)>, boost::_bi::list3<boost::arg<1>, boost::arg<2>, boost::_bi::value<void (*)(Tango::DeviceUnlocked const&)> > >, bool, boost::python::detail::exception_handler const&, boost::function0<void> const&>::invoke(boost::detail::function::function_buffer&, boost::python::detail::exception_handler const&, boost::function0<void> const&) () from /opt/project/tango/_tango.cpython-38-x86_64-linux-gnu.so
#15 0x00007f90ce9cf98a in boost::python::detail::exception_handler::operator()(boost::function0<void> const&) const () from /opt/conda/envs/env-py3.8-tango9.3.4/lib/libboost_python38.so.1.73.0
#16 0x00007f90cfcec024 in boost::detail::function::function_obj_invoker2<boost::_bi::bind_t<bool, boost::python::detail::translate_exception<Tango::EventSystemFailed, void (*)(Tango::EventSystemFailed const&)>, boost::_bi::list3<boost::arg<1>, boost::arg<2>, boost::_bi::value<void (*)(Tango::EventSystemFailed const&)> > >, bool, boost::python::detail::exception_handler const&, boost::function0<void> const&>::invoke(boost::detail::function::function_buffer&, boost::python::detail::exception_handler const&, boost::function0<void> const&) () from /opt/project/tango/_tango.cpython-38-x86_64-linux-gnu.so
#17 0x00007f90ce9cf98a in boost::python::detail::exception_handler::operator()(boost::function0<void> const&) const () from /opt/conda/envs/env-py3.8-tango9.3.4/lib/libboost_python38.so.1.73.0
#18 0x00007f90cfcebfd4 in boost::detail::function::function_obj_invoker2<boost::_bi::bind_t<bool, boost::python::detail::translate_exception<Tango::AsynReplyNotArrived, void (*)(Tango::AsynReplyNotArrived const&)>, boost::_bi::list3<boost::arg<1>, boost::arg<2>, boost::_bi::value<void (*)(Tango::AsynReplyNotArrived const&)> > >, bool, boost::python::detail::exception_handler const&, boost::function0<void> const&>::invoke(boost::detail::function::function_buffer&, boost::python::detail::exception_handler const&, boost::function0<void> const&) () from /opt/project/tango/_tango.cpython-38-x86_64-linux-gnu.so
#19 0x00007f90ce9cf98a in boost::python::detail::exception_handler::operator()(boost::function0<void> const&) const () from /opt/conda/envs/env-py3.8-tango9.3.4/lib/libboost_python38.so.1.73.0
#20 0x00007f90cfcebf84 in boost::detail::function::function_obj_invoker2<boost::_bi::bind_t<bool, boost::python::detail::translate_exception<Tango::AsynCall, void (*)(Tango::AsynCall const&)>, boost::_bi::list3<boost::arg<1>, boost::arg<2>, boost::_bi::value<void (*)(Tango::AsynCall const&)> > >, bool, boost::python::detail::exception_handler const&, boost::function0<void> const&>::invoke(boost::detail::function::function_buffer&, boost::python::detail::exception_handler const&, boost::function0<void> const&) () from /opt/project/tango/_tango.cpython-38-x86_64-linux-gnu.so
#21 0x00007f90ce9cf98a in boost::python::detail::exception_handler::operator()(boost::function0<void> const&) const () from /opt/conda/envs/env-py3.8-tango9.3.4/lib/libboost_python38.so.1.73.0
#22 0x00007f90cfcebf34 in boost::detail::function::function_obj_invoker2<boost::_bi::bind_t<bool, boost::python::detail::translate_exception<Tango::NonSupportedFeature, void (*)(Tango::NonSupportedFeature const&)>, boost::_bi::list3<boost::arg<1>, boost::arg<2>, boost::_bi::value<void (*)(Tango::NonSupportedFeature const&)> > >, bool, boost::python::detail::exception_handler const&, boost::function0<void> const&>::invoke(boost::detail::function::function_buffer&, boost::python::detail::exception_handler const&, boost::function0<void> const&) () from /opt/project/tango/_tango.cpython-38-x86_64-linux-gnu.so
#23 0x00007f90ce9cf98a in boost::python::detail::exception_handler::operator()(boost::function0<void> const&) const () from /opt/conda/envs/env-py3.8-tango9.3.4/lib/libboost_python38.so.1.73.0
#24 0x00007f90cfcebee4 in boost::detail::function::function_obj_invoker2<boost::_bi::bind_t<bool, boost::python::detail::translate_exception<Tango::WrongData, void (*)(Tango::WrongData const&)>, boost::_bi::list3<boost::arg<1>, boost::arg<2>, boost::_bi::value<void (*)(Tango::WrongData const&)> > >, bool, boost::python::detail::exception_handler const&, boost::function0<void> const&>::invoke(boost::detail::function::function_buffer&, boost::python::detail::exception_handler const&, boost::function0<void> const&) () from /opt/project/tango/_tango.cpython-38-x86_64-linux-gnu.so
#25 0x00007f90ce9cf98a in boost::python::detail::exception_handler::operator()(boost::function0<void> const&) const () from /opt/conda/envs/env-py3.8-tango9.3.4/lib/libboost_python38.so.1.73.0
#26 0x00007f90cfcebe94 in boost::detail::function::function_obj_invoker2<boost::_bi::bind_t<bool, boost::python::detail::translate_exception<Tango::NonDbDevice, void (*)(Tango::NonDbDevice const&)>, boost::_bi::list3<boost::arg<1>, boost::arg<2>, boost::_bi::value<void (*)(Tango::NonDbDevice const&)> > >, bool, boost::python::detail::exception_handler const&, boost::function0<void> const&>::invoke(boost::detail::function::function_buffer&, boost::python::detail::exception_handler const&, boost::function0<void> const&) () from /opt/project/tango/_tango.cpython-38-x86_64-linux-gnu.so
#27 0x00007f90ce9cf98a in boost::python::detail::exception_handler::operator()(boost::function0<void> const&) const () from /opt/conda/envs/env-py3.8-tango9.3.4/lib/libboost_python38.so.1.73.0
#28 0x00007f90cfcebe44 in boost::detail::function::function_obj_invoker2<boost::_bi::bind_t<bool, boost::python::detail::translate_exception<Tango::WrongNameSyntax, void (*)(Tango::WrongNameSyntax const&)>, boost::_bi::list3<boost::arg<1>, boost::arg<2>, boost::_bi::value<void (*)(Tango::WrongNameSyntax const&)> > >, bool, boost::python::detail::exception_handler const&, boost::function0<void> const&>::invoke(boost::detail::function::function_buffer&, boost::python::detail::exception_handler const&, boost::function0<void> const&) () from /opt/project/tango/_tango.cpython-38-x86_64-linux-gnu.so
#29 0x00007f90ce9cf98a in boost::python::detail::exception_handler::operator()(boost::function0<void> const&) const () from /opt/conda/envs/env-py3.8-tango9.3.4/lib/libboost_python38.so.1.73.0
#30 0x00007f90cfcebdf4 in boost::detail::function::function_obj_invoker2<boost::_bi::bind_t<bool, boost::python::detail::translate_exception<Tango::CommunicationFailed, void (*)(Tango::CommunicationFailed const&)>, boost::_bi::list3<boost::arg<1>, boost::arg<2>, boost::_bi::value<void (*)(Tango::CommunicationFailed const&)> > >, bool, boost::python::detail::exception_handler const&, boost::function0<void> const&>::invoke(boost::detail::function::function_buffer&, boost::python::detail::exception_handler const&, boost::function0<void> const&) () from /opt/project/tango/_tango.cpython-38-x86_64-linux-gnu.so
#31 0x00007f90ce9cf98a in boost::python::detail::exception_handler::operator()(boost::function0<void> const&) const () from /opt/conda/envs/env-py3.8-tango9.3.4/lib/libboost_python38.so.1.73.0
#32 0x00007f90cfcebda4 in boost::detail::function::function_obj_invoker2<boost::_bi::bind_t<bool, boost::python::detail::translate_exception<Tango::ConnectionFailed, void (*)(Tango::ConnectionFailed const&)>, boost::_bi::list3<boost::arg<1>, boost::arg<2>, boost::_bi::value<void (*)(Tango::ConnectionFailed const&)> > >, bool, boost::python::detail::exception_handler const&, boost::function0<void> const&>::invoke(boost::detail::function::function_buffer&, boost::python::detail::exception_handler const&, boost::function0<void> const&) () from /opt/project/tango/_tango.cpython-38-x86_64-linux-gnu.so
#33 0x00007f90ce9cf98a in boost::python::detail::exception_handler::operator()(boost::function0<void> const&) const () from /opt/conda/envs/env-py3.8-tango9.3.4/lib/libboost_python38.so.1.73.0
#34 0x00007f90cfcebd54 in boost::detail::function::function_obj_invoker2<boost::_bi::bind_t<bool, boost::python::detail::translate_exception<Tango::DevFailed, void (*)(Tango::DevFailed const&)>, boost::_bi::list3<boost::arg<1>, boost::arg<2>, boost::_bi::value<void (*)(Tango::DevFailed const&)> > >, bool, boost::python::detail::exception_handler const&, boost::function0<void> const&>::invoke(boost::detail::function::function_buffer&, boost::python::detail::exception_handler const&, boost::function0<void> const&) () from /opt/project/tango/_tango.cpython-38-x86_64-linux-gnu.so
#35 0x00007f90ce9cf73f in boost::python::handle_exception_impl(boost::function0<void>) () from /opt/conda/envs/env-py3.8-tango9.3.4/lib/libboost_python38.so.1.73.0
#36 0x00007f90ce9c5f9a in function_call () from /opt/conda/envs/env-py3.8-tango9.3.4/lib/libboost_python38.so.1.73.0
#37 0x000055783cb6a85f in _PyObject_MakeTpCall (callable=0x55783e2d3ca0, args=<optimized out>, nargs=<optimized out>, keywords=<optimized out>) at /tmp/build/80754af9/python_1599203911753/work/Objects/call.c:159
#38 0x000055783cbb8fa1 in _PyObject_Vectorcall (kwnames=<optimized out>, nargsf=1, args=0x7f90bfd2f920, callable=0x55783e2d3ca0) at /tmp/build/80754af9/python_1599203911753/work/Include/cpython/abstract.h:125
#39 method_vectorcall (method=<optimized out>, args=0x7f90bfd2f928, nargsf=<optimized out>, kwnames=<optimized out>) at /tmp/build/80754af9/python_1599203911753/work/Objects/classobject.c:60
#40 0x000055783cb2d77f in _PyObject_Vectorcall (kwnames=0x0, nargsf=<optimized out>, args=0x7f90bfd2f928, callable=0x7f90d01a7280) at /tmp/build/80754af9/python_1599203911753/work/Include/cpython/abstract.h:127
#41 call_function (kwnames=0x0, oparg=<optimized out>, pp_stack=<synthetic pointer>, tstate=0x55783de549c0) at /tmp/build/80754af9/python_1599203911753/work/Python/ceval.c:4963
#42 _PyEval_EvalFrameDefault (f=<optimized out>, throwflag=<optimized out>) at /tmp/build/80754af9/python_1599203911753/work/Python/ceval.c:3469
#43 0x000055783cbb7f9f in _PyEval_EvalCodeWithName (_co=0x7f90bfd6d920, globals=<optimized out>, locals=<optimized out>, args=<optimized out>, argcount=<optimized out>, kwnames=0x0, kwargs=0x7f90d01c1058, kwcount=<optimized out>, kwstep=1, defs=0x0, defcount=0, kwdefs=0x0, closure=0x7f90bfd33b20, name=0x7f90bfd6b9b0, qualname=0x7f90bfd6c450) at /tmp/build/80754af9/python_1599203911753/work/Python/ceval.c:4298
#44 0x000055783cbb8943 in _PyFunction_Vectorcall (func=<optimized out>, stack=0x7f90d01c1058, nargsf=<optimized out>, kwnames=<optimized out>) at /tmp/build/80754af9/python_1599203911753/work/Objects/call.c:435
#45 0x000055783cb6a041 in PyVectorcall_Call (callable=0x7f90bfd41040, tuple=<optimized out>, kwargs=<optimized out>) at /tmp/build/80754af9/python_1599203911753/work/Objects/call.c:199
#46 0x000055783cbef99b in do_call_core (kwdict=0x7f90bfe2c600, callargs=0x7f90d01c1040, func=0x7f90bfd41040, tstate=<optimized out>) at /tmp/build/80754af9/python_1599203911753/work/Python/ceval.c:5010
#47 _PyEval_EvalFrameDefault (f=<optimized out>, throwflag=<optimized out>) at /tmp/build/80754af9/python_1599203911753/work/Python/ceval.c:3559
#48 0x000055783cbb7a92 in _PyEval_EvalCodeWithName (_co=0x7f90bfe2b500, globals=<optimized out>, locals=<optimized out>, args=<optimized out>, argcount=<optimized out>, kwnames=0x7f90bfd687a8, kwargs=0x55783e39daa8, kwcount=<optimized out>, kwstep=1, defs=0x7f90bfe2d198, defcount=4, kwdefs=0x0, closure=0x0, name=0x7f90d0183670, qualname=0x7f90bfe27b20) at /tmp/build/80754af9/python_1599203911753/work/Python/ceval.c:4298
#49 0x000055783cbb8d20 in _PyFunction_Vectorcall (kwnames=<optimized out>, nargsf=<optimized out>, stack=0x55783e39da98, func=0x7f90bfe25af0) at /tmp/build/80754af9/python_1599203911753/work/Objects/call.c:435
#50 _PyObject_Vectorcall (kwnames=<optimized out>, nargsf=<optimized out>, args=0x55783e39da98, callable=0x7f90bfe25af0) at /tmp/build/80754af9/python_1599203911753/work/Include/cpython/abstract.h:127
#51 method_vectorcall (method=<optimized out>, args=0x55783e39daa0, nargsf=<optimized out>, kwnames=<optimized out>) at /tmp/build/80754af9/python_1599203911753/work/Objects/classobject.c:60
#52 0x000055783cb2d11a in _PyObject_Vectorcall (kwnames=0x7f90bfd68790, nargsf=<optimized out>, args=<optimized out>, callable=0x7f90cfffbd80) at /tmp/build/80754af9/python_1599203911753/work/Include/cpython/abstract.h:127
#53 call_function (kwnames=0x7f90bfd68790, oparg=<optimized out>, pp_stack=<synthetic pointer>, tstate=<optimized out>) at /tmp/build/80754af9/python_1599203911753/work/Python/ceval.c:4963
#54 _PyEval_EvalFrameDefault (f=<optimized out>, throwflag=<optimized out>) at /tmp/build/80754af9/python_1599203911753/work/Python/ceval.c:3515
#55 0x000055783cbb7f9f in _PyEval_EvalCodeWithName (_co=0x7f90bfd6d9d0, globals=<optimized out>, locals=<optimized out>, args=<optimized out>, argcount=<optimized out>, kwnames=0x7f90bfd33838, kwargs=0x7f90bfd3ea78, kwcount=<optimized out>, kwstep=1, defs=0x7f90bfd6ce98, defcount=6, kwdefs=0x0, closure=0x0, name=0x7f90bfd6baf0, qualname=0x7f90bfd6baf0) at /tmp/build/80754af9/python_1599203911753/work/Python/ceval.c:4298
#56 0x000055783cbb8943 in _PyFunction_Vectorcall (func=<optimized out>, stack=0x7f90bfd3ea70, nargsf=<optimized out>, kwnames=<optimized out>) at /tmp/build/80754af9/python_1599203911753/work/Objects/call.c:435
#57 0x000055783cbb94cb in _PyObject_FastCallDict (callable=0x7f90bfd2dd30, args=0x7f90d00fe958, nargsf=<optimized out>, kwargs=<optimized out>) at /tmp/build/80754af9/python_1599203911753/work/Objects/call.c:104
#58 0x000055783cc584be in partial_fastcall.isra.2 (kwargs=0x7f90d012fd40, nargs=<optimized out>, args=<optimized out>) at /tmp/build/80754af9/python_1599203911753/work/Modules/_functoolsmodule.c:169
#59 partial_call (pto=0x7f90d005f900, args=<optimized out>, kwargs=<optimized out>) at /tmp/build/80754af9/python_1599203911753/work/Modules/_functoolsmodule.c:224
#60 0x000055783cb6a85f in _PyObject_MakeTpCall (callable=0x7f90d005f900, args=<optimized out>, nargs=<optimized out>, keywords=<optimized out>) at /tmp/build/80754af9/python_1599203911753/work/Objects/call.c:159
#61 0x000055783cbede35 in _PyObject_Vectorcall (kwnames=0x0, nargsf=<optimized out>, args=0x55783e3bb088, callable=<optimized out>) at /tmp/build/80754af9/python_1599203911753/work/Include/cpython/abstract.h:125
#62 call_function (kwnames=0x0, oparg=<optimized out>, pp_stack=<synthetic pointer>, tstate=0x55783de549c0) at /tmp/build/80754af9/python_1599203911753/work/Python/ceval.c:4963
#63 _PyEval_EvalFrameDefault (f=<optimized out>, throwflag=<optimized out>) at /tmp/build/80754af9/python_1599203911753/work/Python/ceval.c:3500
#64 0x000055783cbb7a92 in _PyEval_EvalCodeWithName (_co=0x7f90bfd6db30, globals=<optimized out>, locals=<optimized out>, args=<optimized out>, argcount=<optimized out>, kwnames=0x0, kwargs=0x7f90d010f5b8, kwcount=<optimized out>, kwstep=1, defs=0x7f90bfd65758, defcount=8, kwdefs=0x0, closure=0x0, name=0x7f90d0183670, qualname=0x7f90d0183670) at /tmp/build/80754af9/python_1599203911753/work/Python/ceval.c:4298
#65 0x000055783cbb8943 in _PyFunction_Vectorcall (func=<optimized out>, stack=0x7f90d010f5b0, nargsf=<optimized out>, kwnames=<optimized out>) at /tmp/build/80754af9/python_1599203911753/work/Objects/call.c:435
#66 0x000055783cb2bb84 in _PyObject_Vectorcall (kwnames=0x0, nargsf=<optimized out>, args=0x7f90d010f5b0, callable=0x7f90bfd2ddc0) at /tmp/build/80754af9/python_1599203911753/work/Include/cpython/abstract.h:127
#67 call_function (kwnames=0x0, oparg=<optimized out>, pp_stack=<synthetic pointer>, tstate=0x55783de549c0) at /tmp/build/80754af9/python_1599203911753/work/Python/ceval.c:4963
#68 _PyEval_EvalFrameDefault (f=<optimized out>, throwflag=<optimized out>) at /tmp/build/80754af9/python_1599203911753/work/Python/ceval.c:3500
#69 0x000055783cbb7a92 in _PyEval_EvalCodeWithName (_co=0x7f90d013d660, globals=<optimized out>, locals=<optimized out>, args=<optimized out>, argcount=<optimized out>, kwnames=0x0, kwargs=0x0, kwcount=<optimized out>, kwstep=2, defs=0x0, defcount=0, kwdefs=0x0, closure=0x0, name=0x0, qualname=0x0) at /tmp/build/80754af9/python_1599203911753/work/Python/ceval.c:4298
#70 0x000055783cbb8754 in PyEval_EvalCodeEx (_co=<optimized out>, globals=<optimized out>, locals=<optimized out>, args=<optimized out>, argcount=<optimized out>, kws=<optimized out>, kwcount=0, defs=0x0, defcount=0, kwdefs=0x0, closure=0x0) at /tmp/build/80754af9/python_1599203911753/work/Python/ceval.c:4327
#71 0x000055783cc46edc in PyEval_EvalCode (co=<optimized out>, globals=<optimized out>, locals=<optimized out>) at /tmp/build/80754af9/python_1599203911753/work/Python/ceval.c:718
#72 0x000055783cc46f84 in run_eval_code_obj (co=0x7f90d013d660, globals=0x7f90d0170b40, locals=0x7f90d0170b40) at /tmp/build/80754af9/python_1599203911753/work/Python/pythonrun.c:1125
#73 0x000055783cc791f4 in run_mod (mod=<optimized out>, filename=<optimized out>, globals=0x7f90d0170b40, locals=0x7f90d0170b40, flags=<optimized out>, arena=<optimized out>) at /tmp/build/80754af9/python_1599203911753/work/Python/pythonrun.c:1147
#74 0x000055783cb416e1 in PyRun_FileExFlags (fp=0x55783debb710, filename_str=<optimized out>, start=<optimized out>, globals=0x7f90d0170b40, locals=0x7f90d0170b40, closeit=1, flags=0x7ffe49b405d8) at /tmp/build/80754af9/python_1599203911753/work/Python/pythonrun.c:1063
#75 0x000055783cb41ac6 in PyRun_SimpleFileExFlags (fp=0x55783debb710, filename=<optimized out>, closeit=1, flags=0x7ffe49b405d8) at /tmp/build/80754af9/python_1599203911753/work/Python/pythonrun.c:428
#76 0x000055783cb4298b in pymain_run_file (cf=0x7ffe49b405d8, config=0x55783de53a10) at /tmp/build/80754af9/python_1599203911753/work/Modules/main.c:387
#77 pymain_run_python (exitcode=0x7ffe49b405d0) at /tmp/build/80754af9/python_1599203911753/work/Modules/main.c:612
#78 Py_RunMain () at /tmp/build/80754af9/python_1599203911753/work/Modules/main.c:691
#79 0x000055783cc7bd19 in Py_BytesMain (argc=<optimized out>, argv=<optimized out>) at /tmp/build/80754af9/python_1599203911753/work/Modules/main.c:1137
#80 0x00007f90d03fa09b in __libc_start_main (main=0x55783cb43460 <main>, argc=3, argv=0x7ffe49b407c8, init=<optimized out>, fini=<optimized out>, rtld_fini=<optimized out>, stack_end=0x7ffe49b407b8) at ../csu/libc-start.c:308
#81 0x000055783cc0be93 in _start () at ../sysdeps/x86_64/elf/start.S:103
(gdb)
(gdb)
ajoubertza commented 3 years ago

@jairomoldes and @tiagocoutinho It would be useful to know what you have already tried, so that we don't repeat those efforts.

tiagocoutinho commented 3 years ago

We tried to intercept the DeviceProxy destructor. In this hook we release the GIL and try to unsubscribe from all events.

Fortunately @jairomoldes kept the reference to our tests. You can find it here: https://github.com/jairomoldes/pytango/tree/315