dripline-python v4.5.0 doesn't seem to handle ctrl-c correctly

[raphaelcervantes]

I upgraded to dripline v4.5.0 and now I'm getting this error when I try to do a ctrl-c.

): Handling termination; signal <2>
^C2020-09-24 00:43:18 [ PROG] ignal_handler.cc(205): Handling termination; signal <2>

For context, there's things running simultaneously. But I think it's really just a bunch of get and set dripline commands.

from dripline.core import Interface
import time
import common_functions
auths_file = '/etc/rabbitmq-secret/authentications.json'
the_interface = Interface(dripline_config={'auth-file': auths_file})
the_interface.set('curved_mirror_move_to_position', 0)
the_interface.set('bottom_dielectric_plate_move_to_position', 0)
the_interface.set('top_dielectric_plate_move_to_position', 0)
common_functions.wait_for_motors()

#returns the sttus of the motor as a string
def get_status():
    bottom_plate_status = the_interface.get('bottom_dielectric_plate_motor_request_status').payload.to_python()['value_raw']
    top_plate_status = the_interface.get('top_dielectric_plate_motor_request_status').payload.to_python()['value_raw']
    curved_mirror_status = the_interface.get('curved_mirror_motor_request_status').payload.to_python()['value_raw']
    return [curved_mirror_status,top_plate_status,bottom_plate_status]
def wait_for_motors():
    while (get_status() != ['R','R','R']):
        print(get_status())
        time.sleep(1)
    print('done waiting')

[raphaelcervantes]

@nsoblath @laroque dripline-python v4.5.1 does not fix this issue. I am reverting back to 4.4.x for my pod that runs data taking scripts.

root@clerk-dripline-python-deployment-7c9456f875-55gct:/usr/local/src/dripline-python-plugin/data_taking_scripts# python motor_control.py 
fail!
fail!
  Configurations  
0  empty_modemap
1  dielectric_modemap
2  ln2_empty_modemap
3  ln2_dielectric_modemap
Pick you configuration (0 - 3): 0
Enter the distance to move in cm (Empty resonator modemap is usually 3): 0.1
Enter the number of measurements needed: 1
Describe the current measurement setup: ^C2021-03-02 18:02:48 [ PROG] ignal_handler.cc(205): Handling termination; signal <2>
^[[A^C2021-03-02 18:02:49 [ PROG] ignal_handler.cc(205): Handling termination; signal <2>
^C2021-03-02 18:02:50 [ PROG] ignal_handler.cc(205): Handling termination; signal <2>
^C2021-03-02 18:02:51 [ PROG] ignal_handler.cc(205): Handling termination; signal <2>
^C2021-03-02 18:02:51 [ PROG] ignal_handler.cc(205): Handling termination; signal <2>
^C2021-03-02 18:02:52 [ PROG] ignal_handler.cc(205): Handling termination; signal <2>
^C2021-03-02 18:02:52 [ PROG] ignal_handler.cc(205): Handling termination; signal <2>
^C2021-03-02 18:02:52 [ PROG] ignal_handler.cc(205): Handling termination; signal <2>

[nsoblath]

I'm addressing this in branch hotfix/4.5.4.

[raphaelcervantes]

@nsoblath I have bad news...

I finally reassembled Orpheus and can now test my data taking scripts in ernest. As of 4.5.5, this problem doesn't seemed to be fixed.

I enter a data taking script and try to exit it with ctrl-c. In the debug messages, it says it's cancelling all cancelables and then just continues with what is going on. Do you know what's going on?

root@clerk-dripline-python-deployment-68f4df4cb-jv42j:/usr/local/src/dripline-python-plugin/data_taking_scripts# python axion_data_taking.py 
2021-07-15 02:32:46 [DEBUG] (tid 139623849248576) ignal_handler.cc(67): Handling SIGABRT (abort() and unhandled exceptions)
2021-07-15 02:32:46 [DEBUG] (tid 139623849248576) ignal_handler.cc(78): Handling SIGTERM
2021-07-15 02:32:46 [DEBUG] (tid 139623849248576) ignal_handler.cc(89): Handling SIGINT (ctrl-c)
2021-07-15 02:32:46 [DEBUG] (tid 139623849248576) ignal_handler.cc(101): Handling SIGQUIT (ctrl-\)
2021-07-15 02:32:46 [DEBUG] (tid 139623849248576) dexed_factory.hh(208): Registered a indexed_factory for class json at 0x7efcb5b34a40, indexed_factory #0 for 0x561de79082a0
2021-07-15 02:32:46 [DEBUG] (tid 139623849248576) dexed_factory.hh(208): Registered a indexed_factory for class json at 0x7efcb5b34a80, indexed_factory #0 for 0x561de7966a70
2021-07-15 02:32:46 [DEBUG] (tid 139623849248576) dexed_factory.hh(208): Registered a indexed_factory for class yaml at 0x7efcb5b34b80, indexed_factory #1 for 0x561de79082a0
2021-07-15 02:32:46 [DEBUG] (tid 139623849248576) dexed_factory.hh(208): Registered a indexed_factory for class yaml at 0x7efcb5b34bc0, indexed_factory #1 for 0x561de7966a70
2021-07-15 02:32:46 [DEBUG] (tid 139623849248576) dexed_factory.hh(208): Registered a indexed_factory for class 0 at 0x7efcb54f00d0, indexed_factory #0 for 0x561de79972a0
2021-07-15 02:32:46 [DEBUG] (tid 139623849248576) dexed_factory.hh(208): Registered a indexed_factory for class 1 at 0x7efcb54f00e0, indexed_factory #1 for 0x561de79972a0
2021-07-15 02:32:46 [DEBUG] (tid 139623849248576) dexed_factory.hh(208): Registered a indexed_factory for class 2 at 0x7efcb54f00f0, indexed_factory #2 for 0x561de79972a0
2021-07-15 02:32:46 [DEBUG] (tid 139623849248576) dexed_factory.hh(208): Registered a indexed_factory for class 3 at 0x7efcb54f0100, indexed_factory #3 for 0x561de79972a0
2021-07-15 02:32:46 [DEBUG] (tid 139623849248576) dexed_factory.hh(208): Registered a indexed_factory for class 4 at 0x7efcb54f0110, indexed_factory #4 for 0x561de79972a0
2021-07-15 02:32:46 [DEBUG] (tid 139623849248576) dexed_factory.hh(208): Registered a indexed_factory for class 5 at 0x7efcb54f0120, indexed_factory #5 for 0x561de79972a0
2021-07-15 02:32:46 [DEBUG] (tid 139623849248576) dexed_factory.hh(208): Registered a indexed_factory for class 100 at 0x7efcb54f0130, indexed_factory #6 for 0x561de79972a0
2021-07-15 02:32:46 [DEBUG] (tid 139623849248576) dexed_factory.hh(208): Registered a indexed_factory for class 101 at 0x7efcb54f0140, indexed_factory #7 for 0x561de79972a0
2021-07-15 02:32:46 [DEBUG] (tid 139623849248576) dexed_factory.hh(208): Registered a indexed_factory for class 102 at 0x7efcb54f0150, indexed_factory #8 for 0x561de79972a0
2021-07-15 02:32:46 [DEBUG] (tid 139623849248576) dexed_factory.hh(208): Registered a indexed_factory for class 200 at 0x7efcb54f0160, indexed_factory #9 for 0x561de79972a0
2021-07-15 02:32:46 [DEBUG] (tid 139623849248576) dexed_factory.hh(208): Registered a indexed_factory for class 201 at 0x7efcb54f0170, indexed_factory #10 for 0x561de79972a0
2021-07-15 02:32:46 [DEBUG] (tid 139623849248576) dexed_factory.hh(208): Registered a indexed_factory for class 202 at 0x7efcb54f0180, indexed_factory #11 for 0x561de79972a0
2021-07-15 02:32:46 [DEBUG] (tid 139623849248576) dexed_factory.hh(208): Registered a indexed_factory for class 203 at 0x7efcb54f0190, indexed_factory #12 for 0x561de79972a0
2021-07-15 02:32:46 [DEBUG] (tid 139623849248576) dexed_factory.hh(208): Registered a indexed_factory for class 300 at 0x7efcb54f01a0, indexed_factory #13 for 0x561de79972a0
2021-07-15 02:32:46 [DEBUG] (tid 139623849248576) dexed_factory.hh(208): Registered a indexed_factory for class 301 at 0x7efcb54f01b0, indexed_factory #14 for 0x561de79972a0
2021-07-15 02:32:46 [DEBUG] (tid 139623849248576) dexed_factory.hh(208): Registered a indexed_factory for class 302 at 0x7efcb54f01c0, indexed_factory #15 for 0x561de79972a0
2021-07-15 02:32:46 [DEBUG] (tid 139623849248576) dexed_factory.hh(208): Registered a indexed_factory for class 303 at 0x7efcb54f01d0, indexed_factory #16 for 0x561de79972a0
2021-07-15 02:32:46 [DEBUG] (tid 139623849248576) dexed_factory.hh(208): Registered a indexed_factory for class 304 at 0x7efcb54f01e0, indexed_factory #17 for 0x561de79972a0
2021-07-15 02:32:46 [DEBUG] (tid 139623849248576) dexed_factory.hh(208): Registered a indexed_factory for class 305 at 0x7efcb54f01f0, indexed_factory #18 for 0x561de79972a0
2021-07-15 02:32:46 [DEBUG] (tid 139623849248576) dexed_factory.hh(208): Registered a indexed_factory for class 306 at 0x7efcb54f0200, indexed_factory #19 for 0x561de79972a0
2021-07-15 02:32:46 [DEBUG] (tid 139623849248576) dexed_factory.hh(208): Registered a indexed_factory for class 307 at 0x7efcb54f0210, indexed_factory #20 for 0x561de79972a0
2021-07-15 02:32:46 [DEBUG] (tid 139623849248576) dexed_factory.hh(208): Registered a indexed_factory for class 308 at 0x7efcb54f0220, indexed_factory #21 for 0x561de79972a0
2021-07-15 02:32:46 [DEBUG] (tid 139623849248576) dexed_factory.hh(208): Registered a indexed_factory for class 310 at 0x7efcb54f0230, indexed_factory #22 for 0x561de79972a0
2021-07-15 02:32:46 [DEBUG] (tid 139623849248576) dexed_factory.hh(208): Registered a indexed_factory for class 400 at 0x7efcb54f0240, indexed_factory #23 for 0x561de79972a0
2021-07-15 02:32:46 [DEBUG] (tid 139623849248576) dexed_factory.hh(208): Registered a indexed_factory for class 401 at 0x7efcb54f0250, indexed_factory #24 for 0x561de79972a0
2021-07-15 02:32:46 [DEBUG] (tid 139623849248576) dexed_factory.hh(208): Registered a indexed_factory for class 402 at 0x7efcb54f0260, indexed_factory #25 for 0x561de79972a0
2021-07-15 02:32:46 [DEBUG] (tid 139623849248576) dexed_factory.hh(208): Registered a indexed_factory for class 403 at 0x7efcb54f0270, indexed_factory #26 for 0x561de79972a0
2021-07-15 02:32:46 [DEBUG] (tid 139623849248576) dexed_factory.hh(208): Registered a indexed_factory for class 404 at 0x7efcb54f0280, indexed_factory #27 for 0x561de79972a0
2021-07-15 02:32:46 [DEBUG] (tid 139623849248576) dexed_factory.hh(208): Registered a indexed_factory for class 999 at 0x7efcb54f0290, indexed_factory #28 for 0x561de79972a0
fail!
fail!
fail!
fail!
  Configurations  
0  empty_resonator
1  dielectric_resonator
Pick you configuration (0 - 1): ^C2021-07-15 02:32:48 [ PROG] (tid 139623849248576) ignal_handler.cc(218): Handling termination; signal <2>
2021-07-15 02:32:48 [DEBUG] (tid 139623849248576) ignal_handler.cc(303): Canceling all cancelables

[raphaelcervantes]

I'm guessing dripline is taking full control of the signal and doesn't let python see it.

[nsoblath]

@raphaelcervantes I haven't been able to replicate this with v4.5.5 using the kv-store. Are you able to replicate the problem with kv-store, or can you give me an example that I can replicate that's still a problem?

[raphaelcervantes]

@nsoblath @laroque I've done a bit of debugging. My diagnosis is that this happens in an execution script after importing the dripline interface. Before the dripline interface is imported, the python script will see the ctrl-c command and will exit promptly. After the python interface is imported, dripline will see the ctrl-c and the python script does not see it.

To diagnose this problem, I'm modified the tutorial's JitterEntity to have a waiting function: https://github.com/axiondarkmatterexperiment/dripline-orpheus/blob/1761817b7104d1eb09115a5d8548e170a2a4ad1c/dripline/extensions/jitter/jitter_endpoint.py#L52

Then I have an execution script where I test the ctrl-c command at different points of execution. https://github.com/axiondarkmatterexperiment/dripline-orpheus/blob/feature/datataking_with_checks_bin_alignment/data_taking_scripts/interface_test.py

The results are in the inline comments. Let me know if you have any questions.

Bottom line: I believe the issue is the python interface.

[nsoblath]

This should be finallly fixed in v4.5.6.