Switching Method set ot "System Command" messes up external scripts

[SysGh-st]

I need to run multiple curl commands on the power on and power off events. For this I create two simple shell scripts ( #!/bin/sh ) that lists each curl command needed.

Script for turning on: /usr/local/bin/3D-Printer_PowerOn

#!/bin/sh
# Hue light ID set below is a power outlet switch that is used to turn the printer on and off.
OCTOPRINT_APIKEY="FFFD7EFEAB1844C97AB087AFDE4DB33F"
OCTOPRINT_HOST="localhost"
HUE_APIKEY="xXxxRQxgktEEWB00b54reGr3atf8KTsCBm6eWtxGo9XX"
HUE_HOST="philips-hue.local"
HUE_LIGHT_ID="5"

# Turn printer on by triggering the hue switch
/usr/bin/curl -s -X PUT -H "Content-Type: application/json" -d "{\"on\":true}" http://$HUE_HOST/api/$HUE_APIKEY/lights/$HUE_LIGHT_ID/state > /dev/null 2>&1
/bin/sleep 6s
# Enable default serial connection to the printer
/usr/bin/curl -s -H "Content-Type: application/json" -H "X-Api-Key:$OCTOPRINT_APIKEY" -X POST -d "{ \"command\" : \"connect\" }" http://$OCTOPRINT_HOST/api/connection > /dev/null 2>&1
/bin/sleep 10s
# Turn on the printers internal case light with the M355 S1 command
/usr/bin/curl -s -H "Content-Type: application/json" -H "X-Api-Key:$OCTOPRINT_APIKEY" -X POST -d "{ \"command\" : \"M355 S1\" }" http://$OCTOPRINT_HOST/api/printer/command > /dev/null 2>&1

Script for turning off: /usr/local/bin/3D-Printer_PowerOff

#!/bin/bash
# Hue light ID set below is a power outlet switch that is used to turn the printer on and off.
OCTOPRINT_APIKEY="FFFD7EFEAB1844C97AB087AFDE4DB33F"
OCTOPRINT_HOST="localhost"
HUE_APIKEY="xXxxRQxgktEEWB00b54reGr3atf8KTsCBm6eWtxGo9XX"
HUE_HOST="philips-hue.local"
HUE_LIGHT_ID="5"

# Turn off the printers internal case light with the M355 S0 command
/usr/bin/curl -s -H "Content-Type: application/json" -H "X-Api-Key:$OCTOPRINT_APIKEY" -X POST -d "{ \"command\" : \"M355 S0\" }" http://$OCTOPRINT_HOST/api/printer/command > /dev/null 2>&1
/bin/sleep 1s
# Disconnect the serial port.
/usr/bin/curl -s -H "Content-Type: application/json" -H "X-Api-Key:$OCTOPRINT_APIKEY" -X POST -d "{ \"command\" : \"disconnect\" }" http://$OCTOPRINT_HOST/api/connection > /dev/null 2>&1
/bin/sleep 1s
# Turn the power off with the Hue switch.
/usr/bin/curl -s -X PUT -H "Content-Type: application/json" -d "{\"on\":false}" http://$HUE_HOST/api/$HUE_APIKEY/lights/$HUE_LIGHT_ID/state > /dev/null 2>&1

As to why I need this is to switch the case lights off when the printer is turned off. I power the internal case lights from the same power supply as the OctoPi computer, so that cannot be part of the power switching routine. There are other things I've added to these on and off scripts as well, but that's outside the scope of this report. This is just a basic example.

These scripts run just fine if they're run from a terminal on the OctoPi computer, so that part works just as intended.

1. Upload the scripts to the OctoPi computer with executable flag set. I store them in /usr/local/bin/ to make them available through the $PATH variable. Ownership set to root:root. Permission is 770 octal.
2. Configure PSUControl for "System Call" and set the on and off fields with the scripts accordingly (With full path just in case).
3. Watch as the OctoPrint/PSUControl fails to properly execute the external scripts.
4. Investigation shows that PSUControl removes all the important quotation from the curl commands, ending up in a malformed curl command getting stuck/hung. curl is shown to have unquoted headers and data, which causes the curl to stay in an interactive state in the background waiting for user input.
5. Rectify the problem by manually logging in and kill the attempted, but incorrectly called, curl commands stuck in backgrounded interactive modes due to the stripped quotation. Example of the curl command without the proper quotes causing it to hang: /usr/bin/curl -s -X PUT -H Content-Type: application/json -d {"on":false} http://philips-hue.local/api/xXxxRQxgktEEWB00b54reGr3atf8KTsCBm6eWtxGo9XX/lights/5/state > /dev/null 2>&1

What did you expect to happen?

I expected PSUControl to be able to properly call a system-side shell-script without messing up the scripts internal quotation. Just as if they where called from a terminal. Mentioned scripts run just fine when manually called from a terminal.

What happened instead?

Scripts are called as intended. But PSUControl messes up important quotation within the script turning the scripts curl commands invalid.

Version of OctoPrint-PSUControl

0.1.9

Operating System running OctoPrint

Raspberry Pi 4 with OctoPi 0.17 uname -a : Linux octopi 5.4.51-v8+ #1333 SMP PREEMPT Mon Aug 10 16:58:35 BST 2020 aarch64 GNU/Linux Recently updated with apt-get update && apt-get-upgrade (2020-08-30)

Printer model & used firmware incl. version

Creality Ender 3 upgraded with BigTreeTech SKR MiniE3 v2.0 32 bit mainboard. Latest Marlin 2.0.x bugfix firmware (2.0.6)

Link to octoprint.log with octoprnt.plugins.psucontrol set to DEBUG

https://gist.github.com/SysGh-st/7e3dcbb12eae4ff284dd9e5e68b5b265

Wiring diagram

[SysGh-st]

I've discovered that the "System Call" fields are called with eval. (The evil eval) This do indeed screw up quotation. Somehow it calls the scripts messing up the internal quotation as well. A long-known problem with eval among other more serious issues with eval. The general recommendation is to avoid eval as much as possible. I suspect eval can't be avoided in this plugin, so I'm currently rewriting my scripts to survive the "evil eval bashing".

Suggestion: Have another option under "Switching Method" called "Scripts" that gives the user two multi-line fileds for "on" and "off" where the user can paste/write complete shell scripts. Then run these scripts without involving eval.

[SysGh-st]

For the records, this is how the scripts now look in order to survive the eval processing. It really takes quite the variable-juggling in order for it all to stay quoted and intact as intended. For such a simple task, it has grown a lot just because of eval. I know it can be done in a more elegant way with loops and cases. But that is tricky to get through the eval process. Not impossible though. It just takes some black belt quote-ninja to do it. Quotes through eval gives me a headache.

/usr/local/bin/3D-Printer_PowerOn

#!/bin/bash
# Be vary when editing this script. It's being called with eval, thus quotation within variables are super-sensitive. 
# See: https://stackoverflow.com/questions/5253782/bash-problem-with-eval-variables-and-quotes
# ----

# Hue light ID set below is a power outlet switch that is used to turn the printer on and off.
OCTOPRINT_APIKEY="<API-KEY>"
OCTOPRINT_HOST="localhost"
HUE_APIKEY="<API-KEY>"
HUE_HOST="philips-hue.local"
HUE_LIGHT_ID="5"

# Turn main power on to the printer by switching on the philips hue light ID (which is a ZigBee-capable power outlet switch)
curlUrl="http://"$HUE_HOST"/api/"$HUE_APIKEY"/lights/"$HUE_LIGHT_ID"/state"
curlContentType="Content-Type:application/json"
curlDataPack="{\"on\":true}"
curlResponse="$(curl -s -X PUT "$curlUrl" -H "$curlContentType" -d "$curlDataPack" )"
echo ${curlResponse}
/bin/sleep 6s

# Connect the serial port.
curlUrl="http://"$OCTOPRINT_HOST"/api/connection"
curlContentType="Content-Type:application/json"
curlXApiKey="X-Api-Key:"$OCTOPRINT_APIKEY
curlDataPack="{\"command\":\"connect\"}"
curlResponse="$(curl -s -X POST "$curlUrl" -H "$curlContentType" -H "$curlXApiKey" -d "$curlDataPack" )"
echo ${curlResponse}
/bin/sleep 10s

# Turn case lighting on by sending the gcode-command M355 S1
curlUrl="http://"$OCTOPRINT_HOST"/api/printer/command"
curlContentType="Content-Type:application/json"
curlXApiKey="X-Api-Key:"$OCTOPRINT_APIKEY
curlDataPack="{\"command\":\"M355 S1\"}"
curlResponse="$(curl -s -X POST "$curlUrl" -H "$curlContentType" -H "$curlXApiKey" -d "$curlDataPack" )"
echo ${curlResponse}

/usr/local/bin/3D-Printer_PowerOff

#!/bin/sh
# Be vary when editing this script. It's being called with eval, thus quotation within variables are super-sensitive. 
# See: https://stackoverflow.com/questions/5253782/bash-problem-with-eval-variables-and-quotes
# ----

# Hue light ID set below is a power outlet switch that is used to turn the printer on and off.
OCTOPRINT_APIKEY="<API-KEY>"
OCTOPRINT_HOST="localhost"
HUE_APIKEY="<API-KEY>"
HUE_HOST="philips-hue.local"
HUE_LIGHT_ID="5"

# Turn case lighting off by sending the gcode-command M355 S0
curlUrl="http://"$OCTOPRINT_HOST"/api/printer/command"
curlContentType="Content-Type:application/json"
curlXApiKey="X-Api-Key:"$OCTOPRINT_APIKEY
curlDataPack="{\"command\":\"M355 S0\"}"
curlResponse="$(curl -s -X POST "$curlUrl" -H "$curlContentType" -H "$curlXApiKey" -d "$curlDataPack" )"
echo ${curlResponse}
/bin/sleep 1s

# Disconnect the serial in order to allow for a clean shutdown.
# If the serial remains connected when powering off, it can leave behind stale dev nodes that can only be cleaned up by rebooting.
curlUrl="http://"$OCTOPRINT_HOST"/api/connection"
curlContentType="Content-Type:application/json"
curlXApiKey="X-Api-Key:"$OCTOPRINT_APIKEY
curlDataPack="{\"command\":\"disconnect\"}"
curlResponse="$(curl -s -X POST "$curlUrl" -H "$curlContentType" -H "$curlXApiKey" -d "$curlDataPack" )"
echo ${curlResponse}
/bin/sleep 3s

# Turn main power off to the printer by switching off the philips hue light ID (which is a ZigBee-capable power outlet switch)
curlUrl="http://"$HUE_HOST"/api/"$HUE_APIKEY"/lights/"$HUE_LIGHT_ID"/state"
curlContentType="Content-Type:application/json"
curlDataPack="{\"on\":false}"
curlResponse="$(curl -s -X PUT "$curlUrl" -H "$curlContentType" -d "$curlDataPack" )"
echo ${curlResponse}

[SysGh-st]

After some more experimentation, I discovered that if one put nohup /bin/bash -c "/path/to/somescript" &>/dev/null & it'll launch /path/to/somescript with no quotations malformed. (and the nohup + backgrounding also makes sure the interface won't get crippled in case the script do hang itself). Best would be if one could call a script without these tricks.

Using /bin/bash -c "/path/to/somescript" alone does not work. I don't know why, but my guess is that nohup puts the script in a "proper" shell environment, which the PSUControl plugin doesn't do.

EDIT: For the records only for those who find this useful and want to use Hue Switches as power-management: This is what I use as a "Sensing System Command": eval $(curl -s -H 'Content-Type: application/json' 'http://philips-hue.local/api/<YourAPIKeySitsHere>/lights/<SwitchID>' | jq -r '.state.on') . Replace YourAPIKeySitsHere and SwitchID with your values.

[kantlivelong]

To be honest I don't understand the problem. You've provided a lot of specific info to your setup but what's really needed is a simplified example with steps to reproduce.

For example I can set senseSystemCommand to echo "\"test\"" > /home/pi/test.txt and it works as expected. I can also set it to /home/pi/test.sh which contains:

#!/bin/bash
echo "\"test\"" > /home/pi/test.txt

and that works as expected.

[kantlivelong]

Closing due to inactivity.

[lightmaster]

Would be awesome if this wasn't closed and was fixed such that you could directly send a curl command without it screwing up.

[kantlivelong]

@lightmaster got an example?

[lightmaster]

Trying to send curl -s -H "Content-Type: application/json" -H "X-Api-Key: <API KEY>" -X POST -d '{ "command":"turnOn", "topic":"Q5Printer", "relayN":"0" }' http://localhost:5000/api/plugin/tasmota_mqtt and it hangs the entire web interface up until I run sudo killall curl from console. As soon as I run that, it actually sends the curl request and turns on the power on. From Postman or bash directly, it works as expected.

[kantlivelong]

Out of curiosity, does it work with https://github.com/kantlivelong/OctoPrint-GCodeSystemCommands ?

[lightmaster]

I'm installing it to try now

[lightmaster]

Yup, sending that curl to turn it off does turn the printer off and I do not have a lingering curl left trying to run forever

[andrimarjonsson]

Having the same issue. System commands that work fine when invoked from the shell but not when entered in the on and off command fields in the plugin settings.

What is more odd is that I extracted the code from the plugin that runs the commands. Entered it in an interactive session using the interpreter bin from the venv and as the octoprint user. When copy pasting the command into there, it works fine. Only when run from octoprint does it not work.

There seems to be an issue with octoprint logging as well now, preventing me from turning on debug logging for the plugin.