Flow_refresh.sh Script failed in Sidecar Container

hinrichd commented 1 year ago

What happened?

I try to import a flow from Config File via the Sidecar Container but the flow_refresh.sh script is failing. The flow will not imported via the API Refresh.

Actually I am using pv/pvc and install flows and nodes manually via web interface. Thought I would be a good way to use a sidecar like kubernetes microservice style.

k logs node-red-.... node-red-sidecar 
{"time": "2023-04-26T12:17:07.098556+00:00", "msg": "Starting collector", "level": "INFO"}
{"time": "2023-04-26T12:17:07.100418+00:00", "msg": "No folder annotation was provided, defaulting to k8s-sidecar-target-directory", "level": "WARNING"}
{"time": "2023-04-26T12:17:07.100861+00:00", "msg": "Loading incluster config ...", "level": "INFO"}
{"time": "2023-04-26T12:17:07.101686+00:00", "msg": "Config for cluster api at 'https://10.200.0.1:443' loaded...", "level": "INFO"}
{"time": "2023-04-26T12:17:07.102336+00:00", "msg": "Unique filenames will not be enforced.", "level": "INFO"}
{"time": "2023-04-26T12:17:07.102516+00:00", "msg": "5xx response content will not be enabled.", "level": "INFO"}
{"time": "2023-04-26T12:17:17.129633+00:00", "msg": "Script failed with error: Command '['sh', '/app/flow_refresh.sh']' returned non-zero exit status 127.", "level": "ERROR"}

How can we reproduce this?

Connecting to the sidecar shell and checking the flow_refresh script:

k exec  node-red-..... -it -c node-red-sidecar -- sh
/app $ sh flow_refresh.sh 
node-red flow refresh api
flow_refresh.sh: line 4: curl: not found
flow_refresh.sh: line 4: jq: not found
flow_refresh.sh: line 5: curl: not found

The k8s-sidecar container is missing curl and jq binary! Is this the common behaviour?

Helm Chart version

0.23.1

Search

[X] I did search for other open and closed issues before opening this.

Code of Conduct

[X] I agree to follow this project's Code of Conduct

Additional context

No response

dirien commented 1 year ago

Yepp @hinrichd, thats what we got asked here too: https://github.com/SchwarzIT/node-red-chart/issues/149

The issue is, I am not sure if this project should take the semi-ownership of the https://github.com/kiwigrid/k8s-sidecar project.

Maybe worth to file a feature-request there and ask to add curl and jq to the base image?

hinrichd commented 1 year ago

Hey @dirien I have opened a ticket in the k8s-sidecar project . Let´s see what they can do.

dirien commented 1 year ago

awesome @hinrichd! Let's see what the answer is!

hinrichd commented 1 year ago

Hi @dirien, here is the answer for the sidecar project...

Adding more binaries to the image does not really match the philosophy of the project having a minimal image and hence minimal attack surface. But we're building upon a generic python image and already do have the python requests package installed. So I think it should be possible to use python instead of separate binaries to perform the requests and parse JSON responses.

I totally agree with this comment. To fit best into the sidecar the flowrefresh script should be using python and the request package/module as dependency.

I am only the mac and bash guy and not python. Will have a look at psf requests in some days, it says it is for humans... ✌️

hinrichd commented 1 year ago

UPDATE: Now, there is a new version of the k8s-sidecar release called 1.24.0. In this version the SCRIPT Environment Variable will execute python scripts as well by respecting the script´s shebang. Before, the script was directly called by sh command and you had to run python from within the script as subprocess. This has some downsides to stdin/out. See this FR related to the original issue.

Meanwhile I rewrite the flow_refresh.sh script to flow_refresh.py due to dependencies of the python based image. Executing manually the flow_refresh.py with .\flow_refresh.py inside the sidecar will reload the node-red flows:

--> Script Output from Sidecar Container

/tmp $ hostname
node-red-54d6679cb-m9qfr
/tmp $ vi flow_refresh.py
/tmp $ chmod 755 flow_refresh.py 
/tmp $ ./flow_refresh.py 
node-red flow refresh api via k8s-sidecar
Sidecar sleeping for 5 seconds...
node-red flow refresh api response code = 200

--> Node-Red Log Output

8 May 12:47:30 - [warn] Verschlüsselte Credentials nicht gefunden
8 May 12:47:30 - [info] Flows werden gestoppt
8 May 12:47:30 - [info] Flows sind gestoppt
8 May 12:47:30 - [info] Flows werden gestartet
8 May 12:47:30 - [info] Flows sind gestartet

flow_refresh.py

Default Chart SLEEP_TIME_SIDECAR must be integer not string (5s)
retry and retry delay not supported
flow_refresh.py has to be executable, otherwise it will be executed be sh an will fail.

#!/usr/bin/env python

import time
import os
import requests
import json
import sys

# SET VARIABLES FROM CONTAINER ENVIRONMENT
SLEEP_TIME_SIDECAR = 5 if os.getenv("SLEEP_TIME_SIDECAR") is None else int(os.getenv("SLEEP_TIME_SIDECAR"))
USERNAME = os.getenv("USERNAME")
PASSWORD = os.getenv("PASSWORD")
URL = os.getenv("URL")

print('node-red flow refresh api via k8s-sidecar')
print('Sidecar sleeping for', SLEEP_TIME_SIDECAR, 'seconds...')
time.sleep(SLEEP_TIME_SIDECAR)

# GET NODE RED BEARER TOKEN
PAYLOAD_TOKEN = {"client_id": "node-red-admin", "grant_type": "password", "scope": "*", "username": USERNAME, "password": PASSWORD}
r_token = requests.post(URL + '/auth/token', data=PAYLOAD_TOKEN, timeout=30,  verify=False)
token = (json.loads(r_token.text)["access_token"])

# FLOW REFRESH/RELOAD FLOWS FROM SECRET/CONFIGMAP
PAYLOAD_FLOW_REFRESH = "{\"flows\": [{\"type\": \"tab\"}]}"
HEADERS_FLOW_REFRESH={
  'Authorization': 'Bearer' + ' ' + token,
  'content-type': 'application/json; charset=utf-8',
  'Node-RED-Deployment-Type': 'reload',
  'Node-RED-API-Version': 'v2'
}

r_flow_refresh = requests.post(URL + '/flows', headers=HEADERS_FLOW_REFRESH, data=PAYLOAD_FLOW_REFRESH, timeout=30,  verify=False) 

if r_flow_refresh.status_code == requests.codes.ok:
    print('node-red flow refresh api respoonse code =', r_flow_refresh.status_code)
    sys.exit(0)   
else:
    sys.exit('Error-Code', r_flow_refresh.status_code)

dirien commented 1 year ago

@hinrichd many thanks for the contribution!

SchwarzIT / node-red-chart