RasaHQ / rasa-x-helm

Rasa Enterprise Helm chart for deploying on Kubernetes (K8s) and OpenShift.
Apache License 2.0
76 stars 104 forks source link

nginx instance will be in a crashloop if there's no custom action server #35

Closed hankshz closed 2 years ago

hankshz commented 4 years ago

The following lines in /opt/bitnami/entrypoint.sh of rasa/nginx will run quite a while: (14(curl timeout for me) + 1(sleep)) * 10 = 150

check_if_app_container_is_connected () {
    number_of_tries=0
    curl_result_code=${CURL_STATUS_CODE_FOR_HOST_NOT_FOUND}

    # Retry this ten times so that we can be sure
    while [[ ${number_of_tries} -lt 10 ]]  && [[ ${curl_result_code} == ${CURL_STATUS_CODE_FOR_HOST_NOT_FOUND} ]]
    do
        curl --output /dev/null app  # Check if custom action server is up
        curl_result_code=$?  # Save the result of the check
        echo ${curl_result_code}
        number_of_tries=$((number_of_tries+1))  # Increment the number of tries

        sleep 1s  # wait 1s between the retries
    done

    return ${curl_result_code}
}

While by default, the liveness probing (https://github.com/RasaHQ/rasa-x-helm/blob/master/charts/rasa-x/values.yaml#L2340) will probably fail by then. I think the proper fix is probably to add a timeout to curl in entrypoint.sh, but docker image rasa/nginx is private so I report it here.

Obviously, the workaround (or fix here) now is to override (or change) the initialProbeDelay with a bigger number.

hankshz commented 4 years ago

I also noticed that the helm package has a default action server. So ideally curl in check_if_app_container_is_connected should not timeout neither. I'm not an expert on k8s. So I don't understand how host 'app' is supposed to setup and why 'curl --output /dev/null app' fails here. Could anyone help? Thanks!

akelad commented 4 years ago

Thanks for raising this issue, @alwx will get back to you about it soon.

RASADSA commented 2 years ago

this is bitnami related - since we include it as a subchart its not fixiable on our side. closing the issue