the service cannot be started when using in k8s pod and response "find: The environment is too large for exec()."

[ericyau82]

used image:

redmine:5.0.5-alpine3.18

the service response

find: The environment is too large for exec().

when starting the service

---

finally, i fixed the issue by updating the official function _fix_permissions() used in

/docker-entrypoint.sh

updated function:

_fix_permissions() {
    # https://www.redmine.org/projects/redmine/wiki/RedmineInstall#Step-8-File-system-permissions
    # local dirs=( config log public/plugin_assets tmp ) args=()
    # if [ "$(id -u)" = '0' ]; then
    #   args+=( ${args[@]:+,} '(' '!' -user redmine -exec chown redmine:redmine '{}' + ')' )

    #   # https://github.com/docker-library/redmine/issues/268 - scanning "files" might be *really* expensive, so we should skip it if it seems like it's "already correct"
    #   local filesOwnerMode
    #   filesOwnerMode="$(stat -c '%U:%a' files)"
    #   if [ "$files" != 'redmine:755' ]; then
    #       dirs+=( files )
    #   fi
    # fi
    # # directories 755, files 644:
    # args+=( ${args[@]:+,} '(' -type d '!' -perm 755 -exec sh -c 'chmod 755 "$@" 2>/dev/null || :' -- '{}' + ')' )
    # args+=( ${args[@]:+,} '(' -type f '!' -perm 644 -exec sh -c 'chmod 644 "$@" 2>/dev/null || :' -- '{}' + ')' )
    # find "${dirs[@]}" "${args[@]}"
    mkdir -p tmp tmp/pdf public/plugin_assets
    chown -R redmine:redmine files log tmp public/plugin_assets
    chmod -R 755 files log tmp public/plugin_assets
}

may i know if there is any other solution? will it be fixed on the coming update? thanks a lot

[tianon]

That sounds like you might have too many (or too big) environment variables, somehow? :grimacing: Does your pod have a lot set?

If that's it, you could fix it in this specific instance by prefixing find with something like env -i (find env -i ...), but if my assumption about environment variables is correct this might just be the tip of the iceberg. :see_no_evil:

[ericyau82]

yup, many environment variables is set by our deployment script before (around 40 up are set) 00|

[tianon]

Are all the variables you're setting on the container actually relevant to Redmine? Does it read them all? Could some of them be files instead? (This seems like a pretty big edge case that's going to be hard for us to justify supporting :see_no_evil:)

[ericyau82]

thx for your reply

not really, actually those variable are set by the deployer(our team use gitlab to trigger a common deployer for deploying the project. And, i can just report to the corresponding team for follow up and cannot update the script directly. While thecorresponding team view it as a edge case too as no other projects reported similar issue and much projects need to be re test for the deployment if the deployer is updated🙈🙈)

actually, the solution is still work for recent update (redmine:5.1.1-alpine3.18), i will keep using it temporarily.

thanks for your follow up

[ericyau82]

btw, finding most of the environment variables are come from the Kubernetes control plane services now (glowing up to over 4000 as our testing projects are using the same Kubernetes cluster, most of them are XXX_SERVICE_HOST, XXX_SERVICE_PORT) i think other developers (especially working on the same company) might face this issue too

ref: https://kubernetes.io/docs/concepts/containers/container-environment/ ...A list of all services that were running when a Container was created is available to that Container as environment variables. This list is limited to services within the same namespace as the new Container's Pod and Kubernetes control plane services... e.g.

FOO_SERVICE_HOST=<the host the service is running on>
FOO_SERVICE_PORT=<the port the service is running on>

FYR

[tianon]

Oh boy, that's interesting -- could you use namespaces to separate some of your services? It seems ... unusual ... to have so many services in a single namespace that the number of environment variables created on containers within that namespace overwhelm the kernel's environment variable limits. :grimacing:

[ericyau82]

yup, might able to seperate the namespace, but needing to check with infra. teammates for the feasibility (don't know if much effort for changing their monitoring tool settings if seperating the namespace🙈) thanks for your suggestion🙏