Curator running failed in docker when upgrade to 8.0.8 as permssion error

[Jae1313]

For usage questions and help

Please create a topic at https://discuss.elastic.co/c/elasticsearch

Perhaps a topic there already has an answer for you!

To submit a bug or report an issue

Expected Behavior

Actual Behavior

Steps to Reproduce the Problem

1. install curator in docker, run the curator as elasticsearch user instead of root user command: su elasticsearch /bin/bash -c "/usr/local/bin/curator --config /config/curator-config.yml /config/curator-actions.yml" curator-config.yml example: elasticsearch: client: hosts:
   • https://aln-tcdvc7vm-69.labs.netscout.com:9200 cloud_id: ca_certs: /config/certs/cacert.pem client_cert: client_key: verify_certs: request_timeout: 30 other_settings: master_only: False username: password: api_key: id: api_key: token:

logging: loglevel: INFO logfile: /logs/es-curator-actions.log logformat: default blacklist: ['elastic_transport', 'urllib3']

Command result as following error: Traceback (most recent call last): File "/usr/local/bin/curator", line 8, in sys.exit(cli()) File "/usr/local/lib/python3.8/dist-packages/click/core.py", line 1157, in call return self.main(args, kwargs) File "/usr/local/lib/python3.8/dist-packages/click/core.py", line 1078, in main rv = self.invoke(ctx) File "/usr/local/lib/python3.8/dist-packages/click/core.py", line 1434, in invoke return ctx.invoke(self.callback, ctx.params) File "/usr/local/lib/python3.8/dist-packages/click/core.py", line 783, in invoke return __callback(args, *kwargs) File "/usr/local/lib/python3.8/dist-packages/click/decorators.py", line 34, in new_func return f(get_current_context(), args, **kwargs) File "/usr/local/lib/python3.8/dist-packages/curator/cli.py", line 316, in cli set_logging(check_logging_config( File "/usr/local/lib/python3.8/dist-packages/curator/config_utils.py", line 51, in set_logging loginfo = LogInfo(log_opts) File "/usr/local/lib/python3.8/dist-packages/curator/logtools.py", line 136, in init self.handler = logging.FileHandler('/proc/1/fd/1') File "/usr/lib/python3.8/logging/init.py", line 1147, in init StreamHandler.init(self, self._open()) File "/usr/lib/python3.8/logging/init.py", line 1176, in _open return open(self.baseFilename, self.mode, encoding=self.encoding) PermissionError: [Errno 13] Permission denied: '/proc/1/fd/1'

Specifications

• Version: curator 8.0.8
• Platform: elasticsearch 8.8.2 docker version with Ubuntu 20.04.6 LTS
• Subsystem:

Context (Environment)

Detailed Description

[untergeek]

This is clearly a container you built using Python 3.8, which means you did not use the provided Dockerfile which builds with 3.11. I can’t say for sure what’s going on here as I do not know what version of Docker you built this with, but the line that includes logging.FileHandler('/proc/1/fd/1') writes logs to the Docker socket on newer versions of Docker so that you can run docker logs CONTAINERNAME to view the logs. This works perfectly with the Curator image published to Docker hub.

Please provide more information.

UPDATE: The code will always try to log to that path when Docker is detected. The following work around is invalid. A manual workaround is to force Curator to log to a file inside the Docker image (which you can, of course, map to a local file path). This will force Curator to not log to a path usable by docker logs but to your specified path.

[Jae1313]

Which docker version have been tested to work fine? I guess maybe my docker version is a bit lower. What I am using is: $ docker --version Docker version 20.10.9, build c2ea9bc

[untergeek]

It’s also the Dockerfile. You’re not using the current Dockerfile if you’re building with Python 3.8. Try building with the current Dockerfile before worrying about the Docker version, or just pull image untergeek/curator:8.0.8 and use that.

[Jae1313]

In our deployment, we do not deploy dockerized curator. What we do is deploy dockerized Elasticsearch, and install curator in Elasticsearch Dockerfile , something like: RUN ln -s /usr/bin/python3.8 /usr/bin/python && \ pip install cryptography elasticsearch-curator

In this case, curator will still detect it as is_docker. In this scenario, I dont think it needs to write logs to the Docker socket.

[untergeek]

I can’t agree that I think this is a good idea, to be honest, precisely because of code changes like this, that turned out to be breaking changes to your setup.

I will see about putting a a write test and ignore that path if it doesn’t have permissions, but it will not be added for a two weeks at best, I would guess.

[Jae1313]

We could wait the fix then upgrade the curator to the latest one, currently our short term solution is downgrade to 8.0.4 which do not have is_docker detect. Thanks for your support.

[Kapitan777]

It’s also the Dockerfile. You’re not using the current Dockerfile if you’re building with Python 3.8. Try building with the current Dockerfile before worrying about the Docker version, or just pull image untergeek/curator:8.0.8 and use that.

When I use this image or create image from https://github.com/elastic/curator/blob/master/Dockerfile always get next output:

{"@timestamp": "2024-01-29T12:37:02.786Z", "function": "run", "linenum": 226, "loglevel": "INFO", "message": "Preparing Action ID: 1, \"delete_indices\"", "name": "curator.cli"} {"@timestamp": "2024-01-29T12:37:02.786Z", "function": "run", "linenum": 233, "loglevel": "INFO", "message": "Creating client object and testing connection", "name": "curator.cli"} {"@timestamp": "2024-01-29T12:37:02.787Z", "function": "get_client", "linenum": 73, "loglevel": "INFO", "message": "Creating client object and testing connection", "name": "curator.helpers.getters"} argument should be a bytes-like object or ASCII string, not 'NoneType'

Wrapper for running curator from source.

When used with Python 3 Curator requires the locale to be unicode. Any unicode definitions are acceptable.

To set the locale to be unicode, try:

$ export LC_ALL=en_US.utf8 $ curator [ARGS]

Alternately, you should be able to specify the locale on the command-line:

$ LC_ALL=en_US.utf8 curator [ARGS]

Be sure to substitute your unicode variant for en_US.utf8

[untergeek]

@Kapitan777 You need to specify which version of Docker you're building with. Chances are it's a very old version of Docker, and you're seeing the exact same issues as the other comments above. But this report doesn't tell me anything other than you're seeing a very particular locale error that should not be happening (and indeed does not happen when I build with Docker:

$ docker version
…
Client:
 Cloud integration: v1.0.35
 Version:           24.0.2
 API version:       1.43
 Go version:        go1.20.4
 Git commit:        cb74dfc
 Built:             Thu May 25 21:51:16 2023
 OS/Arch:           darwin/arm64
 Context:           desktop-linux

[Kapitan777]

@untergeek Docker version doesn't matter in my question, because output's result the same as I use your image untergeek/curator:8.0.8 . May be the issue in the python code? I run image in AKS 1.27.7

docker version Client: Cloud integration: v1.0.25 Version: 20.10.16 API version: 1.41 Go version: go1.17.10 Git commit: aa7e414 Built: Thu May 12 09:17:07 2022 OS/Arch: windows/amd64 Context: default Experimental: true

Server: Docker Desktop 4.9.1 (81317) Engine: Version: 20.10.16 API version: 1.41 (minimum version 1.12) Go version: go1.17.10 Git commit: f756502 Built: Thu May 12 09:15:42 2022 OS/Arch: linux/amd64 Experimental: false containerd: Version: 1.6.4 GitCommit: 212e8b6fa2f44b9c21b2798135fc6fb7c53efc16 runc: Version: 1.1.1 GitCommit: v1.1.1-0-g52de29d docker-init: Version: 0.19.0 GitCommit: de40ad0

[untergeek]

@Kapitan777 If it does the same with my image, this is unrelated to this issue. Please open a new issue. My take is that this is a problem with AKS and not my image, though. If you're getting the locale warning, it means whatever is running Curator does not have a unicode locale set, which is highly unusual and atypical.

[untergeek]

@Jae1313 this should be addressed in Curator 8.0.9 which was just released. Please let me know if you still encounter problems.

[Jae1313]

Just upgrade to 8.0.9. Still some permission error when use 'elasticsearch' user to do the curator job.

su elasticsearch /bin/bash -c "source /config/env.config.sh; /usr/local/bin/curator --config /config/curator-config.yml /config/curator-actions.yml"

Traceback (most recent call last): File "/usr/local/bin/curator", line 8, in sys.exit(cli()) File "/usr/local/lib/python3.8/dist-packages/click/core.py", line 1157, in call return self.main(args, kwargs) File "/usr/local/lib/python3.8/dist-packages/click/core.py", line 1078, in main rv = self.invoke(ctx) File "/usr/local/lib/python3.8/dist-packages/click/core.py", line 1434, in invoke return ctx.invoke(self.callback, ctx.params) File "/usr/local/lib/python3.8/dist-packages/click/core.py", line 783, in invoke return __callback(args, *kwargs) File "/usr/local/lib/python3.8/dist-packages/click/decorators.py", line 33, in new_func return f(get_current_context(), args, **kwargs) File "/usr/local/lib/python3.8/dist-packages/curator/cli.py", line 249, in cli configure_logging(cfg, ctx.params) File "/usr/local/lib/python3.8/dist-packages/es_client/helpers/logging.py", line 175, in configure_logging set_logging(check_logging_config({'logging': logcfg})) File "/usr/local/lib/python3.8/dist-packages/es_client/helpers/logging.py", line 274, in set_logging loginfo = LogInfo(log_opts) File "/usr/local/lib/python3.8/dist-packages/es_client/helpers/logging.py", line 56, in init with open(fpath, 'wb+', buffering=0) as fhdl: PermissionError: [Errno 13] Permission denied: '/proc/1/fd/1'

++++++++++++++++++++++++ If use root, it is okay. Installed version:

pip freeze

certifi==2023.11.17 cffi==1.16.0 click==8.1.7 cryptography==41.0.7 ecs-logging==2.1.0 elastic-transport==8.11.0 elasticsearch-curator==8.0.9 elasticsearch8==8.12.0 es-client==8.12.3 pycparser==2.21 PyYAML==6.0.1 six==1.16.0 supervisor==4.1.0 urllib3==2.1.0 voluptuous==0.14.1

[untergeek]

I just built 8.0.10 with a try/except block one layer higher. It hopefully addresses it.

[Jae1313]

Thanks. Will try when 8.0.10 is ready.

[untergeek]

It’s already out

[Jae1313]

Just tried, it works! Thanks.

[untergeek]

Closing via es_client and #1700