Closed Jae1313 closed 7 months ago
This is clearly a container you built using Python 3.8, which means you did not use the provided Dockerfile which builds with 3.11. I can’t say for sure what’s going on here as I do not know what version of Docker you built this with, but the line that includes logging.FileHandler('/proc/1/fd/1')
writes logs to the Docker socket on newer versions of Docker so that you can run docker logs CONTAINERNAME
to view the logs. This works perfectly with the Curator image published to Docker hub.
Please provide more information.
UPDATE: The code will always try to log to that path when Docker is detected. The following work around is invalid. A manual workaround is to force Curator to log to a file inside the Docker image (which you can, of course, map to a local file path). This will force Curator to not log to a path usable by docker logs
but to your specified path.
Which docker version have been tested to work fine? I guess maybe my docker version is a bit lower. What I am using is: $ docker --version Docker version 20.10.9, build c2ea9bc
It’s also the Dockerfile. You’re not using the current Dockerfile if you’re building with Python 3.8. Try building with the current Dockerfile before worrying about the Docker version, or just pull image untergeek/curator:8.0.8 and use that.
In our deployment, we do not deploy dockerized curator. What we do is deploy dockerized Elasticsearch, and install curator in Elasticsearch Dockerfile , something like: RUN ln -s /usr/bin/python3.8 /usr/bin/python && \ pip install cryptography elasticsearch-curator
In this case, curator will still detect it as is_docker. In this scenario, I dont think it needs to write logs to the Docker socket.
I can’t agree that I think this is a good idea, to be honest, precisely because of code changes like this, that turned out to be breaking changes to your setup.
I will see about putting a a write test and ignore that path if it doesn’t have permissions, but it will not be added for a two weeks at best, I would guess.
We could wait the fix then upgrade the curator to the latest one, currently our short term solution is downgrade to 8.0.4 which do not have is_docker detect. Thanks for your support.
It’s also the Dockerfile. You’re not using the current Dockerfile if you’re building with Python 3.8. Try building with the current Dockerfile before worrying about the Docker version, or just pull image untergeek/curator:8.0.8 and use that.
When I use this image or create image from https://github.com/elastic/curator/blob/master/Dockerfile always get next output:
{"@timestamp": "2024-01-29T12:37:02.786Z", "function": "run", "linenum": 226, "loglevel": "INFO", "message": "Preparing Action ID: 1, \"delete_indices\"", "name": "curator.cli"} {"@timestamp": "2024-01-29T12:37:02.786Z", "function": "run", "linenum": 233, "loglevel": "INFO", "message": "Creating client object and testing connection", "name": "curator.cli"} {"@timestamp": "2024-01-29T12:37:02.787Z", "function": "get_client", "linenum": 73, "loglevel": "INFO", "message": "Creating client object and testing connection", "name": "curator.helpers.getters"} argument should be a bytes-like object or ASCII string, not 'NoneType'
Wrapper for running curator from source.
When used with Python 3 Curator requires the locale to be unicode. Any unicode definitions are acceptable.
To set the locale to be unicode, try:
$ export LC_ALL=en_US.utf8 $ curator [ARGS]
Alternately, you should be able to specify the locale on the command-line:
$ LC_ALL=en_US.utf8 curator [ARGS]
Be sure to substitute your unicode variant for en_US.utf8
@Kapitan777 You need to specify which version of Docker you're building with. Chances are it's a very old version of Docker, and you're seeing the exact same issues as the other comments above. But this report doesn't tell me anything other than you're seeing a very particular locale error that should not be happening (and indeed does not happen when I build with Docker:
$ docker version
…
Client:
Cloud integration: v1.0.35
Version: 24.0.2
API version: 1.43
Go version: go1.20.4
Git commit: cb74dfc
Built: Thu May 25 21:51:16 2023
OS/Arch: darwin/arm64
Context: desktop-linux
@untergeek Docker version doesn't matter in my question, because output's result the same as I use your image untergeek/curator:8.0.8 . May be the issue in the python code? I run image in AKS 1.27.7
docker version Client: Cloud integration: v1.0.25 Version: 20.10.16 API version: 1.41 Go version: go1.17.10 Git commit: aa7e414 Built: Thu May 12 09:17:07 2022 OS/Arch: windows/amd64 Context: default Experimental: true
Server: Docker Desktop 4.9.1 (81317) Engine: Version: 20.10.16 API version: 1.41 (minimum version 1.12) Go version: go1.17.10 Git commit: f756502 Built: Thu May 12 09:15:42 2022 OS/Arch: linux/amd64 Experimental: false containerd: Version: 1.6.4 GitCommit: 212e8b6fa2f44b9c21b2798135fc6fb7c53efc16 runc: Version: 1.1.1 GitCommit: v1.1.1-0-g52de29d docker-init: Version: 0.19.0 GitCommit: de40ad0
@Kapitan777 If it does the same with my image, this is unrelated to this issue. Please open a new issue. My take is that this is a problem with AKS and not my image, though. If you're getting the locale warning, it means whatever is running Curator does not have a unicode locale set, which is highly unusual and atypical.
@Jae1313 this should be addressed in Curator 8.0.9 which was just released. Please let me know if you still encounter problems.
Just upgrade to 8.0.9. Still some permission error when use 'elasticsearch' user to do the curator job.
Traceback (most recent call last):
File "/usr/local/bin/curator", line 8, in
++++++++++++++++++++++++ If use root, it is okay. Installed version:
certifi==2023.11.17 cffi==1.16.0 click==8.1.7 cryptography==41.0.7 ecs-logging==2.1.0 elastic-transport==8.11.0 elasticsearch-curator==8.0.9 elasticsearch8==8.12.0 es-client==8.12.3 pycparser==2.21 PyYAML==6.0.1 six==1.16.0 supervisor==4.1.0 urllib3==2.1.0 voluptuous==0.14.1
I just built 8.0.10 with a try/except block one layer higher. It hopefully addresses it.
Thanks. Will try when 8.0.10 is ready.
It’s already out
Just tried, it works! Thanks.
For usage questions and help
Please create a topic at https://discuss.elastic.co/c/elasticsearch
Perhaps a topic there already has an answer for you!
To submit a bug or report an issue
Expected Behavior
Actual Behavior
Steps to Reproduce the Problem
logging: loglevel: INFO logfile: /logs/es-curator-actions.log logformat: default blacklist: ['elastic_transport', 'urllib3']
Command result as following error: Traceback (most recent call last): File "/usr/local/bin/curator", line 8, in
sys.exit(cli())
File "/usr/local/lib/python3.8/dist-packages/click/core.py", line 1157, in call
return self.main(args, kwargs)
File "/usr/local/lib/python3.8/dist-packages/click/core.py", line 1078, in main
rv = self.invoke(ctx)
File "/usr/local/lib/python3.8/dist-packages/click/core.py", line 1434, in invoke
return ctx.invoke(self.callback, ctx.params)
File "/usr/local/lib/python3.8/dist-packages/click/core.py", line 783, in invoke
return __callback(args, *kwargs)
File "/usr/local/lib/python3.8/dist-packages/click/decorators.py", line 34, in new_func
return f(get_current_context(), args, **kwargs)
File "/usr/local/lib/python3.8/dist-packages/curator/cli.py", line 316, in cli
set_logging(check_logging_config(
File "/usr/local/lib/python3.8/dist-packages/curator/config_utils.py", line 51, in set_logging
loginfo = LogInfo(log_opts)
File "/usr/local/lib/python3.8/dist-packages/curator/logtools.py", line 136, in init
self.handler = logging.FileHandler('/proc/1/fd/1')
File "/usr/lib/python3.8/logging/init.py", line 1147, in init
StreamHandler.init(self, self._open())
File "/usr/lib/python3.8/logging/init.py", line 1176, in _open
return open(self.baseFilename, self.mode, encoding=self.encoding)
PermissionError: [Errno 13] Permission denied: '/proc/1/fd/1'
Specifications
Context (Environment)
Detailed Description