Closed tchiotludo closed 3 weeks ago
on EE there is also an issue that the UI doesn't show all logs (they are cut at a certain limit), see https://preview-ee-kafka.kestra.io/ui/anna/executions/test/python/4z80tyGbgiylub4ngU5Nah/logs?level=TRACE&page=1
repro:
id: python
namespace: test
tasks:
- id: python
type: io.kestra.plugin.scripts.python.Script
taskRunner:
type: io.kestra.plugin.core.runner.Process
beforeCommands:
- pip install requests
script: |
import requests
response = requests.get('https://api.github.com')
print(response.status_code)
when you download the log you will see there are more logs captured by the backend than the UI is showing:
2024-07-01T21:05:33.488Z INFO Provided 0 input(s).
2024-07-01T21:05:33.559Z INFO Provided 1 input(s).
2024-07-01T21:05:33.683Z DEBUG Starting command with pid 9097 [/bin/sh -c set -e
pip install requests
python /tmp/2cpot3DpAi5KjB3ZjztdHs/9987286607004781761.py]
2024-07-01T21:05:37.148Z INFO Defaulting to user installation because normal site-packages is not writeable
2024-07-01T21:05:37.512Z INFO Collecting requests
2024-07-01T21:05:37.592Z INFO Downloading requests-2.32.3-py3-none-any.whl (64 kB)
2024-07-01T21:05:37.615Z INFO ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 64.9/64.9 KB 2.9 MB/s eta 0:00:00
2024-07-01T21:05:37.707Z INFO Collecting certifi>=2017.4.17
2024-07-01T21:05:37.723Z INFO Downloading certifi-2024.6.2-py3-none-any.whl (164 kB)
2024-07-01T21:05:37.754Z INFO ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 164.4/164.4 KB 12.2 MB/s eta 0:00:00
2024-07-01T21:05:37.805Z INFO Collecting idna<4,>=2.5
2024-07-01T21:05:37.817Z INFO Downloading idna-3.7-py3-none-any.whl (66 kB)
2024-07-01T21:05:37.835Z INFO ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 66.8/66.8 KB 6.9 MB/s eta 0:00:00
2024-07-01T21:05:38.073Z INFO Collecting charset-normalizer<4,>=2
2024-07-01T21:05:38.087Z INFO Downloading charset_normalizer-3.3.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (142 kB)
2024-07-01T21:05:38.103Z INFO ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 142.1/142.1 KB 17.9 MB/s eta 0:00:00
2024-07-01T21:05:38.175Z INFO Collecting urllib3<3,>=1.21.1
2024-07-01T21:05:38.184Z INFO Downloading urllib3-2.2.2-py3-none-any.whl (121 kB)
2024-07-01T21:05:38.200Z INFO ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 121.4/121.4 KB 11.1 MB/s eta 0:00:00
2024-07-01T21:05:38.246Z INFO Installing collected packages: urllib3, idna, charset-normalizer, certifi, requests
2024-07-01T21:05:38.479Z WARN WARNING: The script normalizer is installed in '/home/kestra/.local/bin' which is not on PATH.
2024-07-01T21:05:38.481Z WARN Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
2024-07-01T21:05:38.554Z INFO Successfully installed certifi-2024.6.2 charset-normalizer-3.3.2 idna-3.7 requests-2.32.3 urllib3-2.2.2
2024-07-01T21:05:39.054Z INFO 200
2024-07-01T21:05:39.094Z DEBUG Command succeed with code 0
This seems to be fixed now with https://github.com/kestra-io/kestra/pull/4468, but I might be testing the wrong thing, can you double check @anna-geller?
on develop image currently:
id: myflow
namespace: company.team
inputs:
- id: nr
type: INT
defaults: 50
tasks:
- id: log
type: io.kestra.plugin.scripts.python.Script
script: |
import time
logs = int({{ inputs.nr }}) + 1
for i in range(1, logs):
print(f"Generating log messages: {i}")
time.sleep(0.01)
they don't appear in the right order and not all are captured
maybe we should try on a local build
I've been testing and re-testing locally with the example flow from your last comment @anna-geller, but can't seem to reproduce the problem.
It might be that develop image wasn't built properly when you were testing it, could you try one more time, as I've ran also the latest image: kestra/kestra:develop-full
and it seems to be working as expected.
yup I can confirm it works
Seems that the #3022 is back again!
When an execution is started, and output lost of log in realtime, when you click on the gantt charts on a running execution, you will see only the logs in realtime and not the previous one.
Clicking after few seconds will display : 5 , 6, 7, and not the 1, 2, 3, 4