apache / incubator-heron

Apache Heron (Incubating) is a realtime, distributed, fault-tolerant stream processing engine from Twitter
https://heron.apache.org/
Apache License 2.0
3.65k stars 598 forks source link

Add SLF4JBridgeHandler #3809

Closed thinker0 closed 2 years ago

thinker0 commented 2 years ago

JavaUtilLogging to SLF4JBridgeHandler

thinker0 commented 2 years ago

https://app.travis-ci.com/github/apache/incubator-heron/builds/248849844 How do I solve this ?

grep: write error: No space left on device
nicknezis commented 2 years ago

https://app.travis-ci.com/github/apache/incubator-heron/builds/248849844 How do I solve this ?

I've restarted the build.

surahman commented 2 years ago

I have restarted the build to see if the build errors are transient. I do not see a reason why updates/changes to logging should cause test failures.

surahman commented 2 years ago

Something is causing the creation of large files in this PR. Since this PR is making changes to the logger I am wondering if the log files are getting too large. The actual production code seems to be fine but there may need to be changes in the log output or log file size.

OSError: [Errno 28] No space left on device

I do not believe this is a build artifact size related issue because I think that would have triggered earlier after build completion.

surahman commented 2 years ago

I have tried to rerun the pipeline three times to see if there was a load issue causing this with Travis CI but they fail in roughly the same place.

There is an issue on SO that suggests this occurs with Python3 and could be due to either a lack of disk space in /tmp or due too many files. Are the log file structures different from before? Are more or larger logs being written?

thinker0 commented 2 years ago

Nice ^____^;

surahman commented 2 years ago

Just one final concern/question: Can the log file size/count affect production systems, and if so, how might we mitigate the issue now?

thinker0 commented 2 years ago

@surahman

jul-to-slf4j : Default FINE
slf4j-to-log4j: Default INFO ?
logback: Default DEBUG

As expected, it seems that it is normal to have less than before. The definition in logback.xml is INFO Level, so some contents should be reduced.

logback.xml: INFO
thinker0 commented 2 years ago
heron % ./scripts/travis/ci.sh 2>&1 | tee ci.log
===========================================================
heron build integration_test    0:00:31
heron install   0:00:22
heron tests install 0:00:06
heron integration_test local    0:04:32
heron integration_test http-server initialization   0:00:00
heron integration_test scala    0:01:12
heron integration_test java 0:12:08
heron integration_test python   0:03:22
heron integration_topology_test java    0:02:49
===> Finished ./scripts/travis/test.sh at 2022-04-06 11:27:40 (0:25:02)
===> Task duration summary for ./scripts/travis/ci.sh
===========================================================
./scripts/travis/build.sh   0:26:35
./scripts/travis/test.sh    0:25:02

ls -la -rt
-rw-r--r--    1 thinker0  staff   2.5M Apr  6 10:56 heron_build.txt
-rw-r--r--    1 thinker0  staff   4.3K Apr  6 10:58 heron_test_non_flaky.txt
-rw-r--r--    1 thinker0  staff   1.7K Apr  6 10:59 heron_test_flaky.txt
-rw-r--r--    1 thinker0  staff   2.0K Apr  6 11:01 heron_build_tarpkgs.txt
-rw-r--r--    1 thinker0  staff   1.1K Apr  6 11:02 heron_build_binpkgs.txt
-rw-r--r--    1 thinker0  staff   3.2K Apr  6 11:03 heron_build_integration_test.txt
-rw-r--r--    1 thinker0  staff   1.3K Apr  6 11:03 heron_install.txt
-rw-r--r--    1 thinker0  staff   1.4K Apr  6 11:03 heron_tests_install.txt
drwxrwx---   67 thinker0  staff   2.1K Apr  6 11:07 .
drwxr-xr-x  137 thinker0  staff   4.3K Apr  6 11:24 results
-rw-r--r--    1 thinker0  staff   429K Apr  6 11:27 ci.log

When I test, the log file is not large.

surahman commented 2 years ago

@nicknezis requested more reviews so I have not merged the PR. I think everything is in order.

Edit: The reason I was looking at the log file size or count was based on the context of the PR and the Python3 error.

nicknezis commented 2 years ago

Adding for context, this was a further refinement on #3807 . Thanks @thinker0 !