Closed mashhurs closed 2 months ago
Hi @mashhurs need some information on this before digging to resolve it.
WDYT?
Hi @mashhurs need some information on this before digging to resolve it.
- How this is different from S3 Output should clean up empty temp folders #34 resolved by Remove empty temp dirs at plugin startup. #39?
The root cause is same. With #34 we introduced retry-like temp folder deletion on LS startup which swipes out empty dirs but those empty dirs are caused by this #28 in a reality.
- Given the
FileUtils.rm_r
with secure option throwsErrno::ENOTEMPTY
ruby/fileutils#111 seems to be fixed by commit ruby/fileutils@ec5d3b8 did you try with updated version? I'm not such confident it resolves the problem, because from this line doesn't seem it test under WinOS
As I remember, this access fix was also included when I was investigating that traversing temp_file
children process was always succeeded. In case if I am mistaken, we need to re-examine. This issue happens especially with multiple uploader threads and small file rotation window.
Thanks a lot for these information, could be useful to try a reproducer.
Logstash information:
Please include the following information:
bin/logstash --version
): 8.6.2integration-aws-v7.0.1
JVM (e.g.
java -version
):If the affected version of Logstash is 7.9 (or earlier), or if it is NOT using the bundled JDK or using the 'no-jdk' version in 7.10 (or higher), please provide the following information:
java -version
)JAVA_HOME
environment variable if set.OS version (
uname -a
if on a Unix-like system): any OS butFileUtils.rm_r
error happens on WindowsDescription of the problem including expected versus actual behavior: 1)
FileUtils.rm_r
fails2) File factory issue with Prefix We introduced
is_deleted?
method for the file factory and it becomes true when file is physically deleted. If any error happens such likeFileUtils.rm_r
, factory lives forever and temporary dirs staySteps to reproduce:
Please include a minimal but complete recreation of the problem, including (e.g.) pipeline definition(s), settings, locale, etc. The easier you make for us to reproduce it, the more likely that somebody will take the time to look at it.
Opened an issue in fileutils after long investigation: https://github.com/ruby/fileutils/issues/111
Provide logs (if relevant):