This is largely a forward-port of logstash-plugins/logstash-output-s3#252 with some minor changes to deal with the integrated plugin's usage of the java-native ConcurrentHashMap in place of the stand-alone plugin's ruby-native Concurrent::Map.
Refactor of S3::FileRepository to avoid several closely-related race conditions:
prevent get_factory() from yielding a factory that was mid-deletion by the stale watcher, which could cause the plugin crash due to the file no longer existing on disk. This is solved by marking a factory's prefix wrapper as deleted while the stale watcher has exclusive access to it, and checking for deletion status before yielding exclusive access to a prefix wrapper's factory.
introduce each_factory, which avoids creating new factories or yielding deleted ones.
refactor each_files to use new each_factory to avoid yielding files whose factories have been deleted.
void-return methods now explicitly emit nil to prevent accidental leaks of synchronization-required resources.
Additionally, S3#rotate_if_needed was migrated to use the now-safer S3::FileRepository#each_factory that avoids initializing new factories (and therefore avoids creating empty files on disk after the existing ones had been stale-reaped).
This is largely a forward-port of logstash-plugins/logstash-output-s3#252 with some minor changes to deal with the integrated plugin's usage of the java-native
ConcurrentHashMapin place of the stand-alone plugin's ruby-nativeConcurrent::Map.Refactor of
S3::FileRepositoryto avoid several closely-related race conditions:get_factory()from yielding a factory that was mid-deletion by the stale watcher, which could cause the plugin crash due to the file no longer existing on disk. This is solved by marking a factory's prefix wrapper as deleted while the stale watcher has exclusive access to it, and checking for deletion status before yielding exclusive access to a prefix wrapper's factory.each_factory, which avoids creating new factories or yielding deleted ones.each_filesto use neweach_factoryto avoid yielding files whose factories have been deleted.nilto prevent accidental leaks of synchronization-required resources.Additionally,
S3#rotate_if_neededwas migrated to use the now-saferS3::FileRepository#each_factorythat avoids initializing new factories (and therefore avoids creating empty files on disk after the existing ones had been stale-reaped).