Some jobs in twitter sometimes fail with FileNotFoundException due to our replication process - during replication temporary file is created in destination folder (and later renamed). If getAllVersions method executes listDir while temporary file exists and later executes getFileSystem().getFileStatus(p) while this temporary file renamed exception is thrown and job fails.
Solution is to move getFileStatus closer to block where it's used and where it's guarded by try-catch.
Some jobs in twitter sometimes fail with
FileNotFoundException
due to our replication process - during replication temporary file is created in destination folder (and later renamed). IfgetAllVersions
method executeslistDir
while temporary file exists and later executesgetFileSystem().getFileStatus(p)
while this temporary file renamed exception is thrown and job fails.Solution is to move
getFileStatus
closer to block where it's used and where it's guarded bytry-catch
.