Prior to this, cached stages will be run once to populate the cache (triggered by write) and once where they're used subsequently.
The alternative to this is to add a cache call to the write chain to avoid ORC read overhead, but we've seen problems with cache on larger stages where the job eventually fails.
This also fixes the local FS cache implementation.
Prior to this, cached stages will be run once to populate the cache (triggered by
write
) and once where they're used subsequently.The alternative to this is to add a
cache
call to the write chain to avoid ORC read overhead, but we've seen problems withcache
on larger stages where the job eventually fails.This also fixes the local FS cache implementation.