madecoste / swarming

Automatically exported from code.google.com/p/swarming
Apache License 2.0
0 stars 1 forks source link

Use global cache instead of per .isolate cache; using {(inode, timestamp):hash} #174

Open GoogleCodeExporter opened 9 years ago

GoogleCodeExporter commented 9 years ago
Current state:
- Each .isolated has a .isolated.state which is a cache of files based on 
relative path name.
- This means processing multiple .isolated doesn't share cache on objects and 
they don't refer to the actual inodes.
- This was done to get away from the parallel execution of isolate.py archive 
during compile, which is no longer used by chromium.

Goal: performance
- Use global cache for significant better performance; as multiple .isolated in 
a single project will likely reuse a least a subset of the same files. Use 
inode when os.stat().st_ino is a valid value. Postpone more complex processing 
for Windows since it requires using FindFirstFileEx()/FindNextFile() so it's 
going to be a completely different code path.

Original issue reported on code.google.com by maruel@chromium.org on 14 Nov 2014 at 8:25