search

SCons / scons

SCons - a software construction tool
http://scons.org
MIT License
2.1k stars 315 forks source link

NodeInfoBase.update always causes csig's to be computed #2453

Open bdbaddog opened 6 years ago

bdbaddog commented 6 years ago

This issue was originally created at: 2009-07-15 15:39:26. This issue was reported by: puntium. puntium said at 2009-07-15 15:39:26

1.2.0 introduces a Decider() hook, which lets us use timestamp checking instead of content hashing to decide up-to-date-ness. While this does improve performance somewhat, it doesn't avoid md5 hash computation because of what happens after a node is built.

The NodeInfoBase.update() looks at the subclass-defined "field_list" (which for File nodes, hard codes 'csig' as an entry), which means that even if you used timestamps to decide that a file needs to be built, the file will be hashed no matter what, after being built, even if nothing ever needs the csig.

We need some way of controlling which fields get automatically re-computed after a node is built. Perhaps another hook along the lines of Decider(), though it gets tricky because you don't want to start involving environment objects here.

garyo said at 2009-08-25 16:57:49

Would the idea of doing this be to improve performance? If so, it would be great if you could try stubbing out the md5 hash call in question and see what improvement you get. That would give us all a sense of how valuable the change would be.

puntium said at 2009-08-25 17:02:14

We've already done this for our internal builds.. we monkey-patch in a custom "update" method that doesn't do csigs on nodes we know don't need them. I forget the exact improvement, but it was big enough for us to keep this ugly workaround.

Our build is currently bottlenecked on scons not being able to produce nodes fast enough to build on large multi-way machines. So taking this out of the critical path was a pretty good win.

gregnoel said at 2009-11-10 17:56:44

Bug party triage. Ludwig, here's another one where you and Ken should work together to see what you can figure out.

dirkbaechle said at 2014-05-17 14:13:37

reassign issue

mwichmann commented 2 years ago

Seems like this is the core topic of #2764 - scons computes md5 even with timestamp-newer

bdbaddog commented 2 years ago

Yup. Wondering if we provided a way to annotated such nodes to exclude the content hashing would that be one way to deal with this/ provide utility?