metomi / fab

Flexible build system for scientific software
https://metomi.github.io/fab/
Other
19 stars 11 forks source link

Archives can contain multiple version of the same file #310

Open hiker opened 2 weeks ago

hiker commented 2 weeks ago

If you build a .a file, then modify one source file and rebuild, the .a will contain both versions of the compiled files, e.g.:

$ ar t ./most.a 
Constants.2de2473b7.o
...
InputOutput.3d229c877.o
InputOutput.37f15353a.o

When using this .a for linking, the oldest version of the .o is found and used in linking, meaning the changes to the source file are not used when creating the binaries. The problem goes away if no archive is created, and the binary is instead directly created from the .o files.

Potential solution:

  1. We add a flag to insert new members for existing members. I didn't try if this fixes the problem! And it might be slow??
  2. We query an existing .a to see if the same .o file is already in with a different hash, and remove it. Again, slow :(
  3. We copy all .o files with hash in the name to the corresponding hash-less name (e.g. InputOutput.37f15353a.o becomes InputOutput.o) and add them to the archive. That should fix the problem, but might be slow with large repositories (and is not nice for inode count :) ).
hiker commented 2 weeks ago

Using -b requires to specify archive member before which all new members are inserted, so that also requires a query of an existing archive. I also have some inconsistent results when trying, potentially caused because the member specified before which the new members should be added, is also part of the new members (e.g. in the example above, when using ar -crb Constants.2de2473b7.o most.a InputOutput.33333,o Constants.2de2473b7.o ... - the Constants...o file is specified as 'relpos', but also added. I could not consistently avoid the problem of the linker picking the wrong version of a .o

hiker commented 2 weeks ago

After some rest, I think the best option might be that the archive step just deletes an existing archive. I don't think this will be much (if at all) slower, since adding members to an existing archive might be expensive(??)