search

grailbio / base

A collection of Go utility packages used by GRAIL's tools
Apache License 2.0
86 stars 24 forks source link

fatbin: make the format more robust #9

Closed mariusae closed 4 years ago

mariusae commented 4 years ago

Fatbin currently relies on the sniffers to be able to determine the offset of the fatbin zip. This is done by inspecting the image format directly. This approach has not proven to be robust. While it seems to work on common configurations inside of GRAIL, we have reports (e.g., issue #8) of the approach failing in other configurations.

This change changes the format to include a magic value to identify fatbin binaries, and a footer that stores the fatbin offset directly. This should prove a much more robust approach as we do not rely on accurately reading the underlying image format with all of of its implementation-specific variations.

cosnicolaou commented 4 years ago

Note, that the problem I ran is go compiler specific - ie. tests with go1.12.9 but not 1.13. I'll comment on your PR momentarily, but wouldn't it better to use elf specifically where it's supported and write a custom elf section?

mariusae commented 4 years ago

Note, that the problem I ran is go compiler specific - ie. tests with go1.12.9 but not 1.13. I'll comment on your PR momentarily, but wouldn't it better to use elf specifically where it's supported and write a custom elf section?

I'd rather not have different implementations for each architecture; this approach strikes me as both simpler and more robust...