facebookarchive / BOLT

Binary Optimization and Layout Tool - A linux command-line utility used for optimizing performance of binaries
2.51k stars 176 forks source link

BOLT Upcoming Refactoring #227

Closed maksfb closed 2 years ago

maksfb commented 2 years ago

I am opening this issue to give a heads-up for all BOLT developers of the upcoming refactoring work we will be pushing in the next couple of weeks. We will break BOLT into multiple components (CORE, IR, Debug, etc.) that will affect the directory structure and will allow using shared-library build mode. The concrete names and the directory structure are not finalized yet. All suggestions are welcome.

In parallel, we will be modifying parts of the IR itself. One thing that comes to mind and that I have wanted to change for a long time is the way BinaryFunctions are stored. The containing map assumes that every function has an address. That was true at one point, but then we started injecting new functions in a binary and adding more files. For PIC code (.so's), e.g. two files can have the same input address creating a conflict in the binary context. We keep finding more reasons to decouple the input address information from the way functions are stored, and the time has come to refactor BinaryContext::BinaryFunctions.

I know there are a big set of patches in the pipeline that are not part of the work at FB, e.g. the support for golang. So please cope with us, as I can imagine you will have to modify your code for the refactoring. If you were planning to create pull requests soon, perhaps you can prioritize them, and we can hold the refactoring for some time.

yota9 commented 2 years ago

Hello @maksfb ! Thank you for your post! I'm currently preparing to open source golang support, I believe it will be soon, so I hope we will be able to do it before global bolt refactoring :) I will give more details about it in about a week or so here, I will need to prepare a few more places, that time there are already few patches prepared and pull requests are opened, I hope we will be able to merge them soon. Thank you!

rafaelauler commented 2 years ago

https://github.com/facebookincubator/BOLT/pull/240 has the details of the refactoring that is being planned.

We welcome any suggestions or input on the future layout of the repo.

yota9 commented 2 years ago

Hello @maksfb @rafaelauler ! I'm glad to tell you that golang support is almost ready to be open sourced! But we've got a blocker - the current PR must be merged, so I will rebase golang support upon them. So I hope we could deal with them in the nearest time. Thank you!

P.S.

For PIC code (.so's), e.g. two files can have the same input address creating a conflict in the binary context.

Does it mean that you want to optimize multiple binaries together? E.g. interbinary inlining & etc

rafaelauler commented 2 years ago

Heads up! This refactoring has landed. It shouldn't disrupt other branches. Please use -c merge.renamelimit=230 git option when rebasing your work on top of this change. For example:

git -c merge.renamelimit=230 rebase

or

git -c merge.renamelimit=230 cherry-pick

If you do not set the rename limit to be large enough, git will fail with lots of unmerged/conflicts. Don't try to manually resolve those. Just cancel the rebase and try again with renamelimit correctly set.

aaupov commented 2 years ago

@maksfb ok to close this announcement?