search

google / ml-compiler-opt

Infrastructure for Machine Learning Guided Optimization (MLGO) in LLVM.
Apache License 2.0
629 stars 93 forks source link

Remove corpus extraction tooling #334

Closed boomanaiden154 closed 9 months ago

boomanaiden154 commented 9 months ago

Now that the corpus extraction tooling has been moved into the LLVM monorepo, we can remove it from within ml-compiler-opt. This patch does the mechanical removal and updates the demos to refer to installing the upstream tooling so they are still functional.

boomanaiden154 commented 9 months ago

I'm currently just putting manual installation steps in the demo. I'm planning on adding mlgo-utils to the MLGO dependencies once we get around to regenerating the lockfile (been putting this off for quite a while now), but I want to do that all at once for several reasons:

  1. Now that the LLVM monorepo is the canonical source for the corpus extraction tooling, it wouldn't be good to refer to the old tooling here.
  2. There are other downstream consumers that are impacted by the changing of the lockfile that I'd rather not have blocking this change.
  3. There are some other issues that need to be cleaned up that come along with some packages being bumped that I haven't gotten around to figuring out yet.
boomanaiden154 commented 9 months ago

lgtm, ready to be merged once we start producing the pip package

I already pushed the pip package once to reserve the name (https://pypi.org/project/mlgo-utils/), and the name won't be changing, so this should be good to go.