search

facebookresearch / CompilerGym

Reinforcement learning environments for compiler and program optimization tasks
https://compilergym.ai/
MIT License
916 stars 129 forks source link

Create Example Repo that Extends Compiler Gym #532

Open mostafaelhoushi opened 2 years ago

mostafaelhoushi commented 2 years ago

🚀 Feature

Create a standalone repo that import compiler_gym and extends it by defining a new environment

Motivation

We want CompilerGym to be a tool for everyone to create their own environments. Currently we have examples within the repo that may require modifying the build of CompilerGym. Ideally we want to show how users can create their own environments "out-of-tree".

Pitch

We can create an example in a standalone repo (perhaps under someone's personal GitHub account) and have a section in CompilerGym documentation or README with out-of-tree examples (that define separate environments but perhaps also that define their own benchmarks, extending existing environments with their own observations/rewards/actions, RL training scripts, auto-tuning scripts, etc.)

Alternatives

We may consider the "CompilerGym-examples" repo that was created for the paper.

We can also consider something like Pytorch's examples repo: https://github.com/pytorch/examples/

Additional context

N/A

ChrisCummins commented 2 years ago

I ❤️ this idea. I would suggest keeping the scope of the standalone repo fairly constrained to just demoing a new environment and calling it CompilerGymEnvironmentExample or something.

If we were to move everything from examples/ into a separate repo (which would almost be possible after #428 except for a couple of bits that need to be built with cmake/bazel) then I imagine that the CI config gets a lot more confusing when testing across two repos and that breaking changes / refactors can be a bit of a headache when two repos need to move in lockstep. What do you think?

I agree that a personal GitHub account is probably the way to go here. We can link to it from here.

Cheers, Chris

mostafaelhoushi commented 2 years ago

I ❤️ this idea. I would suggest keeping the scope of the standalone repo fairly constrained to just demoing a new environment and calling it CompilerGymEnvironmentExample or something.

If we were to move everything from examples/ into a separate repo (which would almost be possible after #428 except for a couple of bits that need to be built with cmake/bazel) then I imagine that the CI config gets a lot more confusing when testing across two repos and that breaking changes / refactors can be a bit of a headache when two repos need to move in lockstep. What do you think?

I agree that a personal GitHub account is probably the way to go here. We can link to it from here.

Cheers, Chris

Yes, I agree we shouldn't move the examples. I was thinking of keeping the examples directory as is and perhaps add to the README.md file in the examples a directory a link to a single external repo for now, and hopefully as time goes on more members of the community will create their own extensions that we may add their links as well.

ChrisCummins commented 2 years ago

Sounds good! Do you want to take the lead on this?

Cheers, Chris

mostafaelhoushi commented 2 years ago

Sure. I just assigned it to myself.

ChrisCummins commented 2 years ago

Brill, thanks!

mostafaelhoushi commented 2 years ago

I imagine that the CI config gets a lot more confusing when testing across two repos and that breaking changes / refactors can be a bit of a headache when two repos need to move in lockstep. What do you think?

I suggest to create an examples repo like PyTorch https://github.com/pytorch/examples, and it could have its own CI that will test using pip install compiler_gym rather than having to build the development branch of CompilerGym.

Basically, CompilerGym repo CI shouldn't run the examples repo's CI. We only run the examples repo CI when we want to create a new release.

ChrisCummins commented 2 years ago

I like the sound of that!

We only run the examples repo CI when we want to create a new release.

It's not super clear how to do that. One way (I think) of achieving this would be to add a manual trigger to the examples CI workflow, where you can enter in flags to append to the pip install compiler_gym setup step. That way we can upload a pre-release wheel of CompilerGym to pypi's testing repository, and run the examples CI against that, before releasing proper.

Cheers, Chris