google-deepmind / bsuite

bsuite is a collection of carefully-designed experiments that investigate core capabilities of a reinforcement learning (RL) agent
Apache License 2.0
1.51k stars 181 forks source link

threadlimit #29

Closed DavidJanz closed 1 year ago

DavidJanz commented 4 years ago

Current implementation thrashes the cpu ---> extremely slow running the sweep.

Suggested fix here, but untested --- I get other errors trying to run the default agents: 1) transformed.apply expecting a random key, recent change in JAX, I suspect you need an older version of JAX / need to update this code 2) setup.py doesn't actually work for me out the box: pip install bsuite[baselines] ERROR: Could not find a version that satisfies the requirement tensorflow==2.1; extra == "baselines" (from bsuite[baselines]) (from versions: 2.2.0rc1, 2.2.0rc2, 2.2.0rc3, 2.2.0rc4, 2.2.0, 2.3.0rc0, 2.3.0rc1, 2.3.0rc2, 2.3.0) ERROR: No matching distribution found for tensorflow==2.1; extra == "baselines" (from bsuite[baselines])

I use this in my code interacting with bsuite envs, it tends to speed it up by 10-100x. I suspect something like this might help on internal run scripts too.

googlebot commented 4 years ago

Thanks for your pull request. It looks like this may be your first contribution to a Google open source project (if not, look below for help). Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

:memo: Please visit https://cla.developers.google.com/ to sign.

Once you've signed (or fixed any issues), please reply here with @googlebot I signed it! and we'll verify it.


What to do if you already signed the CLA

Individual signers
Corporate signers

ℹ️ Googlers: Go here for more info.

DavidJanz commented 4 years ago

@googlebot I signed it!

googlebot commented 4 years ago

CLAs look good, thanks!

ℹ️ Googlers: Go here for more info.

iosband commented 4 years ago

Hi David - looks like there is something wrong with this pull request... I'm having a look if this improves performance

DavidJanz commented 4 years ago

Fixed a typo that was making actual tests fail. It should now be runnable. PyTest is failing, but I don't think that's anything to do with me? I made another pull request here which was just changing a doc string, and that also has PyTests failing.

aslanides commented 4 years ago

Hey David! There were some breakages upstream of us (rlax/chex) which should be fixed soon. I nthe mean time I have made a fix on our end, so this should be OK.

iosband commented 4 years ago

I had a look at this change locally and within Google and I don't think I saw a benefit... bit confused what's going on here

DavidJanz commented 4 years ago

I'll put together an example script that times this stuff using the external interface over the next couple of days and send it over.