Fix pretraining more than once Issue #538

imontesino commented 3 years ago

Description

Changed the reuse parameter when creating the tf variables in the pretrain method to tf.AUTO_REUSE. This prevents tf from trying to create the variables again when pretraining more than once and raising the error:

ValueError: Variable pretrain/model/shared_fc0/w/Adam/ already exists, disallowed. Did you mean to set reuse=True or reuse=tf.AUTO_REUSE in VarScope?

Motivation and Context

Closes #538

[x] I have raised an issue to propose this change (required for new features and bug fixes)

Types of changes

[x] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to change)
[ ] Documentation (update in the documentation)

Checklist:

[x] I've read the CONTRIBUTION guide (required)
[x] I have updated the changelog accordingly (required).
[ ] My change requires a change to the documentation.
[ ] I have updated the tests accordingly (required for a bug fix or a new feature).
[ ] I have updated the documentation accordingly.
[x] I have ensured pytest and pytype both pass (by running make pytest and make type).

araffin commented 3 years ago

the build is failing due to pytype, you can probably add an exception for it (and check locally that the type check passes)

imontesino commented 3 years ago

The pytype error seems to be in record_expert.py. The code in this PR passes locally as well as in CI (see the line 55 of the typecheck CI)

[39/107] check stable_baselines.common.base_class

araffin commented 3 years ago

The pytype error seems to be in record_expert.py. The code in this PR passes locally as well as in CI (see the line 55 of the typecheck CI)

yes, I know, the error is due to a newer pytype version I assume, so what I was suggesting is ignoring explicitly that error (cf pytype doc, you need to add # pytype: disable=...

araffin commented 3 years ago

The build is failing because of atari-py :see_no_evil: (see https://github.com/openai/atari-py/pull/79) Could you pin the atari-py version as in https://github.com/DLR-RM/stable-baselines3/pull/448 ? (atari-py==0.2.5)

Sorry for the CI issues...

hill-a / stable-baselines