replace tf.AUTO_REUSE with explicit reuse patterns

a tf-using class (Actor, Critic, NNDynamicsMode) usually follows this pattern:

class SomeTFClass:
    def __init__(self):
        input = tf.placeholder()
        output = build_mlp('some-scope', ph, reuse=tf.AUTO_REUSE)

    def tf_output(different_input):
        return build_mlp('some-scope', different_input, reuse=tf.AUTO_REUSE)

This uses the same variables in both MLPs because the first time the function is called, at creation, the tf.get_variable / reuse semantics will dictate that since the variable does not exist it should be created. After the class is initialized, a different user wishing to evaluate the MLP within the class in question at its current variable values but for a different input tensor would query via tf_output(their_input).

This is a lazy use of tf.AUTO_REUSE an should be replaced with the explicit reuse semantics that we are expecting:

class SomeTFClass:
    def __init__(self):
        input = tf.placeholder()
        output = build_mlp('some-scope', ph, reuse=None)

    def tf_output(different_input):
        return build_mlp('some-scope', different_input, reuse=True)

vlad17 / mve

replace tf.AUTO_REUSE with explicit reuse patterns #321