Broken Tutorials - DQN

[caseyjconger]

I have been trying to work through some of the tutorials provided but I'm finding it a little difficult.

1. I tried cloning and running locally, but the tutorials use xvfb which isn't supported by macOS without workarounds. Ok, so i'll just run it in colab...

2. Get error that xdpyinfo is not installed. But let's keep going... [I do see that this was addressed in another issue, but the warning message is still there. If nothing else, maybe update comments in the notebook to warn people not to worry and waste time trying to fix it.]

3. AttributeError: module 'tensorflow_probability.python.util' has no attribute 'SeedStream' In call to configurable 'DqnAgent' (<function DqnAgent.init at 0x7f2b9ae8cd90>)

For the tutorials, I'd suggest that there be more emphasis on making these stable, portable and easily runnable.

I was going through the tutorials to test out if I wanted to use tf-agents or some other framework, and running into these issues while going through the nice, pre-packaged introductory tutorials, it makes me think that it'll only get worse once I dive into something complex and/or custom, and so I'd be better served using a different framework.

Nothing kills motivation to learn something new than having to spend time tracking down unrelated errors and broken dependencies. So, probably better to keep the tutorials basic and stable than trying to use the most recent versions and features.

[kbanoop]

Thanks for raising this! I just tried it on googlecolab, and didn't run into (3). Were you running it locally?

I think the issue is related to a new version of TF Probability, and the "!pip install --upgrade tensorflow-probability" in the colabs was a temporary solution. We will release a new version of TF-Agents with the correct TFP version to fix it properly.

[tfboyd]

Another suggestion is to start with the stable release and branch, which were testing with specific versions of the supporting libraries. I fully understand it is frustrating; we likely need to push that option harder and respond to issues with that suggestion.

https://github.com/tensorflow/agents#stable

Not an excuse. There are many moving parts upstream from tf-agents and the best way we can stabilize them for you is to push our stable releases as the best option vs. starting at head and with nightly builds. I welcome feedback regarding taking that approach. Sorry your experience was not great.

[caseyjconger]

@kbanoop I was running it on google colab, but it does seem to be working now, so after some thorough debugging, I've narrowed it down to either:

A) most boring hallucination ever B) I had tried to install some extra dependency on my colab instance, broke stuff and forgot I did it C) magic

I'm thinking C).

@tfboyd I've been an open-source lurker for a while now; if I find things I think could be better I suppose I could just try to fix it myself :)

Maybe there could be some scheduled tests on the tutorials in colab? I can't really say how much work that would be, but just so that you know that someone can always run the tutorials.

Only other suggestion would be to build a basic docker image that can run the tutorials locally in case of spotty internet connection.

@kbanoop @tfboyd Thanks both for your responses. I'll close the issue now since it seems like it was mostly on my end.

[tfboyd]

@caseyjconger I actually have a docker, I used to test the colabs and they are being tested nightly. The rub is the depenencies they are tested under may not matchup. What you did was totally valid and ended up doing it at a time where the upstream libraries moved. I will try to publish the docker build script. We had a pretty long talk about this as our team sync today. We 100% heard your frustration.