Things PTAN and the book can improve on.

[giongto35]

I really like your book. The knowledge with implementation helps me understand many concepts faster. However, I cannot make progress from Chapter 09 since the introduction of PTAN. PTAN is a great library and I intended to apply it to my game environment (which is not openAI Gym). However, I had hard time doing that for the following reasons. One problem is that you are trying to generalize the library to fit many agents and use-cases while targeting for only Open AI gym environment, which leads to un-reusability for mainstream development. I believe this hindered the popularity of this library also. For example, the experience source implementation is overcomplicated with many options, many if else conditions and untidy __iter function. It greatly reduces the readability and readers of the book because as a reader, we need an easy to read and reuse code for other practical experiments, not a fully-packed library. I know PTAN helps abstract repeated implementation. However, the abstraction here hurt the understanding of readers when you try to put much stuff inside the library.

[Shmuma]

Hi!

Yes, you're right. If you check the history of experience.py, you may notice that most of complications were introduced with vectorized envs (needed by Universe envs). Before that, it was way simpler and easier to understand (if you're curious, it was introduced here: https://github.com/Shmuma/ptan/commit/aaa563c61fbde3bd76bd8c7e390d07e963ff74dc#diff-c322277f55a770387f614aaad769104d). I still not sure that my implementation was the correct way of doing this, probably it could be better to implement it in a separate class.

In terms of supporting non-gym environments, that's an interesting idea. In my experience it was usually simpler to wrap some non-gym env into gym class or follow gym api from the beginning. From my perspective, gym api is simple but general enough to cover most of RL scenarios. But if you have examples of other use cases, which doesn't fit into gym Env api, we can think about generalisation, sure.

In fact, ptan is missing lots of things, like docs, examples (most of them are outdated) and could be improved in many other ways. I have plans to start dedicating more time to it, but can't say when it will happen :).

[giongto35]

I agree with you. The code without vectorized is very clear and concise. I think separating the class will be better. My current way is wrapping my environment to open AI gym too. I think OpenAI is standard, so sticking with it is ideal. Maybe, I felt it was too tightly coupled with openAI Gym because the library is modified to support "vectorized", which is not necessary generally. Btw, thanks for the fast response and your support. I think with a little bit improvement, PTAN can help a lot more developers like me.