As a roboticist, I am very interested in real-world reinforcement learning, and highly encourage all research directed at making RL work robustly and efficiently on physical robot systems. Your work certainly raises awareness for the issues that arise in doing so, and I think that's great.
With this background, I was also a bit surprised to find out that the real world you consider is a simulator. To me, the real world is the physical world that humans and robots physically occupy, rather than our simulations of this world, such as MuJoCo, Bullet, etc. (we cannot exclude that what we consider to be the real world is in fact a Matrix-like simulation, but let that philosophical byway not distract us here).
The issue to me is that if simulated worlds are now considered to be the "real world", I'm no longer sure how to call the world in which we roboticists conduct our experiments. What would you call this world?
As a roboticist, I am very interested in real-world reinforcement learning, and highly encourage all research directed at making RL work robustly and efficiently on physical robot systems. Your work certainly raises awareness for the issues that arise in doing so, and I think that's great.
With this background, I was also a bit surprised to find out that the real world you consider is a simulator. To me, the real world is the physical world that humans and robots physically occupy, rather than our simulations of this world, such as MuJoCo, Bullet, etc. (we cannot exclude that what we consider to be the real world is in fact a Matrix-like simulation, but let that philosophical byway not distract us here).
The issue to me is that if simulated worlds are now considered to be the "real world", I'm no longer sure how to call the world in which we roboticists conduct our experiments. What would you call this world?