Toyama, Daniel; Philippe Hamel; Anita Gergely; Gheorghe Comanici; Amelia Glaese; Zafarali Ahmed, Tyler Jackson; Shibl Mourad; Doina Precup
Year of Publication
2021
Summary
This paper introduces AndroidEnv, an open-source reinforcement learning research platform built on the Android ecosystem. The platform allows multiple reinforcement learning agents to interact with various apps and services through a universal touchscreen interface.
The platform offers various salient features, which are listed below.
Real-time execution: The Android OS works independently of the reinforcement learning agents and functions at its own pace. Agents cannot control the behavior of the OS, which makes it a naturalistic platform for agent training.
Action Interface: The platform supports simple actions such as TOUCH, LIFT, REPEAT and combinations of actions to perform gestures, which helps us produce meaningful results.
Observations: The observation data structure is task-agnostic and comprises of parameters like pixels, time-delta, and orientation. Additionally, it also sends some structured data with observation data to aid the understanding of the task.
The platform defines a preset task-definition in the form of protocol buffer messages, which can be used to define the variety of tasks on Android, for example: setting an alarm, turning the night mode on, etc. Additionally, the platform also provides an initial set of ready-to-use tasks in different domains such as time-sensitive tasks, physics-based tasks, puzzles, card games, UI Navigation, strategy games, etc., which serve as a reference for the user to create their tasks.
The paper also provides some experimental results by using multiple baseline RL agents on different types of tasks. After experimentation, it is found that all the agents perform particularly well on tasks that have a simple action interface, whereas the combination of a highly structured interface, time sensitivity, and sparse rewards makes it difficult for the agents to solve.
In the final section, various RL Research platforms are described, and the environmental properties of each platform are compared. While platforms like Atari, DM Lab, DM Control Suite, and Minecraft lack some of the important environmental properties, newer platforms such as OpenAI Universe, World of Bits, and Android Env support features such as Universal Interface, Extensible Task Suite, Real-Time Execution, and Continuous Action Space.
Contributions of The Paper
The platform provides a general unified interface built on the Android ecosystem, which builds upon the approach that the accuracy of reinforcement learning agents can be measured via diverse challenges. This approach also avoids the pitfalls of agents overspecializing on a particular task and failing to perform general tasks.
The platform’s state-of-the-art features, such as real-time simulation and universal observation and action space, make it a suitable medium for reinforcement learning research for a wide range of tasks.
The reinforcement agents trained on the platform could help us create real-world impact, such as voice navigation, better user experience, and device testing and quality assurance.
Publisher
arXiv Preprint
Link to The Paper
https://arxiv.org/pdf/2105.13231.pdf
Name of The Authors
Toyama, Daniel; Philippe Hamel; Anita Gergely; Gheorghe Comanici; Amelia Glaese; Zafarali Ahmed, Tyler Jackson; Shibl Mourad; Doina Precup
Year of Publication
2021
Summary
This paper introduces AndroidEnv, an open-source reinforcement learning research platform built on the Android ecosystem. The platform allows multiple reinforcement learning agents to interact with various apps and services through a universal touchscreen interface.
The platform offers various salient features, which are listed below.
The platform defines a preset task-definition in the form of protocol buffer messages, which can be used to define the variety of tasks on Android, for example: setting an alarm, turning the night mode on, etc. Additionally, the platform also provides an initial set of ready-to-use tasks in different domains such as time-sensitive tasks, physics-based tasks, puzzles, card games, UI Navigation, strategy games, etc., which serve as a reference for the user to create their tasks.
The paper also provides some experimental results by using multiple baseline RL agents on different types of tasks. After experimentation, it is found that all the agents perform particularly well on tasks that have a simple action interface, whereas the combination of a highly structured interface, time sensitivity, and sparse rewards makes it difficult for the agents to solve.
In the final section, various RL Research platforms are described, and the environmental properties of each platform are compared. While platforms like Atari, DM Lab, DM Control Suite, and Minecraft lack some of the important environmental properties, newer platforms such as OpenAI Universe, World of Bits, and Android Env support features such as Universal Interface, Extensible Task Suite, Real-Time Execution, and Continuous Action Space.
Contributions of The Paper
Comments
No response