While large language models (LMs) have shown remarkable capabilities acrossnumerous tasks, they often struggle with simple reasoning and planning inphysical environments, such as understanding object permanence or planninghousehold activities. The limitation arises from the fact that LMs are trainedonly on written text and miss essential embodied knowledge and skills. In thispaper, we propose a new paradigm of enhancing LMs by finetuning them with worldmodels, to gain diverse embodied knowledge while retaining their generallanguage capabilities. Our approach deploys an embodied agent in a world model,particularly a simulator of the physical world (VirtualHome), and acquires adiverse set of embodied experiences through both goal-oriented planning andrandom exploration. These experiences are then used to finetune LMs to teachdiverse abilities of reasoning and acting in the physical world, e.g., planningand completing goals, object permanence and tracking, etc. Moreover, it isdesirable to preserve the generality of LMs during finetuning, whichfacilitates generalizing the embodied knowledge across tasks rather than beingtied to specific simulations. We thus further introduce the classical elasticweight consolidation (EWC) for selective weight updates, combined with low-rankadapters (LoRA) for training efficiency. Extensive experiments show ourapproach substantially improves base LMs on 18 downstream tasks by 64.28% onaverage. In particular, the small LMs (1.3B and 6B) enhanced by our approachmatch or even outperform much larger LMs (e.g., ChatGPT).
URL
Affiliations
Abstract
Translation (by gpt-3.5-turbo)
Summary (by gpt-3.5-turbo)