search

MineDojo / Voyager

An Open-Ended Embodied Agent with Large Language Models
https://voyager.minedojo.org/
MIT License
5.36k stars 492 forks source link

Bot will get stuck when attempting to open an existing chest because the chest itself is buried in sand or dirt #93

Closed nero19960329 closed 1 year ago

nero19960329 commented 1 year ago

Before submitting an issue, make sure you read the FAQ.md

Briefly describe your issue

The bot accidentally discovered a chest while exploring the map and attempted to open it. However, the chest was buried in sand, causing the bot to get stuck. This is likely because the checkItemInsideChest function did not give the bot the ability to dig.

Please provide your python, nodejs, Minecraft, and Fabric versions here

[If applicable] Please provide the GPT conversations that are printed each round.

Details are at below:

``` ****Curriculum Agent human message**** Biome: river Nearby blocks: grass_block, water, oak_leaves, dirt, stone, crafting_table, coal_ore, sand, grass, oak_log Other blocks that are recently seen: None Nearby entities: None Position: x=-31.5, y=64.0, z=48.5 Equipment: [None, None, None, None, None, None] Inventory (16/36): {'furnace': 2, 'wooden_pickaxe': 1, 'oak_planks': 1, 'dirt': 11, 'birch_log': 1, 'coal': 9, 'diorite': 5, 'stick': 9, 'iron_pickaxe': 1, 'iron_ingot': 3, 'stone_sword': 1, 'raw_iron': 3, 'bucket': 1, 'cobblestone': 112, 'oak_sapling': 1} Chests: (-23, 62, 57): Unknown items inside Completed tasks so far: Mine 1 wood log, Obtain 4 oak logs, Craft a wooden pickaxe, Mine 3 stone, Craft a stone pickaxe, Obtain a birch log, Mine 3 coal_ore, Mine 2 iron_ore, Craft a furnace, Smelt 2 raw iron, Craft a stone sword, Craft 1 bucket, Craft an iron pickaxe Failed tasks that are too hard: Craft a wooden axe, Smelt 9 raw iron, Gather 1 mushroom (red or brown) ****Curriculum Agent ai message**** Reasoning: Since there's an unknown chest nearby, it would be a good idea to check its contents to discover potentially useful items. Task: Open chest at (-23, 62, 57) and check items inside. Curriculum Agent Question: How to open chest at (-23, 62, 57) and check items inside in Minecraft? Curriculum Agent Answer: You need to be standing right in front of the chest at the given coordinates and right-click on it to open it. You can then hover your mouse over the items inside to check what they are. Starting task Open chest at (-23, 62, 57) and check items inside for at most 4 times Mineflayer process has exited, restarting Subprocess mineflayer started with PID 19224. Server started on port 3000 Skill Manager retrieving for 5 skills Skill Manager retrieved skills: craftBucket, craftFurnace, craftIronPickaxe, craftStoneSword, craftStonePickaxe Render Action Agent system message with 5 skills ****Action Agent human message**** Code from the last round: No code in the first round Execution error: No error Chat log: None Biome: river Time: day Nearby blocks: grass_block, water, oak_leaves, dirt, stone, coal_ore, sand, grass, oak_log Nearby entities (nearest to farthest): None Health: 20.0/20 Hunger: 20.0/20 Position: x=-31.5, y=64.0, z=48.5 Equipment: [None, None, None, None, 'crafting_table', None] Inventory (17/36): {'furnace': 2, 'wooden_pickaxe': 1, 'oak_planks': 1, 'dirt': 11, 'birch_log': 1, 'coal': 9, 'diorite': 5, 'stick': 9, 'iron_pickaxe': 1, 'crafting_table': 1, 'iron_ingot': 3, 'stone_sword': 1, 'raw_iron': 3, 'bucket': 1, 'cobblestone': 112, 'oak_sapling': 1} Chests: (-23, 62, 57): Unknown items inside Task: Open chest at (-23, 62, 57) and check items inside Context: Question: How to open chest at (-23, 62, 57) and check items inside in Minecraft? Answer: You need to be standing right in front of the chest at the given coordinates and right-click on it to open it. You can then hover your mouse over the items inside to check what they are. Critique: None ****Action Agent ai message**** Explain: There's no error or missing steps in the previous code. Now, we need to open the chest at the specified coordinates and check the items inside. Plan: 1) Move to the chest at (-23, 62, 57) 2) Open the chest and check the items inside 3) Close the chest Code: ```javascript async function openAndCheckChest(bot) { const chestPosition = new Vec3(-23, 62, 57); await checkItemInsideChest(bot, chestPosition); bot.chat("Checked items inside the chest."); } ``` Mineflayer process has exited, restarting Subprocess mineflayer started with PID 21056. Server started on port 3000 Your last round rollout terminated due to error: HTTPConnectionPool(host='127.0.0.1', port=7890): Read timed out. (read timeout=300) Failed to complete task Open chest at (-23, 62, 57) and check items inside. Skipping to next task. Completed tasks: Mine 1 wood log, Obtain 4 oak logs, Craft a wooden pickaxe, Mine 3 stone, Craft a stone pickaxe, Obtain a birch log, Mine 3 coal_ore, Mine 2 iron_ore, Craft a furnace, Smelt 2 raw iron, Craft a stone sword, Craft 1 bucket, Craft an iron pickaxe Failed tasks: Craft a wooden axe, Smelt 9 raw iron, Gather 1 mushroom (red or brown), Open chest at (-23, 62, 57) and check items inside ****Curriculum Agent human message**** Biome: river Nearby blocks: poppy, grass_block, grass, dandelion, dirt Other blocks that are recently seen: None Nearby entities: None Position: x=-31.5, y=67.0, z=32.5 Equipment: [None, None, None, None, 'furnace', None] Inventory (16/36): {'iron_ingot': 3, 'stone_sword': 1, 'raw_iron': 3, 'bucket': 1, 'cobblestone': 112, 'oak_sapling': 1, 'furnace': 2, 'wooden_pickaxe': 1, 'oak_planks': 1, 'dirt': 11, 'birch_log': 1, 'coal': 9, 'diorite': 5, 'stick': 9, 'iron_pickaxe': 1} Chests: (-23, 62, 57): Unknown items inside Completed tasks so far: Mine 1 wood log, Obtain 4 oak logs, Craft a wooden pickaxe, Mine 3 stone, Craft a stone pickaxe, Obtain a birch log, Mine 3 coal_ore, Mine 2 iron_ore, Craft a furnace, Smelt 2 raw iron, Craft a stone sword, Craft 1 bucket, Craft an iron pickaxe Failed tasks that are too hard: Craft a wooden axe, Smelt 9 raw iron, Gather 1 mushroom (red or brown), Open chest at (-23, 62, 57) and check items inside ****Curriculum Agent ai message**** Reasoning: Since there is an unknown chest nearby, it would be useful to explore its contents to potentially find valuable items and resources. Task: Open chest at (-23, 62, 57) and check items inside. Starting task Open chest at (-23, 62, 57) and check items inside for at most 4 times ```
xieleo5 commented 1 year ago

Hi, thanks for pointing out this issue. This should be a very uncommon case. If the bot gets buried in sand, it's probably that it will just die and respawn. Also, there's a max_retry for each task, so the bot should not get stuck for a very long time. If you still get stuck, you can manually call /kill bot in the game to respawn the bot.

Let us know if this situation happens very often in your experiments.