MineDojo / Voyager

An Open-Ended Embodied Agent with Large Language Models
https://voyager.minedojo.org/
MIT License
5.65k stars 536 forks source link

Biome Field Empty String #160

Closed egeozguroglu closed 5 months ago

egeozguroglu commented 5 months ago

When we run Voyager, the biome field returned by mineflayer (event state) is always an empty string. Below is a sample of the iteration log.

Our version is: fabric-server-mc.1.19-loader.0.14.18-launcher.0.11.2 We're running a dedicated MC server (to retrieve an MC port), as in: https://github.com/MineDojo/Voyager/pull/100

It's not a human readable rendering issue either, the event states saved in ckpt are consistent with biome = ""

@xieleo5 any insights on this?

Mineflayer process has exited, restarting Subprocess mineflayer started with PID 2902281. Server started on port 3000

Starting task Mine 1 wood log for at most 4 times Mineflayer process has exited, restarting Subprocess mineflayer started with PID 2902408. Server started on port 3000

Render Action Agent system message with 0 skills Action Agent human message Code from the last round: No code in the first round

Execution error: No error

Chat log: None

Biome:

Time: day

Nearby blocks: grass, dirt, stone, grass_block, fern, oak_leaves, jungle_log, vine, coal_ore

Nearby entities (nearest to farthest): sheep, chicken, rabbit

Health: 20.0/20

Hunger: 20.0/20

Position: x=848.5, y=75.0, z=848.5

Equipment: [None, None, None, None, None, None]

Inventory (0/36): Empty

Chests: None

Task: Mine 1 wood log

Context: You can mine one of oak, birch, spruce, jungle, acacia, dark oak, or mangrove logs.

Critique: None

 Action Agent ai message Explain: The task is to mine one wood log. The nearby blocks include jungle_log, which is a valid type of wood log. The bot has no equipment or items in its inventory, so it can directly mine the jungle_log.

Plan: 1) Locate the nearest jungle_log block. 2) Mine the jungle_log block. 3) Confirm that the log has been added to the inventory.

Code:


async function mineWoodLog(bot) {
    // Locate the nearest jungle_log block
    const jungleLog = bot.findBlock({
        matching: mcData.blocksByName["jungle_log"].id,
        maxDistance: 32,
    });

    if (!jungleLog) {
        bot.chat("No jungle log found nearby.");
        return;
    }

    // Mine the jungle_log block
    await mineBlock(bot, "jungle_log", 1);

    // Confirm that the log has been added to the inventory
    const log = bot.inventory.findInventoryItem(mcData.itemsByName["jungle_log"].id);
    if (log) {
        bot.chat("Successfully mined a jungle log.");
    } else {
        bot.chat("Failed to mine a jungle log.");
    }
}
```
****Recorder message: 166 ticks have elapsed****
****Recorder message: 1 iteration passed****
****Critic Agent human message****
Biome: 

Time: day

Nearby blocks: grass_block, dirt, stone, vine, fern, grass, oak_leaves

Health: 20.0/20

Hunger: 20.0/20

Position: x=844.7, y=75.0, z=845.5

Equipment: [None, None, None, None, None, None]

Inventory (0/36): Empty

Chests: None

Task: Mine 1 wood log

Context: You can mine one of oak, birch, spruce, jungle, acacia, dark oak, or mangrove logs.


****Critic Agent ai message****
{
    "reasoning": "Your inventory is empty, which means you haven't mined any wood logs yet. The task requires you to mine at least one wood log, but there are no wood logs in your inventory.",
    "success": false,
    "critique": "Find a tree and mine one of its logs. You can mine oak, birch, spruce, jungle, acacia, dark oak, or mangrove logs to complete the task."
}
****Action Agent human message****
Code from the last round:
async function mineWoodLog(bot) {
  // Locate the nearest jungle_log block
  const jungleLog = bot.findBlock({
    matching: mcData.blocksByName["jungle_log"].id,
    maxDistance: 32
  });
  if (!jungleLog) {
    bot.chat("No jungle log found nearby.");
    return;
  }

  // Mine the jungle_log block
  await mineBlock(bot, "jungle_log", 1);

  // Confirm that the log has been added to the inventory
  const log = bot.inventory.findInventoryItem(mcData.itemsByName["jungle_log"].id);
  if (log) {
    bot.chat("Successfully mined a jungle log.");
  } else {
    bot.chat("Failed to mine a jungle log.");
  }
}

Execution error: No error

Chat log: Collect finish!
Failed to mine a jungle log.

Biome: 

Time: day

Nearby blocks: grass_block, dirt, stone, vine, fern, grass, oak_leaves

Nearby entities (nearest to farthest): None

Health: 20.0/20

Hunger: 20.0/20

Position: x=844.7, y=75.0, z=845.5

Equipment: [None, None, None, None, None, None]

Inventory (0/36): Empty

Chests: None

Task: Mine 1 wood log

Context: You can mine one of oak, birch, spruce, jungle, acacia, dark oak, or mangrove logs.

Critique: Find a tree and mine one of its logs. You can mine oak, birch, spruce, jungle, acacia, dark oak, or mangrove logs to complete the task.


****Action Agent ai message****
Explain: The previous code was specifically looking for a "jungle_log" block, but there were no jungle logs nearby. The task allows for any type of wood log, so we should search for any of the acceptable log types.

Plan:
1) Search for any type of wood log (oak, birch, spruce, jungle, acacia, dark oak, or mangrove) within a 32-block radius.
2) If no logs are found, use `exploreUntil` to explore the area until a log is found.
3) Mine the found log.
4) Confirm that the log has been added to the inventory and notify the user.

Code:
```javascript
async function mineAnyWoodLog(bot) {
  const logTypes = ["oak_log", "birch_log", "spruce_log", "jungle_log", "acacia_log", "dark_oak_log", "mangrove_log"];
  let logBlock = null;

  // Search for any type of wood log within a 32-block radius
  for (const logType of logTypes) {
    logBlock = bot.findBlock({
      matching: mcData.blocksByName[logType].id,
      maxDistance: 32
    });
    if (logBlock) break;
  }

  // If no logs are found, explore the area until a log is found
  if (!logBlock) {
    bot.chat("No logs found nearby. Exploring...");
    logBlock = await exploreUntil(bot, new Vec3(1, 0, 1), 60, () => {
      for (const logType of logTypes) {
        const foundLog = bot.findBlock({
          matching: mcData.blocksByName[logType].id,
          maxDistance: 32
        });
        if (foundLog) return foundLog;
      }
      return null;
    });
  }

  if (!logBlock) {
    bot.chat("Failed to find any logs.");
    return;
  }

  // Mine the found log
  await mineBlock(bot, logBlock.name, 1);

  // Confirm that the log has been added to the inventory
  const log = bot.inventory.findInventoryItem(mcData.itemsByName[logBlock.name].id);
  if (log) {
    bot.chat("Successfully mined a log.");
  } else {
    bot.chat("Failed to mine a log.");
  }
}
```
****Recorder message: 985 ticks have elapsed****
****Recorder message: 2 iteration passed****
****Critic Agent human message****
Biome: 

Time: day

Nearby blocks: grass_block, dirt, stone, grass, fern, oak_leaves, jungle_log, vine

Health: 20.0/20

Hunger: 20.0/20

Position: x=863.5, y=71.0, z=846.5

Equipment: [None, None, None, None, None, None]

Inventory (0/36): Empty

Chests: None

Task: Mine 1 wood log

Context: You can mine one of oak, birch, spruce, jungle, acacia, dark oak, or mangrove logs.


****Critic Agent ai message****
{
    "reasoning": "You need to mine 1 wood log, and there is a jungle_log nearby. However, your inventory is empty, which means you haven't mined any wood logs yet.",
    "success": false,
    "critique": "Mine the jungle_log nearby to complete the task of mining 1 wood log."
}
****Action Agent human message****
Code from the last round:
async function mineAnyWoodLog(bot) {
  const logTypes = ["oak_log", "birch_log", "spruce_log", "jungle_log", "acacia_log", "dark_oak_log", "mangrove_log"];
  let logBlock = null;

  // Search for any type of wood log within a 32-block radius
  for (const logType of logTypes) {
    logBlock = bot.findBlock({
      matching: mcData.blocksByName[logType].id,
      maxDistance: 32
    });
    if (logBlock) break;
  }

  // If no logs are found, explore the area until a log is found
  if (!logBlock) {
    bot.chat("No logs found nearby. Exploring...");
    logBlock = await exploreUntil(bot, new Vec3(1, 0, 1), 60, () => {
      for (const logType of logTypes) {
        const foundLog = bot.findBlock({
          matching: mcData.blocksByName[logType].id,
          maxDistance: 32
        });
        if (foundLog) return foundLog;
      }
      return null;
    });
  }
  if (!logBlock) {
    bot.chat("Failed to find any logs.");
    return;
  }

  // Mine the found log
  await mineBlock(bot, logBlock.name, 1);

  // Confirm that the log has been added to the inventory
  const log = bot.inventory.findInventoryItem(mcData.itemsByName[logBlock.name].id);
  if (log) {
    bot.chat("Successfully mined a log.");
  } else {
    bot.chat("Failed to mine a log.");
  }
}

Execution error: No error

Chat log: Collect finish!
Failed to mine a log.

Biome: 

Time: day

Nearby blocks: grass_block, dirt, stone, grass, fern, oak_leaves, jungle_log, vine

Nearby entities (nearest to farthest): sheep

Health: 20.0/20

Hunger: 20.0/20

Position: x=863.5, y=71.0, z=846.5

Equipment: [None, None, None, None, None, None]

Inventory (0/36): Empty

Chests: None

Task: Mine 1 wood log

Context: You can mine one of oak, birch, spruce, jungle, acacia, dark oak, or mangrove logs.

Critique: Mine the jungle_log nearby to complete the task of mining 1 wood log.


****Action Agent ai message****
Explain: The previous code attempted to find and mine any type of wood log but failed to correctly identify and mine the nearby `jungle_log`. The issue likely lies in the `mineBlock` function call, which expects the block name as a string, not the block object.

Plan:
1) Identify the nearby `jungle_log` block.
2) Use the `mineBlock` function to mine the `jungle_log`.
3) Confirm that the log has been added to the inventory.

Code:
```javascript
async function mineWoodLog(bot) {
  const logType = "jungle_log";

  // Search for the jungle log within a 32-block radius
  const logBlock = bot.findBlock({
    matching: mcData.blocksByName[logType].id,
    maxDistance: 32
  });

  if (!logBlock) {
    bot.chat("No jungle logs found nearby.");
    return;
  }

  // Mine the found jungle log
  await mineBlock(bot, logType, 1);

  // Confirm that the log has been added to the inventory
  const log = bot.inventory.findInventoryItem(mcData.itemsByName[logType].id);
  if (log) {
    bot.chat("Successfully mined a jungle log.");
  } else {
    bot.chat("Failed to mine a jungle log.");
  }
}
```
****Recorder message: 1132 ticks have elapsed****
****Recorder message: 3 iteration passed****
****Critic Agent human message****
Biome: 

Time: day

Nearby blocks: dirt, stone, grass_block, grass, fern, vine, oak_leaves

Health: 20.0/20

Hunger: 20.0/20

Position: x=864.3, y=68.0, z=841.5

Equipment: [None, None, None, None, None, None]

Inventory (0/36): Empty

Chests: None

Task: Mine 1 wood log

Context: You can mine one of oak, birch, spruce, jungle, acacia, dark oak, or mangrove logs.


****Critic Agent ai message****
{
    "reasoning": "Your inventory is empty, which means you haven't mined any wood logs yet. The task requires you to mine at least one wood log, but there are no wood logs in your inventory.",
    "success": false,
    "critique": "Find a tree and mine one of its logs. You can mine oak, birch, spruce, jungle, acacia, dark oak, or mangrove logs to complete the task."
}
****Action Agent human message****
Code from the last round:
async function mineWoodLog(bot) {
  const logType = "jungle_log";

  // Search for the jungle log within a 32-block radius
  const logBlock = bot.findBlock({
    matching: mcData.blocksByName[logType].id,
    maxDistance: 32
  });
  if (!logBlock) {
    bot.chat("No jungle logs found nearby.");
    return;
  }

  // Mine the found jungle log
  await mineBlock(bot, logType, 1);

  // Confirm that the log has been added to the inventory
  const log = bot.inventory.findInventoryItem(mcData.itemsByName[logType].id);
  if (log) {
    bot.chat("Successfully mined a jungle log.");
  } else {
    bot.chat("Failed to mine a jungle log.");
  }
}

Execution error: No error

Chat log: Collect finish!
Failed to mine a jungle log.

Biome: 

Time: day

Nearby blocks: dirt, stone, grass_block, grass, fern, vine, oak_leaves

Nearby entities (nearest to farthest): None

Health: 20.0/20

Hunger: 20.0/20

Position: x=864.3, y=68.0, z=841.5

Equipment: [None, None, None, None, None, None]

Inventory (0/36): Empty

Chests: None

Task: Mine 1 wood log

Context: You can mine one of oak, birch, spruce, jungle, acacia, dark oak, or mangrove logs.

Critique: Find a tree and mine one of its logs. You can mine oak, birch, spruce, jungle, acacia, dark oak, or mangrove logs to complete the task.


****Action Agent ai message****
Explain: The previous code was specifically looking for a "jungle_log" within a 32-block radius. However, there were no jungle logs nearby, which caused the task to fail. The task can be completed by mining any type of wood log, so we should search for any type of log instead of just jungle logs.

Plan:
1) Search for any type of wood log (oak, birch, spruce, jungle, acacia, dark oak, or mangrove) within a 32-block radius.
2) If no logs are found, use `exploreUntil` to explore the area until a log is found.
3) Mine the found log.
4) Confirm that the log has been added to the inventory and notify the user.

Code:
```javascript
async function mineWoodLog(bot) {
  const logTypes = ["oak_log", "birch_log", "spruce_log", "jungle_log", "acacia_log", "dark_oak_log", "mangrove_log"];

  // Function to find any type of log
  function findLog() {
    for (const logType of logTypes) {
      const logBlock = bot.findBlock({
        matching: mcData.blocksByName[logType].id,
        maxDistance: 32
      });
      if (logBlock) {
        return logBlock;
      }
    }
    return null;
  }

  // Search for any log within a 32-block radius
  let logBlock = findLog();
  if (!logBlock) {
    bot.chat("No logs found nearby. Exploring...");
    // Explore until a log is found
    logBlock = await exploreUntil(bot, new Vec3(1, 0, 1), 60, findLog);
    if (!logBlock) {
      bot.chat("Failed to find any logs.");
      return;
    }
  }

  // Determine the type of the found log
  const logType = logTypes.find(type => mcData.blocksByName[type].id === logBlock.type);

  // Mine the found log
  await mineBlock(bot, logType, 1);

  // Confirm that the log has been added to the inventory
  const log = bot.inventory.findInventoryItem(mcData.itemsByName[logType].id);
  if (log) {
    bot.chat("Successfully mined a log.");
  } else {
    bot.chat("Failed to mine a log.");
  }
}
```