Problem parsing XML or accessing AgentObservationsPort?

[seannyD]

I just installed gym-minecraft and I'm trying to run the first default mission ('MinecraftBasic-v0'), but having problems getting the mission to start. I'm using the test.py code:

import logging
logging.basicConfig(level=logging.DEBUG)

import gym
import gym_minecraft

env = gym.make('MinecraftBasic-v0')
env.init(start_minecraft=True)
env.reset()

done = False
while not done:
        env.render()
        action = env.action_space.sample()
        obs, reward, done, info = env.step(action)

env.close()

Minecraft starts, but then I get this error:

INFO:minecraft_py:Minecraft process ready
INFO:gym_minecraft.envs.minecraft_env:Started Minecraft on port 10000, overriding client_pool.
DEBUG:gym_minecraft.envs.minecraft_env:ContinuousMovement:crouch
DEBUG:gym_minecraft.envs.minecraft_env:ContinuousMovement:jump
DEBUG:gym_minecraft.envs.minecraft_env:ContinuousMovement:move
DEBUG:gym_minecraft.envs.minecraft_env:ContinuousMovement:pitch
DEBUG:gym_minecraft.envs.minecraft_env:ContinuousMovement:strafe
DEBUG:gym_minecraft.envs.minecraft_env:ContinuousMovement:turn
DEBUG:gym_minecraft.envs.minecraft_env:ContinuousMovement:use
DEBUG:gym_minecraft.envs.minecraft_env:Discrete(14)
WARNING:gym_minecraft.envs.minecraft_env:Error starting mission: Internal error: the XML we generate does not validate: instance document parsing failed
:1:2897 error: element 'AgentObservationsPort' is not allowed for content model '(ClientIPAddress,ClientMissionControlPort,ClientCommandsPort,AgentIPAddress,AgentMissionControlPort,AgentVideoPort,AgentDepthPort,AgentLuminancePort,AgentObservationsPort,AgentRewardsPort,AgentColourMapPort)'
<?xml version="1.0" encoding="UTF-8" standalone="no" ?><MissionInit xmlns="http://ProjectMalmo.microsoft.com" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://ProjectMalmo.microsoft.com MissionInit.xsd"><Mission><About><Summary>Find the goal!</Summary></About><ModSettings><PrioritiseOffscreenRendering>true</PrioritiseOffscreenRendering></ModSettings><ServerSection><ServerInitialConditions><Time><StartTime>6000</StartTime><AllowPassageOfTime>false</AllowPassageOfTime></Time><Weather>clear</Weather><AllowSpawning>false</AllowSpawning></ServerInitialConditions><ServerHandlers><FlatWorldGenerator destroyAfterUse="true" forceReset="false" generatorString="3;7,220*1,5*3,2;3;,biome_1" seed=""/><ClassroomDecorator seed="__SEED__"><specification><width>7</width><height>7</height><length>7</length><pathLength>0</pathLength><divisions><southNorth>0</southNorth><eastWest>0</eastWest><aboveBelow>0</aboveBelow></divisions><horizontalObstacles><gap>0</gap><bridge>0</bridge><door>0</door><puzzle>0</puzzle><jump>0</jump></horizontalObstacles><verticalObstacles><stairs>0</stairs><ladder>0</ladder><jump>0</jump></verticalObstacles><hintLikelihood>1</hintLikelihood></specification></ClassroomDecorator><ServerQuitFromTimeUp description="out_of_time" timeLimitMs="30000"/><ServerQuitWhenAnyAgentFinishes description=""/></ServerHandlers></ServerSection><AgentSection mode="Survival"><Name>James Bond</Name><AgentStart><Placement pitch="0" x="-203.5" y="81" yaw="0" z="217.5"/></AgentStart><AgentHandlers><ObservationFromFullStats/><VideoProducer viewpoint="0" want_depth="false"><Width>320</Width><Height>240</Height></VideoProducer><RewardForTouchingBlockType dimension="0"><Block behaviour="oncePerBlock" cooldownInMs="1000" reward="20" type="gold_ore diamond_ore redstone_ore"/></RewardForTouchingBlockType><RewardForSendingCommand dimension="0" reward="-1"/><RewardForMissionEnd dimension="0" rewardForDeath="-10000"><Reward description="found_goal" reward="1000"/><Reward description="out_of_time" reward="-1000"/></RewardForMissionEnd><ContinuousMovementCommands turnSpeedDegs="180"><ModifierList type="deny-list"><command>attack</command></ModifierList></ContinuousMovementCommands><AgentQuitFromTouchingBlockType><Block description="found_goal" type="gold_block diamond_block redstone_block"/></AgentQuitFromTouchingBlockType></AgentHandlers></AgentSection></Mission><ExperimentUID>experiment_id</ExperimentUID><ClientRole>0</ClientRole><ClientAgentConnection><ClientIPAddress>127.0.0.1</ClientIPAddress><ClientMissionControlPort>10000</ClientMissionControlPort><ClientCommandsPort>0</ClientCommandsPort><AgentIPAddress>127.0.0.1</AgentIPAddress><AgentMissionControlPort>10035</AgentMissionControlPort><AgentVideoPort>10472</AgentVideoPort><AgentObservationsPort>10416</AgentObservationsPort><AgentRewardsPort>10309</AgentRewardsPort></ClientAgentConnection></MissionInit>

INFO:gym_minecraft.envs.minecraft_env:Sleeping for 10 seconds...

I haven't changed any of the xml files, and the xml text is structurally valid. I also tried loading the mission file from another source:

env = gym.make('MinecraftBasic-v0')
env.load_mission_file("basic2.xml")

The script loaded the mission file correctly, but I still get the same error.

Then there's the other error: error: element 'AgentObservationsPort' is not allowed for content model. This seems to be linked to the ClientAgentConnection in Malmo https://github.com/Microsoft/malmo/blob/master/Schemas/MissionInit.xsd

Any help would be much appreciated!

[tambetm]

That looks weird, because basic.xml does not have AgentObservationsPort element at all. Can you check the location where gym-minecraft is installed with import gym_minecraft; print(gym_minecraft.__file__) and then make sure that assets/basic.xml under that directory does not mention AgentObservationsPort?

[seannyD]

gym-minecraft was installed at /Library/Python/2.7/site-packages/gym_minecraft-0.0.2-py2.7.egg/gym_minecraft/__init__.pyc, and the assets/basic.xml file is identical to the one in the repository (no mention of AgentObservationsPort).

[tambetm]

Did you use minecraft_py repository version? Recent pull request added opportunity to use different Malmo installation by setting MALMO_XSD_PATH environment variable. Do you have that set and maybe that points to newer version of Malmo than required by basic.xml?

[seannyD]

You're right - I was using another version of Malmo because I couldn't get the repository version to work. Here's the problem:

If I set MALMO_XSD_PATH to ~/Documents/Bristol/Minecraft_Malmo/minecraft-py/minecraft_py/Malmo/Schemas/, then I get this error:

OSError: [Errno 2] No such file or directory: '/Users/sgroberts/Documents/Bristol/Minecraft_Malmo/minecraft-py/minecraft_py/Malmo/Schemas/Minecraft'

So I set MALMO_XSD_PATH to ~/Documents/Bristol/Minecraft_Malmo/minecraft-py/minecraft_py/Malmo/ (which has the Minecraft folder), but then I get this error:

RuntimeError: Schema file Mission.xsd not found in folder specified by MALMO_XSD_PATH environment variable: /Users/sgroberts/Documents/Bristol/Minecraft_Malmo/minecraft-py/minecraft_py/Malmo/

I tried running launchClient.sh from minecraft-py/minecraft_py/Malmo/Minecraft, but I get this error during runClient:

java.lang.reflect.InvocationTargetException
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_151]
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_151]
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_151]
    at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_151]
    at net.minecraft.launchwrapper.Launch.launch(Launch.java:135) [launchwrapper-1.12.jar:?]
    at net.minecraft.launchwrapper.Launch.main(Launch.java:28) [launchwrapper-1.12.jar:?]
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_151]
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_151]
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_151]
    at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_151]
    at net.minecraftforge.gradle.GradleStartCommon.launch(Unknown Source) [start/:?]
    at GradleStart.main(Unknown Source) [start/:?]
Caused by: java.lang.NullPointerException
    at net.minecraft.crash.CrashReportCategory.firstTwoElementsOfStackTraceMatch(Unknown Source) ~[CrashReportCategory.class:?]
    at net.minecraft.crash.CrashReport.makeCategoryDepth(Unknown Source) ~[CrashReport.class:?]
    at net.minecraft.crash.CrashReport.makeCategory(Unknown Source) ~[CrashReport.class:?]
    at net.minecraft.client.Minecraft.run(Unknown Source) ~[Minecraft.class:?]
    at net.minecraft.client.main.Main.main(Unknown Source) ~[Main.class:?]
    ... 12 more

[tambetm]

Either don't set MALMO_XSD_PATH at all or set it to ~/Documents/Bristol/Minecraft_Malmo/minecraft-py/minecraft_py/Malmo/Schemas, i.e. without trailing slash. That might be confusing the code.

Alternatively, if you want to keep using your latest Malmo version, set MALMO_XSD_PATH as it was previously and manually load the latest basic.xml.

I'm not really sure about the last Java error. If you are unlucky then two Minecraft installations might have installed conflicting Gradle versions. But I really hope not.

[seannyD]

Thanks, I set it without the trailing slash, and now it starts properly, but fails here:

Traceback (most recent call last):
  File "Test_Malmo_Gym.py", line 10, in <module>
    env.init(start_minecraft=True)
  File "/Library/Python/2.7/site-packages/gym_minecraft-0.0.2-py2.7.egg/gym_minecraft/envs/minecraft_env.py", line 114, in init
    self.mc_process, port = minecraft_py.start()
  File "/Library/Python/2.7/site-packages/minecraft_py-0.0.2-py2.7.egg/minecraft_py/__init__.py", line 76, in start
    raise EOFError("Minecraft process finished unexpectedly")
EOFError: Minecraft process finished unexpectedly

I suspect it is something to do with conflicting versions. But the xml problem is fixed, so I suggest you close this issue and I'll try reinstalling everything.

Thanks for your help!

[tambetm]

I think this comes down to the NullPointerException above. It will be hard to troubleshoot generic error like this remotely. But your independently installed Malmo version seemed to work fine, you could set MALMO_XSD_PATH back as it was previously and try with latest basic.xml from Malmo repo.