Error parsing maps

[drticcz]

Describe the bug

Saving maps in valetudo seems to be failing. I can see completed map after the first cleanup though. However there are two issues: It never persist over two cleanups or reboot and I cannot see the zones created by the robot, althoug they are in the png file saved in the robot...

Strange is that he is trying to save the map even after device reboot - apparently the map from previous cleanup stays in the robot but it doesn't get to the Valetudo.

To Reproduce

Screenshots

Vacuum Model

Xiaomi branded viomi v7? Model STYTJ02YM

/etc/sysconf/device.conf:model=viomi.vacuum.v7

Valetudo Version

2022.02.0

Expected behavior

Map is saved including the zones and it persist between the cleanups

Additional context

Logs

valetudo.log

Full log file attached, this is just a snippet of the error part.

[2022-02-16T04:08:26.009Z] [DEBUG] Map upload started with: {
  query: {
    ts: '1867295070766',
    suffix: 'urls',
    Expires: '1644985154',
    index: '0',
    method: '_sync.gen_tmp_presigned_url'
  },
  params: { filename: undefined }
}
[2022-02-16T04:08:26.089Z] [ERROR] Error parsing map. Dump saved in /tmp/mapdata34
[2022-02-16T04:08:26.091Z] [ERROR] unhandledRejection {
  reason: Error: Invalid pixels array
      at new MapLayer (/snapshot/Valetudo/backend/lib/entities/map/MapLayer.js:29:19)
      at ViomiMapParser.convertToValetudoMap (/snapshot/Valetudo/backend/lib/robots/viomi/ViomiMapParser.js:280:25)
      at ViomiMapParser.parse (/snapshot/Valetudo/backend/lib/robots/viomi/ViomiMapParser.js:216:21)
      at ViomiV7ValetudoRobot.parseMap (/snapshot/Valetudo/backend/lib/robots/viomi/ViomiValetudoRobot.js:510:50)
      at /snapshot/Valetudo/backend/lib/robots/MiioValetudoRobot.js:470:42
      at runMicrotasks (<anonymous>)
      at processTicksAndRejections (node:internal/process/task_queues:96:5),
  stack: 'Error: Invalid pixels array\n' +
    '    at new MapLayer (/snapshot/Valetudo/backend/lib/entities/map/MapLayer.js:29:19)\n' +
    '    at ViomiMapParser.convertToValetudoMap (/snapshot/Valetudo/backend/lib/robots/viomi/ViomiMapParser.js:280:25)\n' +
    '    at ViomiMapParser.parse (/snapshot/Valetudo/backend/lib/robots/viomi/ViomiMapParser.js:216:21)\n' +
    '    at ViomiV7ValetudoRobot.parseMap (/snapshot/Valetudo/backend/lib/robots/viomi/ViomiValetudoRobot.js:510:50)\n' +
    '    at /snapshot/Valetudo/backend/lib/robots/MiioValetudoRobot.js:470:42\n' +
    '    at runMicrotasks (<anonymous>)\n' +
    '    at processTicksAndRejections (node:internal/process/task_queues:96:5)',
  promise: Promise {
    <rejected> Error: Invalid pixels array
        at new MapLayer (/snapshot/Valetudo/backend/lib/entities/map/MapLayer.js:29:19)
        at ViomiMapParser.convertToValetudoMap (/snapshot/Valetudo/backend/lib/robots/viomi/ViomiMapParser.js:280:25)
        at ViomiMapParser.parse (/snapshot/Valetudo/backend/lib/robots/viomi/ViomiMapParser.js:216:21)
        at ViomiV7ValetudoRobot.parseMap (/snapshot/Valetudo/backend/lib/robots/viomi/ViomiValetudoRobot.js:510:50)
        at /snapshot/Valetudo/backend/lib/robots/MiioValetudoRobot.js:470:42
        at runMicrotasks (<anonymous>)
        at processTicksAndRejections (node:internal/process/task_queues:96:5)
  }
}
[2022-02-16T04:08:26.096Z] [DEBUG] <<< cloud: {"result":["ok"],"id":43}
[2022-02-16T04:08:26.097Z] [DEBUG] << cloud: ignoring response for non-pending request {"result":["ok"],"id":43}
[2022-02-16T04:08:26.098Z] [WARN] Error while saving unparsable map null

valetudo.log mapdata34.zip

Map PNG I got from the robot filesystem

Map as it looks like during the cleaning in the Valetudo:

[drticcz]

Valetudo conf:

{
  "embedded": true,
  "robot": {
    "implementation": "auto",
    "implementationSpecificConfig": {
      "ip": "127.0.0.1"
    }
  },
  "webserver": {
    "port": 80,
    "basicAuth": {
      "enabled": false,
      "username": "valetudo",
      "password": "valetudo"
    },
    "blockExternalAccess": true
  },
  "zonePresets": {},
  "goToLocationPresets": {},
  "mqtt": {
    "enabled": false,
    "connection": {
      "host": "foobar.example",
      "port": 1883,
      "tls": {
        "enabled": false,
        "ca": ""
      },
      "authentication": {
        "credentials": {
          "enabled": false,
          "username": "",
          "password": ""
        },
        "clientCertificate": {
          "enabled": false,
          "certificate": "",
          "key": ""
        }
      }
    },
    "identity": {
      "friendlyName": "",
      "identifier": ""
    },
    "interfaces": {
      "homie": {
        "enabled": true,
        "addICBINVMapProperty": false,
        "cleanAttributesOnShutdown": false
      },
      "homeassistant": {
        "enabled": true,
        "cleanAutoconfOnShutdown": false
      }
    },
    "customizations": {
      "topicPrefix": "",
      "provideMapData": true
    }
  },
  "ntpClient": {
    "enabled": true,
    "server": "192.168.1.2",
    "port": 123,
    "interval": 28800000,
    "timeout": 10000
  },
  "timers": {},
  "logLevel": "debug",
  "debug": {
    "systemStatInterval": false,
    "debugHassAnchors": false,
    "storeRawUploadedMaps": false
  },
  "networkAdvertisement": {
    "enabled": true
  },
  "updater": {
    "enabled": true,
    "updateProvider": {
      "type": "github",
      "implementationSpecificConfig": {}
    }
  }
}

[Hypfer]

What fw version are you using?

That map file seems to be just empty in regards to map image data. If you inflate it and view it in a hex editor, all you can see are nullbytes for most of that file.

[drticcz]

I have 3.5.3_0047.. I found out, that if I switch the viomi to the debug mode (creating /mnt/UDISK/debug_mode file), the map suddenly appears including the segments. I now managed to start another cleaning on the same map (probably just by switching to debug) but after I manually stopped that cleaning, valetudo returned error about handover of the STOP command and the map got lost again - with same symptoms in the log.

So according to some strace lookup of the running RobotApp and other internal services, they apparently check for the internal ini config files in /mnt/UDISK/config - perhaps there is something in the config file that cause robot to talk to valetudo better but I cannot figure out what it is.

What is the default or minimal working scenario? miio_client connected to the valetudo map server running on port 8053? How about port 8079?

valetudo_2.log

.. about the mapdata - I traced that issue again and it first pops up on the file that is actually bigger and contains some data, followed by few empty files or attempts... check the attachment. mapdata.zip

[drticcz]

.. and surprisingly after docking, I got the map back - strange! :)

[Hypfer]

Huh. Are we redirecting those? if not, it almost seems like it's not entirely unclouded

What is the default or minimal working scenario?

No idea unfortunately. I never had a viomi

[drticcz]

Huh. Are we redirecting those? if not, it almost seems like it's not entirely unclouded

Yes, it is all localhost traffic - I have these URL's in the hosts file:

What is the default or minimal working scenario?

No idea unfortunately. I never had a viomi

I meant for the Valetudo in general- apart from the map server, does it expect any other communication from the robot software?

I am particularly curious about that connection in TIME_WAIT on the 8079 port which I cannot identify

[Hypfer]

Port 8053 is the mocked miio cloud provided by valetudo Port 8079 is the mocked fds http server for map uploads provided by valetudo

Basically you have commands, status data etc which is done via miio and large blobs which are uploaded via http to something that is just some blob storage similar to aws s3. In Valetudo, said mock blob storage does not store the blobs but directly processes them, parsing the map into valetudo-specific data structures and then discarding the raw data.

Every n seconds, Valetudo is sending the poll map miio command which in turn makes the robot upload the latest map data to that mock fds endpoint on port 8079 https://github.com/Hypfer/Valetudo/blob/635d7f91c6fd679d0326bb64efb048626fe07f38/backend/lib/robots/viomi/ViomiValetudoRobot.js#L473-L476

[drticcz]

Seems like this has been magically solved by resetting the map and disabling and enabling again the persistent maps in Valetudo.. it now just works