Cannot render map with mockPosProvider to debug (still no GNSS)

Hi @lspr98, thanks for the very good project.

I've configured the code to use with my T-Display S3 Pro and the bootscreen is fine. I got Display, SDCard and Map OK. However, I like to test it with no GNSS module and I got no sucess rendering the map with mockPosProvider.

How can I set up to accomplish that?

Can you help me pls?

I've included some of the log: Bike-Companion Build-date: May 31 2024 Attempting to initialize the map... Header is available. Initializing map renderer... Initializing TileBlockRenderer... Zoom scale set to: 1.17 Per tile buffer size: 121508 Total buffer size for all tiles: 1093572 Buffer for tile data allocated successfully. TileBlockRenderer initialized successfully. No position provider available. Proceeding without it... Map initialization successful.

bool UIRenderer::initializeMap(SharedSPISDCard* sd) { Serial.println("Attempting to initialize the map...");

if (_hasHeader) {
    Serial.println("Header is available. Initializing map renderer...");
    _mapRenderer.initialize(_header, sd, _disp);

    if (_hasPositionProvider) {
        Serial.println("Position provider is available. Setting position provider...");
        _mapRenderer.setPositionProvider(_posProvider);
    } else {
        Serial.println("No position provider available. Proceeding without it...");
    }

    Serial.println("Map initialization successful.");
    return true;
} else {
    Serial.println("Map initialization failed: No header available.");
    return false;
}

}

Hey Alessandro,

the map renderer requires a position provider, otherwise it does not know which part of the map to load and render. According to your logs, it seems like you are not registering any position provider. To register the mock position provider and completely disable GNSS initialization, you need to make a few changes in main.cpp:

Provide the interpolation position provider with the mock position provider instead of the gnss. So change line 46 to:
```
InterpPositionProvider ipos(&mockPosProvider, ((float) GNSS_MIN_UPDATE_TIME_MS) / ((float) TARGET_FRAME_TIME_MS));
```
If you have no GNSS connected, you will also need to comment out lines 63 and 80 to disable GNSS initialization.

By default, the mock position is set to a location in Munich, which can be changed in line 38 by altering lat/lon. Make sure to provide a position that is within your map, otherwise it will show a blank screen.

Hi Lspr98, Thanks for the reply! I was able to render a tile by changing this line UIRENDERER.setPositionProvider(&mockPosProvider). I also had to change DEFAULT_ZOOM_LEVEL to 0.3 as the bb and position calculations don't seem to like my display size 480x222 :) Regarding the osm2simpletile, I had to download and test it using Bayern region, as the conversion of Brazil map shows nothing, checking the header... maybe this 0 nodes is the problem?

Brazil map: Bike-Companion Build-date: Jun 2 2024 Header read successfully: Map X: -8210585 Map Y: -3996034 Map Width: 8210585 Map Height: 4592942 Number of X Tiles: 16037 Tile Size: 512 Number of Tiles: 143867927 Max Nodes: 60754 Number of Nodes: 0 Number of Ways: 20972051

This picture shows a still displaced tile for Munich example position

Bayern map: Bike-Companion Build-date: Jun 2 2024 Header read successfully: Map X: 998232 Map Y: 5982838 Map Width: 553372 Map Height: 564671 Number of X Tiles: 1081 Tile Size: 512 Number of Tiles: 1192343 Max Nodes: 2753 Number of Nodes: 22475028 Number of Ways: 10979219

Initializing TileBlockRenderer... Zoom scale set to: 0.28 Per tile buffer size: 5506 Total buffer size for all tiles: 49554 Buffer for tile data allocated successfully. TileBlockRenderer initialized successfully.

Setting new position provider... Position provider set successfully. Provider address: 0x3fc97060

Updating tile buffer... Tile ID: 308655 Initial size: 0 Tile ID: 308656 Initial size: 0 Tile ID: 308657 Initial size: 0 Tile ID: 309736 Initial size: 0 Tile ID: 309737 Initial size: 0 Tile ID: 309738 Initial size: 0 Tile ID: 310817 Initial size: 0 Tile ID: 310818 Initial size: 0 Tile ID: 310819 Initial size: 0

Seeking to pointer offset: 2469320 Tile ID: 308655 Pointer to tile: 26546796 Pointer to next tile: 26549508 Calculated tile size: 2712 Bytes read for tile ID 308655: 2712 Tile ID: 308655 Size after read: 2712 Seeking to pointer offset: 2469328 Failed to seek to tile pointer. Tile ID: 308656 Size after read: 0 Seeking to pointer offset: 2469336 Failed to seek to tile pointer. Tile ID: 308657 Size after read: 0 Seeking to pointer offset: 2477968 Failed to seek to tile pointer. Tile ID: 309736 Size after read: 0 Seeking to pointer offset: 2477976 Failed to seek to tile pointer. Tile ID: 309737 Size after read: 0 Seeking to pointer offset: 2477984 Failed to seek to tile pointer. Tile ID: 309738 Size after read: 0 Seeking to pointer offset: 2486616 Failed to seek to tile pointer. Tile ID: 310817 Size after read: 0 Seeking to pointer offset: 2486624 Failed to seek to tile pointer. Tile ID: 310818 Size after read: 0 Seeking to pointer offset: 2486632 Failed to seek to tile pointer. Tile ID: 310819 Size after read: 0

Tile ID: 308655 LL_x: 1290072 LL_y: 6128758 center.x() 1290702 center.y() 6129547 Offset_x: 630 Offset_y: 789 Disp_LL_x: -223 Disp_LL_y: 394 Disp_UR_x: 1483 Disp_UR_y: 1183 x0: 69 y0: 209 x1: 72 y1: 207 OutOfBounds: No Tile ID: 308655 LL_x: 1290072 LL_y: 6128758 center.x() 1290702 center.y() 6129547 Offset_x: 630 Offset_y: 789 Disp_LL_x: -223 Disp_LL_y: 394 Disp_UR_x: 1483 Disp_UR_y: 1183 x0: 77 y0: 218 x1: 84 y1: 214 OutOfBounds: No Tile ID: 308655 LL_x: 1290072 LL_y: 6128758 center.x() 1290702 center.y() 6129547 Offset_x: 630 Offset_y: 789 Disp_LL_x: -223 Disp_LL_y: 394 Disp_UR_x: 1483 Disp_UR_y: 1183 x0: 84 y0: 214 x1: 92 y1: 210 OutOfBounds: No Tile ID: 308655 LL_x: 1290072 LL_y: 6128758 center.x() 1290702 center.y() 6129547 Offset_x: 630 Offset_y: 789 Disp_LL_x: -223 Disp_LL_y: 394 Disp_UR_x: 1483 Disp_UR_y: 1183 x0: 92 y0: 210 x1: 93 y1: 209 OutOfBounds: No Tile ID: 308655 LL_x: 1290072 LL_y: 6128758 center.x() 1290702 center.y() 6129547 Offset_x: 630 Offset_y: 789 Disp_LL_x: -223 Disp_LL_y: 394 Disp_UR_x: 1483 Disp_UR_y: 1183 x0: 93 y0: 209 x1: 95 y1: 208 OutOfBounds: No Tile ID: 308655 LL_x: 1290072 LL_y: 6128758 center.x() 1290702 center.y() 6129547 Offset_x: 630 Offset_y: 789 Disp_LL_x: -223 Disp_LL_y: 394 Disp_UR_x: 1483 Disp_UR_y: 1183 x0: 95 y0: 208 x1: 97 y1: 207 OutOfBounds: No Tile ID: 308655 LL_x: 1290072 LL_y: 6128758 center.x() 1290702 center.y() 6129547 Offset_x: 630 Offset_y: 789 Disp_LL_x: -223 Disp_LL_y: 394 Disp_UR_x: 1483 Disp_UR_y: 1183 x0: 97 y0: 207 x1: 99 y1: 207 OutOfBounds: No Tile ID: 308655 LL_x: 1290072 LL_y: 6128758 center.x() 1290702 center.y() 6129547 Offset_x: 630 Offset_y: 789 Disp_LL_x: -223 Disp_LL_y: 394 Disp_UR_x: 1483 Disp_UR_y: 1183 x0: 99 y0: 207 x1: 101 y1: 207 OutOfBounds: No Tile ID: 308655 LL_x: 1290072 LL_y: 6128758 center.x() 1290702 center.y() 6129547 Offset_x: 630 Offset_y: 789 Disp_LL_x: -223 Disp_LL_y: 394 Disp_UR_x: 1483 Disp_UR_y: 1183 x0: 101 y0: 207 x1: 106 y1: 215 OutOfBounds: No Tile ID: 308655 LL_x: 1290072 LL_y: 6128758 center.x() 1290702 center.y() 6129547 Offset_x: 630 Offset_y: 789 Disp_LL_x: -223 Disp_LL_y: 394 Disp_UR_x: 1483 Disp_UR_y: 1183 x0: 106 y0: 215 x1: 107 y1: 217 OutOfBounds: No

I'm interested in your simple tile format and probably faster SD Card/bin reading performance. Currently I developed a version using tilemaker/.mbtiles(zoom/x/y) with vtzero decoding that displays very nicely... but the database performance needs to improve. This picture is from my current version: myapp

Would it be feasible to adapt your bin format to use zoom/x/y queries?

Hey Alessandro,

regarding the display issue, it is hard to tell what the error is for DEFAULT_ZOOM_LEVEL = 1.0. Officially, I can only provide support for the display I own (which is the one in the BOM). If you tell me what error you experience for 1.0 zoom, I can try to give you some hints.

However, looking at the other logs you sent, it seems like only the first tile is loaded correctly and the rest fails to load. The first tile is the lower left tile of the buffer, which is coincidentally exactly the area that is rendered in the first image you sent. My first guess would be that there is an issue with the shared SPI bus between your display and the SD-card reader. Whatever changes you made to connect your display, the class that manages your display should inherit from SharedSPIDevice and consequently implement the enableCS and disableCS methods to enable and disable the chip select pin of your display.

Regarding the conversion issue with the Brazil map, I can confirm that the conversion tool fails an assert statement for the Brazil map from Geofabrik (https://download.geofabrik.de/south-america/brazil.html). I will keep you updated on the issue.

For your last question: my format is not able to have varying zoom levels in one single file. It is optimized regarding speed to make it work on the ESP, so some tradeoffs had to be made. However, one thing you could do is to export multiple maps with different tile sizes (changing tile_size in main.cpp of the converter tool) and change the map file to use based on the desired zoom level. Keep in mind that my format does currently not support map styling like you see in the second picture you sent. That is something that is possible to implement and when I have enough time I will look into that but for now, all roads have the same thickness and color.

Hi, I see. There's no error using any zoom, I only reduced it to check the rendering of the tile with content. I'm using LilyGo T-Display-S3-Pro, which relies on TFT_eSPI library and I'll check how/if it may be affecting the SPI bus. With this board, reading from SD Card and using the display is seamless as I don't manage this on my code.

Thanks for checking on Brazil's map.

Regarging your map format, there is no need to have varying zoom nor map styling, the beauty is its simplicity.

Currently, I only use one zoom level (15) with extent 4096. From there, I manage under/over zoom within ranges not to impact performance nor map precision. The whole country mbtile file is about 650MB.

However, the index based on zoom/x/y (XYZ or TMS) to get the correct tileData is needed for the codebase. The tileData would have two important things: Features with linestring (1) Geometry points and (2) Properties: name(of the road), maxspeed(when available) and class (type of the road). These Properties will be used on the code for styling.

It would be great if you can check this possibility.

Example of the pbf tile I current decode:

Layer number: 0 Layer name: roads Layer version: 2 Layer extent: 4096 Keys: 0: name:latin 1: maxspeed 2: class Values: 0: "Alameda Embaixador Sánchez Gavito" [string] 1: "Travessa Mário de Castro" [string] 2: ...

...

Feature 0: ID: (none) Geometry type: linestring Geometry: RAW geometry: [9, 6310, 2648, 234, 181, 126, 147, 104, 427, 314, 137, 104, 41, 36, 4, 60, 18, 28, 66, 10, 54, 3, 232, 121, 178, 125, 32, 0, 24, 16, 8, 20, 1, 26, 27, 34, 471, 410, 33, 36, 11, 28, 8, 30, 28, 10, 38, 2, 56, 13, 590, 267, 30, 4, 46, 12, 12, 26, 1, 36, 217, 230] Linestring count: 30 Linestring points: (3155, 1324), (3064, 1387), (2990, 1439), (2776, 1596), (2707, 1648), (2686, 1666), (2688, 1696), (2697, 1710), (2730, 1715), (2757, 1713), (2873, 1652), (2962, 1589), (2978, 1589), (2990, 1597), (2994, 1607), (2993, 1620), (2979, 1637), (2743, 1842), (2726, 1860), (2720, 1874), (2724, 1889), (2738, 1894), (2757, 1895), (2785, 1888), (3080, 1754), (3095, 1756), (3118, 1762), (3124, 1775), (3123, 1793), (3014, 1908), End of Linestring Properties: name:latin: "Alameda Embaixador Sánchez Gavito" [string] maxspeed: "40" [string] Feature 1: ...

Hey Alessandro,

Regarding the Brazil map issue: I fixed the issue in the conversion tool and it should work with the Brazil map now. I tested it with https://download.geofabrik.de/south-america/brazil.html without any issues. There are some strange highway entries in the Brazil map file, with a large distance between two consecutive nodes (see for example this one https://www.openstreetmap.org/way/298746992#map=9/-18.8350/-55.8435 between Corumba and Coxim). The map format would require additional node interpolation in the conversion tool to display these proposed roads, but I don't have the time to implement that currently. So far, this only applies to very few roads and until now only occurred in Brazil. The affected roads will not render, which is something to keep in mind.

Regarding the SD-Card issue: According to the datasheet of your LilyGo T-Display-S3-Pro, it uses two separate SPI busses for the SD-Card and the display, so two active chip-selects are probably not the issue. You could try a different SD-Card from a different manufacturer (mine is from Samsung) and make sure that it is formatted as FAT32. Otherwise, I would need instructions of how to reproduce the issue on an ESP32C3 board to help you.

Regarding the map format: I'm not sure what you are asking for, but I'll try my best to give you some relevant information. My tile format only stores relative coordinates of each node. Nodes are grouped together by zero separators to make up roads. There are no fancy geometries, street names, speed limit etc... stored. In a simplified version, the data for a tile may look like this:

i  0  1  2  3  4  5  6
-----------------------
x 10 30 10  0  6  8  0
y -5 33 13  0 -2  5  0

where i is the coordinate index and x and y are mercator coordinates relative to the lower left corner of the tile. As you can see, at i==3 and i==6, both coordinates are 0, which indicates the end of a road. Hence, the example tile consists of two roads, the first one has 3 nodes, the second one has 2 nodes. The map is created in such a way, that it is not possible that both coordinates can be negative to reserve space for meta data for each road. That makes space for metadata very restricted (effectively 2x15 bit).

At the beginning of the bin file, there is a header containing some global metadata about the map. With that metadata and any lat/lon coordinate, you can directly calculate the tile index, the lower left corner of the tile and the memory address at which to find a pointer to the tile data (this is basically the index to find the right tile given lat/lon coordinates). Then, simply calculate the difference between the current tile data pointer and the next tile data pointer to get the tile data size and copy the respective memory portion in the render buffer.

I hope this helps

Hi LS, Thanks for the help and useful information. I used your idea of pointers. I packed my custom tilemaker .pbf on a bin file with a header and an index. With TileKey, I calculate the index position to get the data offset and length, then I read the data. I did it today with a GPT hand :) Generating the file took em 15 minutes for the entire Brazil. The best part is that the time to fetch a tile lowered from 5 to 16 times when compared to mbtiles/sqlite3, depending of size of the map.

https://github.com/lspr98/bike-computer-32/assets/20618327/6a2e1b18-99d9-4e0e-8685-21b73123e368

struct TileKey {
    int zoom, x, y;
    TileKey(int zoom, int x, int y) : zoom(zoom), x(x), y(y) {}
    bool operator==(const TileKey& other) const {
        return zoom == other.zoom && x == other.x && y == other.y;
    }
    struct Hash {
        size_t operator()(const TileKey& k) const {
            return std::hash<int>()(k.zoom) ^ (std::hash<int>()(k.x) << 1) ^ (std::hash<int>()(k.y) << 2);
        }
    };
};

struct TileData {
    std::vector<char> data;
    uint32_t offset;
    uint32_t length;
};

Hey Alessandro,

I'm glad that it could help you improve the loading times for your map format. If that resolves your issue, please feel free to close the issue :)

lspr98 / bike-computer-32

Cannot render map with mockPosProvider to debug (still no GNSS) #4