Help wanted

[jgauchia]

Hi @aresta , If you need some help in your project, feel free to ask me... (Maybe can I help you)

Good job!

[Matko890]

Hey , could I get some help ? so I managed to get the map files " 2_9, 2_10, 2_11, 2_12, 2_13, 2_14, 3_9, 3_10, 3_11, 3_12, 3_13, 3_14, 4_9, 4_10, 4_11, 4_12, 4_13, 4_14, 5_9, 5_10, 5_11, 5_12, 5_13, 5_14, 6_9, 6_10, 6_11, 6_12, 6_13, 6_14" but when I put them on the sd card in /"mymap"/ folder on the tft it says "map file not found in folder: /mymap/" I tried reorienting the files so each one has its own folder and I tried a couple of other things but nope it does not work, I tried multiple sd cards and still nothing, do I have to edit something in the , add someother file to the sd card , reorient them diffrently? ps: this is is my first "ask for help on github" I dont know if I included enough information if not just ask I apologise for my errors in english it is not my native language thank you for any kind of help .

[jgauchia]

Hi @Matko890

Have you created the maps with the latest version of OSM_Extract ?

The folder structure has changed.

Inside these folders there are some files like 0_1.fmp, etc...

Latest version: folder_name = f"{MAP_FOLDER}/{folder_name_x:+04d}{folder_name_y:+04d}" file_name = f"{folder_name}/{block_x}_{block_y}"

Previous version: file_name = f"{MAP_FOLDER}/{folder_name_x}_{folder_name_y}/{block_x}_{block_y}"

I believe, @Matko890 that you have the maps generated for an earlier version of the current GPS code of this project, or you should regenerate the files with the latest version of OSM_Extract.

Perhaps you copied the files without maintaining the folder structure generated by OSM_Extract?

What I'm saying, is it correct, @aresta ?

[Matko890]

Thank you for your response, But it looks like I am using the newest verison (copied from "extract_features.py" " # folder_name numbers: sign forced (+,-) and 4 chars length, left padded with zeros. e.g: '-009+081' folder_name = f"{MAP_FOLDER}/{folder_name_x:+04d}{folder_name_y:+04d}" file_name = f"{folder_name}/{blockx}{block_y}" os.makedirs( folder_name, exist_ok=True)

print(f"File: {file_name}.fmp")" ... )

the problem I think it is is that if I did not put the files in the right order on the sd card, like in some directory or something and another problem could be that while I make the fmp files with this function " C:/Users/pc/Desktop/OSM_Extract-main/scripts/extract_features.py 15.9749 45.7615 16.1267 45.8963 C:/Users/pc/Desktop/ " I get a error that goes like this " Step 1/5 reading lines files Traceback (most recent call last): File "C:\Users\pc\Desktop\OSM_Extract-main\scripts\extract_features.py", line 33, in lines = json.load(open(LINES_INPUT_FILE, "r")) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "E:\anaconda\Lib\json__init__.py", line 293, in load return loads(fp.read(), ^^^^^^^^^ File "E:\anaconda\Lib\encodings\cp1250.py", line 23, in decode return codecs.charmap_decode(input,self.errors,decoding_table)[0] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ UnicodeDecodeError: 'charmap' codec can't decode byte 0x88 in position 11381: character maps to " so becuse of that I changed this in "extract_features.py" "print(" Step 1/5 reading lines files") lines = json.load(open(LINES_INPUT_FILE, "r", encoding="utf-8")) print(" Step 2/5 reading polygons files") polygons = json.load( open( POLYGONS_INPUT_FILE, "r", encoding="utf-8"))" I added this ", encoding="utf-8"" and then the code menaged to make the files in a directory called +027+087 . ( im creating files in git bash )

[jgauchia]

Do you mean, that if you add the encoding, files are generated in correct folders?

[Matko890]

Do you mean, that if you add the encoding, files are generated in correct folders?

Yes. When I add that line files are generated in +027+087 folder all the files from 2_9.fmp to 6_9.fmp and I get a test images folder and the test images seem correct. Here are the files if you want to test them "https://drive.google.com/drive/folders/1tpAyd4E28BMoTN-uULDs3W6jS_m5oAlb?usp=drive_link" and could you tell me how should I put the files in directories I doubt that is wrong but just to check is it like SdCard>mymap>"files" or something like SdCard>mymap>+027+087>"files" btw thank you soo much for trying to help me :) .

[jgauchia]

The format is this: SdCard>mymap>+027+087>"files"

[Matko890]

The format is this: SdCard>mymap>+027+087>"files"

Have you tried testing the files if they are good, I did put the files like this and it did not work. And is that type of encoding good?

[jgauchia]

No, I don't test It (sorry the LCD of my project is broken and i'm waiting for a New one). What is the output in serial monitor.? The file and folder for your position is correct? I think that in serial monitor maybe show It. Function get_map_blocks from maps.cpp returns the folder and file expected to be loaded

    log_d("load from disk (%i, %i) %i", block_min_x, block_min_y, millis());

[Matko890]

The format is this: SdCard>mymap>+027+087>"files"

Have you tried testing the files if they are good, I did put the files like this and it did not work. And is that type of encoding good?

Hey I have some private things to do so somewhere in saturday I will be back. Thank you for all the help you provided me.

[Matko890]

File and folder position is correct. This is all I get in the serial monitor when I run it "rst:0x10 (RTCWDT_RTC_RESET),boot:0x37 (SPI_FAST_FLASH_BOOT) configsip: 0, SPIWP:0xee clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00 mode:DIO, clock div:1 load:0x3fff0030,len:1344 load:0x40078000,len:13964 load:0x40080400,len:3600 entry 0x400805f0 SD Card Type: SDHC SD Card Size: 14910MB initialisation done." I get nothing else about " log_d("load from disk (%i, %i) %i", block_min_x, block_min_y, millis());" .... and this is my pinout/specs "

define SCREEN_WIDTH 240 // tft.width()

define SCREEN_HEIGHT 240 // tft.height()

define PIXEL_SIZE_DEF 2 // in meters

define MAX_ZOOM 7

extern int zoom_level;

// Buttons GPIO nums

define UP_BUTTON 32

define DOWN_BUTTON 33

define LEFT_BUTTON 25

define RIGHT_BUTTON 14

define SELECT_BUTTON 02

define TFT_OFF_BUTTON 16

define TFT_BLK_PIN 02

define TFT_CS 15

define MENU_BUTTON 17

define SD_MISO_PIN 19

define SD_CS_PIN 13

define SD_SCK_PIN 18

define SD_MOSI_PIN 23

define GPS_RX 27

define GPS_TX 26

// #define GPS_CE 14" btw I just want to check the size if its roughly alright so I have a map of "Zagreb" 10 mb of fmp files around 460 sq km with "conf" configurations Not changed.

[jgauchia]

File and folder position is correct. This is all I get in the serial monitor when I run it "rst:0x10 (RTCWDT_RTC_RESET),boot:0x37 (SPI_FAST_FLASH_BOOT) configsip: 0, SPIWP:0xee clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00 mode:DIO, clock div:1 load:0x3fff0030,len:1344 load:0x40078000,len:13964 load:0x40080400,len:3600 entry 0x400805f0 SD Card Type: SDHC SD Card Size: 14910MB initialisation done." I get nothing else about " log_d("load from disk (%i, %i) %i", block_min_x, block_min_y, millis());" .... and this is my pinout/specs " #define SCREEN_WIDTH 240 // tft.width() #define SCREEN_HEIGHT 240 // tft.height() #define PIXEL_SIZE_DEF 2 // in meters #define MAX_ZOOM 7 extern int zoom_level;

// Buttons GPIO nums #define UP_BUTTON 32 #define DOWN_BUTTON 33 #define LEFT_BUTTON 25 #define RIGHT_BUTTON 14 #define SELECT_BUTTON 02 #define TFT_OFF_BUTTON 16 #define TFT_BLK_PIN 02 #define TFT_CS 15 #define MENU_BUTTON 17

define SD_MISO_PIN 19 #define SD_CS_PIN 13 #define SD_SCK_PIN 18 #define SD_MOSI_PIN 23

define GPS_RX 27 #define GPS_TX 26 // #define GPS_CE 14" btw I just want to check the size if its roughly alright so I have a map of "Zagreb" 10 mb of fmp files around 460 sq km with "conf" configurations Not changed.

Do you have GPS fixed?

[Matko890]

Do you have GPS fixed?

No im working on it, I don't know whats wrong ? I still get this error "Map file not found in folder" and all the problems/specs listed above .

[Matko890]

okey so this is what I get in the serial monitor ( i finally managed to see the logging statments ) "09:50:11.865 -> [ 750][E][esp32-hal-cpu.c:110] addApbChangeCallback(): duplicate func=0x4200e050 arg=0x3fc94800 09:50:11.901 -> SD Card Type: SDHC 09:50:11.901 -> SD Card Size: 14910MB 09:50:11.901 -> initialisation done. 09:50:11.901 -> [ 767][D][maps.cpp:246] get_map_blocks(): get_map_blocks 767 09:50:11.901 -> [ 768][D][maps.cpp:266] get_map_blocks(): load from disk (233472, 5070848) 768 09:50:11.901 -> [ 786][E][maps.cpp:278] get_map_blocks(): File not found on SD card: /mymap/+003+077/9_6.fmp 09:50:11.933 -> [ 787][D][maps.cpp:266] get_map_blocks(): load from disk (233472, 5074944) 787 09:50:11.933 -> [ 808][E][maps.cpp:278] get_map_blocks(): File not found on SD card: /mymap/+003+077/9_7.fmp 09:50:11.933 -> [ 809][D][maps.cpp:266] get_map_blocks(): load from disk (233472, 5074944) 809 09:50:11.967 -> [ 830][E][maps.cpp:278] get_map_blocks(): File not found on SD card: /mymap/+003+077/9_7.fmp 09:50:11.967 -> [ 831][D][maps.cpp:266] get_map_blocks(): load from disk (233472, 5070848) 831 09:50:11.967 -> [ 852][E][maps.cpp:278] get_map_blocks(): File not found on SD card: /mymap/+003+077/9_6.fmp 09:50:12.002 -> [ 853][D][maps.cpp:302] get_map_blocks(): memCache size: 0 853 09:50:12.048 -> [ 891][D][draw.cpp:62] draw(): Draw start 891 09:50:12.048 -> [ 891][D][draw.cpp:113] draw(): Total 0 ms 09:50:12.048 -> [ 891][D][draw.cpp:122] draw(): Draw done! 891" now this is weird becuse this is not my folder name its SDcard>+027+028> "and then these files ... from " 2_9 to 6_14

[jgauchia]

okey so this is what I get in the serial monitor ( i finally managed to see the logging statments ) "09:50:11.865 -> [ 750][E][esp32-hal-cpu.c:110] addApbChangeCallback(): duplicate func=0x4200e050 arg=0x3fc94800 09:50:11.901 -> SD Card Type: SDHC 09:50:11.901 -> SD Card Size: 14910MB 09:50:11.901 -> initialisation done. 09:50:11.901 -> [ 767][D][maps.cpp:246] get_map_blocks(): get_map_blocks 767 09:50:11.901 -> [ 768][D][maps.cpp:266] get_map_blocks(): load from disk (233472, 5070848) 768 09:50:11.901 -> [ 786][E][maps.cpp:278] get_map_blocks(): File not found on SD card: /mymap/+003+077/9_6.fmp 09:50:11.933 -> [ 787][D][maps.cpp:266] get_map_blocks(): load from disk (233472, 5074944) 787 09:50:11.933 -> [ 808][E][maps.cpp:278] get_map_blocks(): File not found on SD card: /mymap/+003+077/9_7.fmp 09:50:11.933 -> [ 809][D][maps.cpp:266] get_map_blocks(): load from disk (233472, 5074944) 809 09:50:11.967 -> [ 830][E][maps.cpp:278] get_map_blocks(): File not found on SD card: /mymap/+003+077/9_7.fmp 09:50:11.967 -> [ 831][D][maps.cpp:266] get_map_blocks(): load from disk (233472, 5070848) 831 09:50:11.967 -> [ 852][E][maps.cpp:278] get_map_blocks(): File not found on SD card: /mymap/+003+077/9_6.fmp 09:50:12.002 -> [ 853][D][maps.cpp:302] get_map_blocks(): memCache size: 0 853 09:50:12.048 -> [ 891][D][draw.cpp:62] draw(): Draw start 891 09:50:12.048 -> [ 891][D][draw.cpp:113] draw(): Total 0 ms 09:50:12.048 -> [ 891][D][draw.cpp:122] draw(): Draw done! 891" now this is weird becuse this is not my folder name its SDcard>+027+028> "and then these files ... from " 2_9 to 6_14

Do you generate files correctly for your location?

[Matko890]

Do you generate files correctly for your location?

yes you can see all the files I provided you with before and those files are on the sd card 14:37:01.685 -> [ 750][E][esp32-hal-cpu.c:110] addApbChangeCallback(): duplicate func=0x4200dfb8 arg=0x3fc94800 14:37:01.685 -> SD Card Type: SDHC 14:37:01.685 -> SD Card Size: 14910MB 14:37:01.685 -> initialisation done. 14:37:01.685 -> [ 768][D][maps.cpp:246] get_map_blocks(): get_map_blocks 768 14:37:01.732 -> [ 768][D][maps.cpp:266] get_map_blocks(): load from disk (233472, 5070848) 768 14:37:01.732 -> [ 769][D][maps.cpp:117] read_map_block(): read_map_block: /mymap/+003+077/9_6 14:37:01.732 -> [ 793][E][vfs_api.cpp:105] open(): /sd/mymap/+003+077/9_6.fmp does not exist, no permits for creation

[jgauchia]

Do you generate files correctly for your location?

yes you can see all the files I provided you with before and those files are on the sd card 14:37:01.685 -> [ 750][E][esp32-hal-cpu.c:110] addApbChangeCallback(): duplicate func=0x4200dfb8 arg=0x3fc94800 14:37:01.685 -> SD Card Type: SDHC 14:37:01.685 -> SD Card Size: 14910MB 14:37:01.685 -> initialisation done. 14:37:01.685 -> [ 768][D][maps.cpp:246] get_map_blocks(): get_map_blocks 768 14:37:01.732 -> [ 768][D][maps.cpp:266] get_map_blocks(): load from disk (233472, 5070848) 768 14:37:01.732 -> [ 769][D][maps.cpp:117] read_map_block(): read_map_block: /mymap/+003+077/9_6 14:37:01.732 -> [ 793][E][vfs_api.cpp:105] open(): /sd/mymap/+003+077/9_6.fmp does not exist, no permits for creation

I think that position is fixed or not adquired by GPS, seems to be 50.7084 lat 2.33472 lon.

[Matko890]

alright so yea it was lon and lat and I was changing it until I got desired values in "env.h" and then I get this in logs "17:25:32.361 -> [ 768][D][maps.cpp:246] get_map_blocks(): get_map_blocks 768 17:25:32.361 -> [ 768][D][maps.cpp:266] get_map_blocks(): load from disk (1810432, 5746688) 768 17:25:32.361 -> [ 770][D][maps.cpp:273] get_map_blocks(): block_min_x: 1810432, block_min_y: 5746688 17:25:32.361 -> [ 777][D][maps.cpp:274] get_map_blocks(): folder_name_x: 27, folder_name_y: 87 17:25:32.393 -> [ 784][D][maps.cpp:275] get_map_blocks(): block_x: 10, block_y: 11 17:25:32.393 -> [ 791][D][maps.cpp:282] get_map_blocks(): Folder being read: +027+087 17:25:32.393 -> [ 797][D][maps.cpp:283] get_map_blocks(): File being read: /mymap/+027+087/10_11 17:25:32.393 -> [ 804][D][maps.cpp:117] read_map_block(): read_map_block: /mymap/+027+087/10_11 17:25:32.428 -> [ 826][E][vfs_api.cpp:105] open(): /sd/mymap/+027+087/10_11.fmp does not exist, no permits for creation" (I renamed a fmp file into "10.11.fmp" but nothing it still wont load it

[jgauchia]

alright so yea it was lon and lat and I was changing it until I got desired values in "env.h" and then I get this in logs "17:25:32.361 -> [ 768][D][maps.cpp:246] get_map_blocks(): get_map_blocks 768 17:25:32.361 -> [ 768][D][maps.cpp:266] get_map_blocks(): load from disk (1810432, 5746688) 768 17:25:32.361 -> [ 770][D][maps.cpp:273] get_map_blocks(): block_min_x: 1810432, block_min_y: 5746688 17:25:32.361 -> [ 777][D][maps.cpp:274] get_map_blocks(): folder_name_x: 27, folder_name_y: 87 17:25:32.393 -> [ 784][D][maps.cpp:275] get_map_blocks(): block_x: 10, block_y: 11 17:25:32.393 -> [ 791][D][maps.cpp:282] get_map_blocks(): Folder being read: +027+087 17:25:32.393 -> [ 797][D][maps.cpp:283] get_map_blocks(): File being read: /mymap/+027+087/10_11 17:25:32.393 -> [ 804][D][maps.cpp:117] read_map_block(): read_map_block: /mymap/+027+087/10_11 17:25:32.428 -> [ 826][E][vfs_api.cpp:105] open(): /sd/mymap/+027+087/10_11.fmp does not exist, no permits for creation" (I renamed a fmp file into "10.11.fmp" but nothing it still wont load it

10.11.fmp or 10_11.fmp ?

[Matko890]

I typed it wrong on the pc... yes I wrote "10_11.fmp" and it still is not working so 2 things I think that could be wrong how I initialize the sdcard "#include

include

include

include

include "graphics.h"

include "maps.h"

include "../lib/conf.h"

SPIClass SD_SPI(VSPI); // Define HSPI for SD card

const String base_folder = "/mymap/"; // TODO: folder selection

bool init_sd_card() { SD_SPI.begin(SD_SCK_PIN, SD_MISO_PIN, SD_MOSI_PIN, SD_CS_PIN); if (!SD.begin(SD_CS_PIN, SD_SPI)) { Serial.println("Card Mount Failed"); return false; } uint8_t cardType = SD.cardType();

if (cardType == CARD_NONE) {
    Serial.println("No SD card attached");
    header_msg("No SD card attached");
    return false;
}

Serial.print("SD Card Type: ");
if (cardType == CARD_MMC) {
    Serial.println("MMC");
} else if (cardType == CARD_SD) {
    Serial.println("SDSC");
} else if (cardType == CARD_SDHC) {
    Serial.println("SDHC");
} else {
    Serial.println("UNKNOWN");
}

uint64_t cardSize = SD.cardSize() / (1024 * 1024);
Serial.printf("SD Card Size: %lluMB\n", cardSize);   
Serial.println("initialisation done.");
return true;

}" and how I creacted the files in the first place using the polygons = json.load( open( POLYGONS_INPUT_FILE, "r", encoding="utf-8")) and adding this "encoding="utf-8"" into it

[jgauchia]

I typed it wrong on the pc... yes I wrote "10_11.fmp" and it still is not working so 2 things I think that could be wrong how I initialize the sdcard "#include

include

include

include

include "graphics.h"

include "maps.h"

include "../lib/conf.h"

SPIClass SD_SPI(VSPI); // Define HSPI for SD card

const String base_folder = "/mymap/"; // TODO: folder selection

bool init_sd_card() { SD_SPI.begin(SD_SCK_PIN, SD_MISO_PIN, SD_MOSI_PIN, SD_CS_PIN); if (!SD.begin(SD_CS_PIN, SD_SPI)) { Serial.println("Card Mount Failed"); return false; } uint8_t cardType = SD.cardType();

if (cardType == CARD_NONE) {
    Serial.println("No SD card attached");
    header_msg("No SD card attached");
    return false;
}

Serial.print("SD Card Type: ");
if (cardType == CARD_MMC) {
    Serial.println("MMC");
} else if (cardType == CARD_SD) {
    Serial.println("SDSC");
} else if (cardType == CARD_SDHC) {
    Serial.println("SDHC");
} else {
    Serial.println("UNKNOWN");
}

uint64_t cardSize = SD.cardSize() / (1024 * 1024);
Serial.printf("SD Card Size: %lluMB\n", cardSize);   
Serial.println("initialisation done.");
return true;

}" and how I creacted the files in the first place using the polygons = json.load( open( POLYGONS_INPUT_FILE, "r", encoding="utf-8")) and adding this "encoding="utf-8"" into it

I think that you SD card is initalized correctly

[aresta]

Sorry, I was disconnected of it for some weeks. Have you been able to set it up? We can check all the steps. The folder name in the SD card at the moment is hard coded, so it should look:

We can go through the steps with the scripts to check where is the problem.