Running on Embedded Systems

[mitunchidamparam]

Hello, I am not able to parse files longer that 10 lines on my microcontroller. The microcontroller I am using is PIC32MZ. It has Program size meomry: 2048 & SRAM:512. When I have a large file program is not able to give any results. Can anyone please give suggestions on how I can proceed?

Thank you.

Regards, Mitun

[pt300]

What do you mean by file? A JSON string? How long are your 10 lines in bytes? Mind posting how you actually do the parsing and how does your data look like?

[mitunchidamparam]

The JSON string is contained inside a text file. Here is an example of what the file lookslike : { "?xml": { "@version": "1.0", "@encoding": "UTF-8" }, "Display": { "@Name": "TFT7", "@Size": "800,480", "@Ressource": "", "@DisplayOrientation": "auto", "Orderinformation": { "@OrderNr": "", "@PageNr": "", "@DeviceNr": "", "@ConfigDate": "" }, } }

I am not sure if I can get the size of the lines in bytes, I will try.

[pt300]

I still need to know how you are parsing that. If you could share parts of your code where you do that it would be great. Otherwise I can just say that the JSON you posted is invalid.

[mitunchidamparam]

To make it clear, if I use a file with few lines then it works. I have problems only with bigger files. Here is my implemetation. ` case File_Read: { int i; int r; jsmn_parser p; jsmntok_t t[128]; / We expect no more than 128 tokens / char str[80];

    int status;
    long fileSize;

    char    *buffer;
    long    numbytes;

    fileSize = SYS_FS_FileSize(appToneTextfileSdcardData.fileHandle);
        if(fileSize != -1)
    {
        // Success
   // sprintf(str,"Size of File  %s\n",fileSize);
//    appToneTextfileSdcardData.res = SYS_FS_FilePrintf(appToneTextfileSdcardData.fileHandle,"File Size : %ld \n",fileSize);        
    }

    else {
            BSP_LED_2On();
        }

numbytes = fileSize;

buffer = (char*)calloc(numbytes, sizeof(char));

    /* memory error */
if (buffer == NULL) {
    BSP_LED_1On();
}

char buf[20];
buffer = (char*)calloc(numbytes, sizeof(char));
size_t nbytes;
size_t bytes_read;

nbytes = sizeof(buffer);

bytes_read = SYS_FS_FileRead( appToneTextfileSdcardData.fileHandle, buffer, numbytes ); 
/*sprintf(str,"Bytes read : %c \n", buf);
appToneTextfileSdcardData.res = SYS_FS_FilePrintf(appToneTextfileSdcardData.fileHandle,str);*/

// SYS_FS_FileStringPut ( appToneTextfileSdcardData.fileHandle, buf); // appToneTextfileSdcardData.res = SYS_FS_FileStringPut ( appToneTextfileSdcardData.fileHandle2, buffer);
if(appToneTextfileSdcardData.res != SYS_FS_RES_SUCCESS) { BSP_LED_3On(); }

    jsmn_init(&p);

    r = jsmn_parse(&p, buffer, strlen(buffer), t, sizeof(t)/sizeof(t[0]));

if (r < 0) {
    appToneTextfileSdcardData.res = SYS_FS_FilePrintf(appToneTextfileSdcardData.fileHandle2,"Failed to parse JSON: %d\n",r);
    printf("Failed to parse JSON: %d\n", r);
    return 1;
}

/* Loop over all keys of the root object */
for (i = 1; i < r; i++) {
    if (jsoneq(buffer, &t[i], "?xml") == 0) {
        /* We may use strndup() to fetch string value */
        printf("?xml: %.*s\n", t[i+1].end-t[i+1].start,buffer + t[i+1].start);
        sprintf(str,"?xml: %.*s\n", t[i + 1].end - t[i + 1].start,buffer + t[i + 1].start);
        appToneTextfileSdcardData.res = SYS_FS_FilePrintf(appToneTextfileSdcardData.fileHandle2,"%s\n",str);
        i++;
    } else if (jsoneq(buffer, &t[i], "@version") == 0) {
        /* We may additionally check if the value is either "true" or "false" */
        printf("@version: %.*s\n", t[i+1].end-t[i+1].start, buffer + t[i+1].start);
        sprintf(str,"@version: %.*s\n", t[i+1].end-t[i+1].start, buffer + t[i+1].start);
        appToneTextfileSdcardData.res = SYS_FS_FilePrintf(appToneTextfileSdcardData.fileHandle2,"%s\n",str);

        i++;
    } else if (jsoneq(buffer, &t[i], "Display") == 0) {
        /* We may want to do strtol() here to get numeric value */
        printf("Display: %.*s\n", t[i+1].end-t[i+1].start, buffer + t[i+1].start);
        sprintf(str,"Display: %.*s\n", t[i+1].end-t[i+1].start, buffer + t[i+1].start);
        appToneTextfileSdcardData.res = SYS_FS_FilePrintf(appToneTextfileSdcardData.fileHandle2,"%s\n",str);

        i++;
    } else if (jsoneq(buffer, &t[i], "Firmwareinformations") == 0) {
        /* We may want to do strtol() here to get numeric value */
        printf("Firmwareinformations: %.*s\n", t[i+1].end-t[i+1].start, buffer + t[i+1].start);
        sprintf(str,"Firmwareinformations: %.*s\n", t[i+1].end-t[i+1].start, buffer + t[i+1].start);
        appToneTextfileSdcardData.res = SYS_FS_FilePrintf(appToneTextfileSdcardData.fileHandle2,"%s\n",str);

        i++;
    } else if (jsoneq(buffer, &t[i], "@Name") == 0) {
        /* We may want to do strtol() here to get numeric value */
        printf("@Name: %.*s\n", t[i+1].end-t[i+1].start, buffer + t[i+1].start);
        sprintf(str,"@Name: %.*s\n", t[i+1].end-t[i+1].start, buffer + t[i+1].start);
        appToneTextfileSdcardData.res = SYS_FS_FilePrintf(appToneTextfileSdcardData.fileHandle2,"%s\n",str);

        i++;
    } else if (jsoneq(buffer, &t[i], "@Size") == 0) {
        /* We may want to do strtol() here to get numeric value */
        printf("@Size: %.*s\n", t[i+1].end-t[i+1].start, buffer + t[i+1].start);
        sprintf(str,"@Size: %.*s\n", t[i+1].end-t[i+1].start, buffer + t[i+1].start);
        appToneTextfileSdcardData.res = SYS_FS_FilePrintf(appToneTextfileSdcardData.fileHandle2,"%s\n",str);

        i++;    
    } else if (jsoneq(buffer, &t[i], "@Ressource") == 0) {
        /* We may want to do strtol() here to get numeric value */
        printf("@Ressource: %.*s\n", t[i+1].end-t[i+1].start, buffer + t[i+1].start);
        sprintf(str,"@Ressource: %.*s\n", t[i+1].end-t[i+1].start, buffer + t[i+1].start);
        appToneTextfileSdcardData.res = SYS_FS_FilePrintf(appToneTextfileSdcardData.fileHandle2,"%s\n",str);

        i++;   
    } else if (jsoneq(buffer, &t[i], "@DisplayOrientation") == 0) {
        /* We may want to do strtol() here to get numeric value */
        printf("@DisplayOrientation: %.*s\n", t[i+1].end-t[i+1].start, buffer + t[i+1].start);
        sprintf(str,"@DisplayOrientation: %.*s\n", t[i+1].end-t[i+1].start, buffer + t[i+1].start);
        appToneTextfileSdcardData.res = SYS_FS_FilePrintf(appToneTextfileSdcardData.fileHandle2,"%s\n",str);

        i++;   

    } else if (jsoneq(buffer, &t[i], "@OrderNr") == 0) {
        /* We may want to do strtol() here to get numeric value */
        printf("@OrderNr: %.*s\n", t[i+1].end-t[i+1].start, buffer + t[i+1].start);
        sprintf(str,"@OrderNr: %.*s\n", t[i+1].end-t[i+1].start, buffer + t[i+1].start);
        appToneTextfileSdcardData.res = SYS_FS_FilePrintf(appToneTextfileSdcardData.fileHandle2,"%s\n",str);

        i++;       
    } else if (jsoneq(buffer, &t[i], "@PageNr") == 0) {
        /* We may want to do strtol() here to get numeric value */
        printf("@PageNr: %.*s\n", t[i+1].end-t[i+1].start, buffer + t[i+1].start);
        sprintf(str,"@PageNr: %.*s\n", t[i+1].end-t[i+1].start, buffer + t[i+1].start);
        appToneTextfileSdcardData.res = SYS_FS_FilePrintf(appToneTextfileSdcardData.fileHandle2,"%s\n",str);

        i++;       
    } else {
        printf("Unexpected key: %.*s\n", t[i].end-t[i].start, buffer + t[i].start);
        sprintf(str,"Unexpected key: %.*s\n", t[i].end-t[i].start, buffer + t[i].start);
        appToneTextfileSdcardData.res = SYS_FS_FilePrintf(appToneTextfileSdcardData.fileHandle2,"%s\n",str);

    }

}    

BSP_LED_1On();  

free(buffer);
appToneTextfileSdcardData.state = IDLE;    
SYS_FS_FileClose(appToneTextfileSdcardData.fileHandle); 
SYS_FS_FileClose(appToneTextfileSdcardData.fileHandle2); 
break;
}`

[pt300]

And now tell me what exactly happens. Do you get an error from jsmn (If so then which one)? Is the data that it returns broke(how exactly)? Or are there other symptoms? I need more details.

[mitunchidamparam]

Hello guys, we can close this thread. I solved the issue. The parser was not working in my microcontroller for large files since, I was using sprintf, without specifying the string size according to requirements. Thank you for your support