Running with uncompressed .data section

[Gator96100]

On my quest to get hardware debugging working on the proxmark3, I ran into the problem of fullimage.elf having a compressed .data section and in the process have all meta information removed.

My idea was to use fullimage.stage1.elf for hardware debugging as it contains all meta information that is required for debugging and the 2.5KB larger image is fine for debugging purposes. The problem is that fullimage.stage1.elf will not work on the proxmark3 and I am unsure if it is possible to run with uncompressed .data sections.

The initial commit Compress the .data section as well make it look like there is little to change to get a fullimage with uncompressed .data section running again, but I am not familiar how the compression is integrated in the proxmark3 firmware and what did change since the commit that could make running with uncompressed .data section impossible.

If it is no longer possible to have uncompressed .data sections, I will have to look into getting meta information back into the fullimage.elf file. Probably with a small tool that will use fullimage.stage1.elf, compress the .data section, updates the meta information and outputs fullimage.elf.

[iceman1001]

@slurdge might be better answering this. It's sadly out of my knowledge.
With that said, it looks like you could just use data instead of compress_data ...

[iceman1001]

Or of course @doegox and @pwpiwi ....

[slurdge]

@Gator96100 it should be possible to run without compressed .data section. Decompression is here: https://github.com/RfidResearchGroup/proxmark3/blob/d500d1d6e9da3df5e00801089869c8f5b985d5eb/armsrc/start.c#L51 Compression & stripping is here: https://github.com/RfidResearchGroup/proxmark3/blob/master/armsrc/Makefile#L203

I believe if you change it to not call FPGA_COMPRESSOR and not decompress it, and also maybe removing all the fiddling with the .o files (basically keeping the .elf as far as possible) it would work

[Gator96100]

@slurdge Unfortunately simply not calling uncompress_data_section(); and not compressing fullimage.stage1.elf does not work.

[slurdge]

Hmm, I'm not sure of what's going wrong. I'm not a specialist of these part. Maybe common_area needs to be pointed to the correct location too.

[doegox]

I tried with the following patch, it seems to work fine.

commit 4a5feb7e3a15a3df376f17a8ed199dd8c04b0495
Author: Philippe Teuwen <phil@teuwen.org>
Date:   Mon Dec 28 12:14:39 2020 +0100

    test without data compress

diff --git a/armsrc/appmain.c b/armsrc/appmain.c
index fe8566363..f6edd1dc6 100644
--- a/armsrc/appmain.c
+++ b/armsrc/appmain.c
@@ -257,6 +257,7 @@ void ReadMem(int addr) {
 /* osimage version information is linked in, cf commonutil.h */
 /* bootrom version information is pointed to from _bootphase1_version_pointer */
 extern char *_bootphase1_version_pointer, _flash_start, _flash_end, __data_src_start__;
+extern char *_bootrom_end, _bootrom_start, __os_size__;
 static void SendVersion(void) {
     char temp[PM3_CMD_DATA_SIZE - 12]; /* Limited data payload in USB packets */
     char VersionString[PM3_CMD_DATA_SIZE - 12] = { '\0' };
@@ -296,8 +297,6 @@ static void SendVersion(void) {
         }
     }
     // Send Chip ID and used flash memory
-    uint32_t text_and_rodata_section_size = (uint32_t)&__data_src_start__ - (uint32_t)&_flash_start;
-    uint32_t compressed_data_section_size = common_area.arg1;

     struct p {
         uint32_t id;
@@ -308,7 +307,7 @@ static void SendVersion(void) {

     struct p payload;
     payload.id = *(AT91C_DBGU_CIDR);
-    payload.section_size = text_and_rodata_section_size + compressed_data_section_size;
+    payload.section_size = (uint32_t)&_bootrom_end - (uint32_t)&_bootrom_start + (uint32_t)&__os_size__;
     payload.versionstr_len = strlen(VersionString) + 1;
     memcpy(payload.versionstr, VersionString, payload.versionstr_len);

diff --git a/armsrc/start.c b/armsrc/start.c
index 70eee5063..9ec0e7f40 100644
--- a/armsrc/start.c
+++ b/armsrc/start.c
@@ -14,28 +14,12 @@

 #include "proxmark3_arm.h"
 #include "appmain.h"
-#include "lz4.h"
 #include "BigBuf.h"
 #include "string.h"

 extern struct common_area common_area;
 extern char __data_src_start__, __data_start__, __data_end__, __bss_start__, __bss_end__;

-
-static void uncompress_data_section(void) {
-    int avail_in;
-    memcpy(&avail_in, &__data_src_start__, sizeof(int));
-    int avail_out = &__data_end__ - &__data_start__;  // uncompressed size. Correct.
-    // uncompress data segment to RAM
-    uintptr_t p = (uintptr_t)&__data_src_start__;
-    int res = LZ4_decompress_safe((char *)p + 4, &__data_start__, avail_in, avail_out);
-
-    if (res < 0)
-        return;
-    // save the size of the compressed data section
-    common_area.arg1 = avail_in;
-}
-
 void __attribute__((section(".startos"))) Vector(void);
 void Vector(void) {
     /* Stack should have been set up by the bootloader */
@@ -48,11 +32,15 @@ void Vector(void) {
     }
     common_area.flags.osimage_present = 1;

-    uncompress_data_section();
+    /* Set up data segment: Copy from flash to ram */
+    char *src = &__data_src_start__;
+    char *dst = &__data_start__;
+    char *end = &__data_end__;
+    while(dst < end) *dst++ = *src++;

     /* Set up (that is: clear) BSS. */
-    char *dst = &__bss_start__;
-    char *end = &__bss_end__;
+    dst = &__bss_start__;
+    end = &__bss_end__;
     while (dst < end) *dst++ = 0;

     AppMain();
diff --git a/pm3 b/pm3
index 0d3b9863a..893fde670 100755
--- a/pm3
+++ b/pm3
@@ -12,7 +12,7 @@ FINDBTDIRECT=true

 PM3PATH=$(dirname "$0")
 EVALENV=""
-FULLIMAGE="fullimage.elf"
+FULLIMAGE="fullimage.stage1.elf"
 BOOTIMAGE="bootrom.elf"
 # try pm3 dirs in current repo workdir
 if [ -d "$PM3PATH/client/" ]; then

Changes in appmain.c are cosmetic, just to report correct ROM usage. So main change is simply replacing decompression by copy, and flashing directly fullimage.stage1.elf on the Proxmark

[Gator96100]

Thank you very much. It does seem to work as intended. I got Segger Ozone working with your patch, now it’s time for some VSCode integration.

[doegox]

BTW the gain by compressing data seems very marginal

• uncompressed data: 8392 (fullimage.data.bin)
• compressed data: 6103 (fullimage.data.bin.z)

[slurdge]

2k is 2k my friend 😄

[iceman1001]

...if we get device debug then uncompressed sounds ok

[doegox]

... only if we get device debug under Linux too :zany_face: