jcmvbkbc / esp32-linux-build

xtensa linux build scripts for the esp32s3 and esp32
http://wiki.osll.ru/doku.php/etc:users:jcmvbkbc:linux-xtensa:esp32s3
68 stars 9 forks source link

Can't run this linux build on a ESP32S3 N16R8 #9

Open arashsaifi opened 2 weeks ago

arashsaifi commented 2 weeks ago

Hey so I made a sort of a diy esp32 board using this tutorial -> https://www.youtube.com/watch?v=tQ0dL_9M1wI&pp=ygULZGl5IGVzcCBcMzI%3D

I was successful in my attempt and I could see this as a COM serial port in my terminal.

I built the linux for it and flashed it (accordingly for the USB serial to work with some modification to a file in the etc partition)

The problem is that when I try to use the "idf.py monitor" command it gets stuck at the following line.

linux ptr = 0x42120000
vectors ptr = 0x4037c000

am I missing something or is this a bug which needs to be fixed

jcmvbkbc commented 2 weeks ago

am I missing something or is this a bug which needs to be fixed

I don't know. I don't have N16R8, and by default the build is configured for N8R8. I'm not sure what happens if you flash it into N16R8 without first reconfiguring it, I imagine that there may be issues with FLASH/PSRAM voltage.

arashsaifi commented 2 weeks ago

That's what I thought and then went ahead and tried it with N8R8 and had the same results. The output on your wiki shows that it boots to a console but mine is stucked at the first two lines.

On Wed, 12 Jun 2024, 10:20 am Max Filippov, @.***> wrote:

am I missing something or is this a bug which needs to be fixed

I don't know. I don't have N16R8, and by default the build is configured for N8R8. I'm not sure what happens if you flash it into N16R8 without first reconfiguring it, I imagine that there may be issues with FLASH/PSRAM voltage.

— Reply to this email directly, view it on GitHub https://github.com/jcmvbkbc/esp32-linux-build/issues/9#issuecomment-2162099760, or unsubscribe https://github.com/notifications/unsubscribe-auth/BH4ZI6WKXVHVL3YAMEC6233ZG7HQ3AVCNFSM6AAAAABJE6DBSGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCNRSGA4TSNZWGA . You are receiving this because you authored the thread.Message ID: @.***>

jcmvbkbc commented 2 weeks ago

Can you share the images that were flashed? Specifically build/build-buildroot-esp32s3/images/*

arashsaifi commented 2 weeks ago

Sure, currently I am outside once I reach home I'll upload them

On Wed, 12 Jun 2024, 2:50 pm Max Filippov, @.***> wrote:

Can you share the images that were flashed? Specifically build/build-buildroot-esp32s3/images/*

— Reply to this email directly, view it on GitHub https://github.com/jcmvbkbc/esp32-linux-build/issues/9#issuecomment-2162527592, or unsubscribe https://github.com/notifications/unsubscribe-auth/BH4ZI6V4YRTK5J5LOOPZSLDZHAHGXAVCNFSM6AAAAABJE6DBSGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCNRSGUZDONJZGI . You are receiving this because you authored the thread.Message ID: @.***>

arashsaifi commented 2 weeks ago

build.tar.gz These were the files generated by the build script

jcmvbkbc commented 2 weeks ago

These were the files generated by the build script I built the linux for it and flashed it (accordingly for the USB serial to work with some modification to a file in the etc partition)

There's no /etc/cmdline in the etc.jffs2 from that archive, so default console is on the UART. And if I flash the kernel and etc from the archive to a devkit board I get the expected kernel boot log. So I wonder what modifications exactly did you do?

arashsaifi commented 2 weeks ago

I might have done something wrong. Can you send me your image with the /etc/cmdline then I can test that out.

On Thu, 13 Jun 2024, 12:19 pm Max Filippov, @.***> wrote:

These were the files generated by the build script I built the linux for it and flashed it (accordingly for the USB serial to work with some modification to a file in the etc partition)

There's no /etc/cmdline in the etc.jffs2 from that archive, so default console is on the UART. And if I flash the kernel and etc from the archive to a devkit board I get the expected kernel boot log. So I wonder what modifications exactly did you do?

— Reply to this email directly, view it on GitHub https://github.com/jcmvbkbc/esp32-linux-build/issues/9#issuecomment-2164670633, or unsubscribe https://github.com/notifications/unsubscribe-auth/BH4ZI6VONUB7Z6SWDSBBS63ZHE6I7AVCNFSM6AAAAABJE6DBSGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCNRUGY3TANRTGM . You are receiving this because you authored the thread.Message ID: @.***>

jcmvbkbc commented 2 weeks ago

etc.jffs2.txt

Also while testing it I've noticed that the jffs2 parser has been ignoring the first file on the file system, which happened to be the cmdline. I've pushed a fix here: https://github.com/jcmvbkbc/esp-hosted/commit/ceab9c2cf41b2356c099bb04145ec9a90d772d8c I made sure that cmdline is not the first file in the attached image, but if you build your own etc.jffs2 with that file, please also update the bootloader.

arashsaifi commented 2 weeks ago

Okay, I will try it as soon as I reach home

On Thu, 13 Jun 2024, 2:24 pm Max Filippov, @.***> wrote:

etc.jffs2.txt https://github.com/user-attachments/files/15816549/etc.jffs2.txt

Also while testing it I've noticed that the jffs2 parser has been ignoring the first file on the file system, which happened to be the cmdline. I've pushed a fix here: @.*** https://github.com/jcmvbkbc/esp-hosted/commit/ceab9c2cf41b2356c099bb04145ec9a90d772d8c I made sure that cmdline is not the first file in the attached image, but if you build your own etc.jffs2 with that file, please also update the bootloader.

— Reply to this email directly, view it on GitHub https://github.com/jcmvbkbc/esp32-linux-build/issues/9#issuecomment-2165040817, or unsubscribe https://github.com/notifications/unsubscribe-auth/BH4ZI6VPHZRBF5ZJDHYAK2TZHFM2RAVCNFSM6AAAAABJE6DBSGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCNRVGA2DAOBRG4 . You are receiving this because you authored the thread.Message ID: @.***>

arashsaifi commented 2 weeks ago

I changed that file with your etc and now it shows a bunch of output but is stuck at some other stage. Here is the output of my board.

linux ptr = 0x42120000
vectors ptr = 0x4037c000
[    0.000000] Linux version 6.10.0-rc2 (idkutellme@computer) (xtensa-esp32s3-linux-uclibcfdpic-gcc (crosstool-NG 1.25.0.183_bec302a) 14.1.0, GNU ld (crosstool-NG 1.25.0.183_bec302a) 2.42.50.20240224) #1 PREEMPT Tue Jun 11 16:43:45 IST 2024
[    0.000000] config ID: c2f0fffe:23090f1f
[    0.000000] earlycon: esp32s3acm0 at MMIO32 0x60038000 (options '')
[    0.000000] printk: legacy bootconsole [esp32s3acm0] enabled
[    0.000000] **********************************************************
[    0.000000] **   NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE   **
[    0.000000] **                                                      **
[    0.000000] ** This system shows unhashed kernel memory addresses   **
[    0.000000] ** via the console, logs, and other interfaces. This    **
[    0.000000] ** might reduce the security of your system.            **
[    0.000000] **                                                      **
[    0.000000] ** If you see this message and you are not debugging    **
[    0.000000] ** the kernel, report this immediately to your system   **
[    0.000000] ** administrator!                                       **
[    0.000000] **                                                      **
[    0.000000] **   NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE   **
[    0.000000] **********************************************************
[    0.000000] Zone ranges:
[    0.000000]   Normal   [mem 0x000000003d800000-0x000000003dffffff]
[    0.000000] Movable zone start for each node
[    0.000000] Early memory node ranges
[    0.000000]   node   0: [mem 0x000000003d800000-0x000000003dffffff]
[    0.000000] Initmem setup node 0 [mem 0x000000003d800000-0x000000003dffffff]
[    0.000000] pcpu-alloc: s0 r0 d32768 u32768 alloc=1*32768
[    0.000000] pcpu-alloc: [0] 0 
[    0.000000] Kernel command line: earlycon=esp32s3acm,mmio32,0x60038000 console=ttyGS3 debug rw root=mtd:rootfs no_hash_pointers
[    0.000000] 
[    0.000000] Dentry cache hash table entries: 1024 (order: 0, 4096 bytes, linear)
[    0.000000] Inode-cache hash table entries: 1024 (order: 0, 4096 bytes, linear)
[    0.000000] Built 1 zonelists, mobility grouping off.  Total pages: 2048
[    0.000000] mem auto-init: stack:off, heap alloc:off, heap free:off, mlocked free:off
[    0.000000] virtual kernel memory layout:
[    0.000000]     lowmem  : 0x3d800000 - 0x3e000000  (    8 MB)
[    0.000000]     .text   : 0x42120000 - 0x4237c068  ( 2416 kB)
[    0.000000]     .rodata : 0x4237d000 - 0x423e2000  (  404 kB)
[    0.000000]     .data   : 0x3d800000 - 0x3d87e3e0  (  504 kB)
[    0.000000]     .init   : 0x3d87e3e0 - 0x3d8825a8  (   16 kB)
[    0.000000]     .bss    : 0x3d8825a8 - 0x3d8ba8e8  (  224 kB)
[    0.000000] Memory: 7312K/8192K available (2416K kernel code, 504K rwdata, 404K rodata, 97K init, 224K bss, 880K reserved, 0K cma-reserved)
[    0.000000] SLUB: HWalign=16, Order=0-3, MinObjects=0, CPUs=1, Nodes=1
[    0.000000] rcu: Preemptible hierarchical RCU implementation.
[    0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 10 jiffies.
[    0.000000] NR_IRQS: 33
[    0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.
[    0.000000] Calibrating CPU frequency 240.00 MHz
[    0.000000] clocksource: ccount: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7963585194 ns
[    0.000066] sched_clock: 32 bits at 240MHz, resolution 4ns, wraps every 8947848189ns
[    0.005114] Calibrating delay loop (skipped)... 240.00 BogoMIPS preset
[    0.007341] pid_max: default: 32768 minimum: 301
[    0.011950] Mount-cache hash table entries: 1024 (order: 0, 4096 bytes, linear)
[    0.014684] Mountpoint-cache hash table entries: 1024 (order: 0, 4096 bytes, linear)
[    0.067419] rcu: Hierarchical SRCU implementation.
[    0.069202] rcu:     Max phase no-delay instances is 1000.
[    0.089964] devtmpfs: initialized
[    0.138413] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604462750000 ns
[    0.141916] futex hash table entries: 256 (order: -1, 3072 bytes, linear)
[    0.145142] pinctrl core: initialized pinctrl subsystem
[    0.159661] NET: Registered PF_NETLINK/PF_ROUTE protocol family
[    0.238172] platform soc: Fixed dependency cycle(s) with /soc/intc@600c2000
[    0.364328] platform soc: Fixed dependency cycle(s) with /soc/intc@600c2000
[    0.433018] clocksource: Switched to clocksource ccount
[    0.499773] NET: Registered PF_INET protocol family
[    0.508758] IP idents hash table entries: 2048 (order: 2, 16384 bytes, linear)
[    0.531970] tcp_listen_portaddr_hash hash table entries: 1024 (order: 0, 4096 bytes, linear)
[    0.538372] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)
[    0.541345] TCP established hash table entries: 1024 (order: 0, 4096 bytes, linear)
[    0.544877] TCP bind hash table entries: 1024 (order: 1, 8192 bytes, linear)
[    0.547464] TCP: Hash tables configured (established 1024 bind 1024)
[    0.551745] UDP hash table entries: 256 (order: 0, 4096 bytes, linear)
[    0.555656] UDP-Lite hash table entries: 256 (order: 0, 4096 bytes, linear)
[    0.561846] NET: Registered PF_UNIX/PF_LOCAL protocol family
[    0.578082] RPC: Registered named UNIX socket transport module.
[    0.580145] RPC: Registered udp transport module.
[    0.581378] RPC: Registered tcp transport module.
[    0.582681] RPC: Registered tcp-with-tls transport module.
[    0.585113] RPC: Registered tcp NFSv4.1 backchannel transport module.
[    0.649206] Initialise system trusted keyrings
[    0.655642] workingset: timestamp_bits=30 max_order=11 bucket_order=0
[    0.667899] NFS: Registering the id_resolver key type
[    0.670907] Key type id_resolver registered
[    0.672556] Key type id_legacy registered
[    0.676257] jffs2: version 2.2. (NAND) © 2001-2006 Red Hat, Inc.
[    0.679892] Key type asymmetric registered
[    0.681558] Asymmetric key parser 'x509' registered
[    0.685484] io scheduler mq-deadline registered
[    0.687237] io scheduler kyber registered
[    0.738966] pinctrl-single 60009004.pinctrl: 49 pins, size 196
[    3.440843] 60000000.serial: ttyS0 at MMIO 0x60000000 (irq = 2, base_baud = 2500000) is a ESP32S3 UART
[    3.475167] 60038000.acm: ttyGS3 at MMIO 0x60038000 (irq = 3, base_baud = 0) is a ESP32S3 ACM
[    3.478164] printk: legacy console [ttyGS3] enabled
[    3.478164] printk: legacy console [ttyGS3] enabled
[    3.480917] printk: legacy bootconsole [esp32s3acm0] disabled
[    3.480917] printk: legacy bootconsole [esp32s3acm0] disabled
[    3.544461] random: crng init done
jcmvbkbc commented 2 weeks ago

stuck at some other stage

It's about the time when the kernel should start accessing flash. Let me take a look at that.

arashsaifi commented 2 weeks ago

Any luck ?

On Thu, 13 Jun 2024, 10:56 pm Max Filippov, @.***> wrote:

stuck at some other stage

It's about the time when the kernel should start accessing flash. Let me take a look at that.

— Reply to this email directly, view it on GitHub https://github.com/jcmvbkbc/esp32-linux-build/issues/9#issuecomment-2166390729, or unsubscribe https://github.com/notifications/unsubscribe-auth/BH4ZI6SY3QM4RIEPTFV2GZTZHHI45AVCNFSM6AAAAABJE6DBSGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCNRWGM4TANZSHE . You are receiving this because you authored the thread.Message ID: @.***>

jcmvbkbc commented 2 weeks ago

Could you try flashing the following kernel image and post its output: xipImage.txt That kernel has debug logging in the flash partition table parser and ipc code.

arashsaifi commented 2 weeks ago

The output after flashing the kernel image is the following:

linux ptr = 0x42120000
vectors ptr = 0x4037c000
[    0.000000] Linux version 6.10.0-rc2-00056-gde20e17aa0f0-dirty (jcmvbkbc@octofox) (xtensa-generic-elf-gcc (GCC) 14.0.1 20240130 (experimental), GNU ld (GNU Binutils) 2.41) #1 PREEMPT Fri Jun 14 10:38:19 PDT 2024
[    0.000000] config ID: c2f0fffe:23090f1f
[    0.000000] earlycon: esp32s3acm0 at MMIO32 0x60038000 (options '')
[    0.000000] printk: legacy bootconsole [esp32s3acm0] enabled
[    0.000000] **********************************************************
[    0.000000] **   NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE   **
[    0.000000] **                                                      **
[    0.000000] ** This system shows unhashed kernel memory addresses   **
[    0.000000] ** via the console, logs, and other interfaces. This    **
[    0.000000] ** might reduce the security of your system.            **
[    0.000000] **                                                      **
[    0.000000] ** If you see this message and you are not debugging    **
[    0.000000] ** the kernel, report this immediately to your system   **
[    0.000000] ** administrator!                                       **
[    0.000000] **                                                      **
[    0.000000] **   NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE   **
[    0.000000] **********************************************************
[    0.000000] Zone ranges:
[    0.000000]   Normal   [mem 0x000000003d800000-0x000000003dffffff]
[    0.000000] Movable zone start for each node
[    0.000000] Early memory node ranges
[    0.000000]   node   0: [mem 0x000000003d800000-0x000000003dffffff]
[    0.000000] Initmem setup node 0 [mem 0x000000003d800000-0x000000003dffffff]
[    0.000000] pcpu-alloc: s0 r0 d32768 u32768 alloc=1*32768
[    0.000000] pcpu-alloc: [0] 0 
[    0.000000] Kernel command line: earlycon=esp32s3acm,mmio32,0x60038000 console=ttyGS3 debug rw root=mtd:rootfs no_hash_pointers
[    0.000000] 
[    0.000000] Dentry cache hash table entries: 1024 (order: 0, 4096 bytes, linear)
[    0.000000] Inode-cache hash table entries: 1024 (order: 0, 4096 bytes, linear)
[    0.000000] Built 1 zonelists, mobility grouping off.  Total pages: 2048
[    0.000000] mem auto-init: stack:off, heap alloc:off, heap free:off, mlocked free:off
[    0.000000] virtual kernel memory layout:
[    0.000000]     lowmem  : 0x3d800000 - 0x3e000000  (    8 MB)
[    0.000000]     .text   : 0x42120000 - 0x4237dfa0  ( 2423 kB)
[    0.000000]     .rodata : 0x4237e000 - 0x423e3000  (  404 kB)
[    0.000000]     .data   : 0x3d800000 - 0x3d87e3e0  (  504 kB)
[    0.000000]     .init   : 0x3d87e3e0 - 0x3d8828c8  (   17 kB)
[    0.000000]     .bss    : 0x3d8828c8 - 0x3d8ba8e8  (  224 kB)
[    0.000000] Memory: 7308K/8192K available (2423K kernel code, 504K rwdata, 404K rodata, 98K init, 224K bss, 884K reserved, 0K cma-reserved)
[    0.000000] SLUB: HWalign=16, Order=0-3, MinObjects=0, CPUs=1, Nodes=1
[    0.000000] rcu: Preemptible hierarchical RCU implementation.
[    0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 10 jiffies.
[    0.000000] NR_IRQS: 33
[    0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.
[    0.000000] Calibrating CPU frequency 240.00 MHz
[    0.000000] clocksource: ccount: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7963585194 ns
[    0.000076] sched_clock: 32 bits at 240MHz, resolution 4ns, wraps every 8947848189ns
[    0.005214] Calibrating delay loop (skipped)... 240.00 BogoMIPS preset
[    0.007398] pid_max: default: 32768 minimum: 301
[    0.012131] Mount-cache hash table entries: 1024 (order: 0, 4096 bytes, linear)
[    0.014948] Mountpoint-cache hash table entries: 1024 (order: 0, 4096 bytes, linear)
[    0.068437] rcu: Hierarchical SRCU implementation.
[    0.070237] rcu:     Max phase no-delay instances is 1000.
[    0.091897] devtmpfs: initialized
[    0.124594] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604462750000 ns
[    0.129039] futex hash table entries: 256 (order: -1, 3072 bytes, linear)
[    0.132569] pinctrl core: initialized pinctrl subsystem
[    0.146977] NET: Registered PF_NETLINK/PF_ROUTE protocol family
[    0.230329] platform soc: Fixed dependency cycle(s) with /soc/intc@600c2000
[    0.361172] platform soc: Fixed dependency cycle(s) with /soc/intc@600c2000
[    0.426946] clocksource: Switched to clocksource ccount
[    0.490167] NET: Registered PF_INET protocol family
[    0.499067] IP idents hash table entries: 2048 (order: 2, 16384 bytes, linear)
[    0.522880] tcp_listen_portaddr_hash hash table entries: 1024 (order: 0, 4096 bytes, linear)
[    0.526020] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)
[    0.531924] TCP established hash table entries: 1024 (order: 0, 4096 bytes, linear)
[    0.534773] TCP bind hash table entries: 1024 (order: 1, 8192 bytes, linear)
[    0.538098] TCP: Hash tables configured (established 1024 bind 1024)
[    0.542866] UDP hash table entries: 256 (order: 0, 4096 bytes, linear)
[    0.545543] UDP-Lite hash table entries: 256 (order: 0, 4096 bytes, linear)
[    0.552158] NET: Registered PF_UNIX/PF_LOCAL protocol family
[    0.568274] RPC: Registered named UNIX socket transport module.
[    0.570336] RPC: Registered udp transport module.
[    0.571581] RPC: Registered tcp transport module.
[    0.572820] RPC: Registered tcp-with-tls transport module.
[    0.574224] RPC: Registered tcp NFSv4.1 backchannel transport module.
[    0.634366] Initialise system trusted keyrings
[    0.641355] workingset: timestamp_bits=30 max_order=11 bucket_order=0
[    0.653897] NFS: Registering the id_resolver key type
[    0.658403] Key type id_resolver registered
[    0.660086] Key type id_legacy registered
[    0.662912] jffs2: version 2.2. (NAND) © 2001-2006 Red Hat, Inc.
[    0.666368] Key type asymmetric registered
[    0.669130] Asymmetric key parser 'x509' registered
[    0.672126] io scheduler mq-deadline registered
[    0.673917] io scheduler kyber registered
[    0.724307] pinctrl-single 60004554.gpio_out_mux: 49 pins, size 196
[    0.740497] pinctrl-single 60009004.pinctrl: 49 pins, size 196
[    3.220860] 60000000.serial: ttyS0 at MMIO 0x60000000 (irq = 2, base_baud = 2500000) is a ESP32S3 UART
[    3.256452] 60038000.acm: ttyGS3 at MMIO 0x60038000 (irq = 3, base_baud = 0) is a ESP32S3 ACM
[    3.261872] printk: legacy console [ttyGS3] enabled
[    3.261872] printk: legacy console [ttyGS3] enabled
[    3.264991] printk: legacy bootconsole [esp32s3acm0] disabled
[    3.264991] printk: legacy bootconsole [esp32s3acm0] disabled
[    3.306761] esp32-ipc 600c0004.ipc: init_hw: regs = 600c0038, queues = 3fc9d838
[    3.309707] random: crng init done
[    3.316678] esp32-ipc 600c0004.ipc: init_hw: queue 0: offset = -1024, mask = 3f
[    3.320481] esp32-ipc 600c0004.ipc: init_hw: queue 1: offset = -512, mask = 3f
[    3.354403] esp32_ipc_tx: q->read = 0, q->write = 0
jcmvbkbc commented 2 weeks ago

The output after flashing the kernel image is the following:

[    3.316678] esp32-ipc 600c0004.ipc: init_hw: queue 0: offset = -1024, mask = 3f
[    3.320481] esp32-ipc 600c0004.ipc: init_hw: queue 1: offset = -512, mask = 3f
[    3.354403] esp32_ipc_tx: q->read = 0, q->write = 0

Ok, so there's no response to the flash read request from the firmware. Let's see what happens there. Could you flash the attached binary to the factory partition and post the boot log? It has logging in the linux IPC code. network_adapter.bin.txt

arashsaifi commented 2 weeks ago

Well I have gone out for the weekend and will get back to you on Tuesday with the output log

On Sat, 15 Jun 2024, 8:27 am Max Filippov, @.***> wrote:

The output after flashing the kernel image is the following:

[ 3.316678] esp32-ipc 600c0004.ipc: init_hw: queue 0: offset = -1024, mask = 3f [ 3.320481] esp32-ipc 600c0004.ipc: init_hw: queue 1: offset = -512, mask = 3f [ 3.354403] esp32_ipc_tx: q->read = 0, q->write = 0

Ok, so there's no response to the flash read request from the firmware. Let's see what happens there. Could you flash the attached binary to the factory partition and post the boot log? It has logging in the linux IPC code. network_adapter.bin.txt https://github.com/user-attachments/files/15845446/network_adapter.bin.txt

— Reply to this email directly, view it on GitHub https://github.com/jcmvbkbc/esp32-linux-build/issues/9#issuecomment-2169069976, or unsubscribe https://github.com/notifications/unsubscribe-auth/BH4ZI6VFCQXOI6L24VOYGV3ZHOUSPAVCNFSM6AAAAABJE6DBSGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCNRZGA3DSOJXGY . You are receiving this because you authored the thread.Message ID: @.***>

arashsaifi commented 2 weeks ago

Can you please tell the steps to attach that to the partition

On Sat, 15 Jun 2024, 8:39 am Arash Saifi, @.***> wrote:

Well I have gone out for the weekend and will get back to you on Tuesday with the output log

On Sat, 15 Jun 2024, 8:27 am Max Filippov, @.***> wrote:

The output after flashing the kernel image is the following:

[ 3.316678] esp32-ipc 600c0004.ipc: init_hw: queue 0: offset = -1024, mask = 3f [ 3.320481] esp32-ipc 600c0004.ipc: init_hw: queue 1: offset = -512, mask = 3f [ 3.354403] esp32_ipc_tx: q->read = 0, q->write = 0

Ok, so there's no response to the flash read request from the firmware. Let's see what happens there. Could you flash the attached binary to the factory partition and post the boot log? It has logging in the linux IPC code. network_adapter.bin.txt https://github.com/user-attachments/files/15845446/network_adapter.bin.txt

— Reply to this email directly, view it on GitHub https://github.com/jcmvbkbc/esp32-linux-build/issues/9#issuecomment-2169069976, or unsubscribe https://github.com/notifications/unsubscribe-auth/BH4ZI6VFCQXOI6L24VOYGV3ZHOUSPAVCNFSM6AAAAABJE6DBSGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCNRZGA3DSOJXGY . You are receiving this because you authored the thread.Message ID: @.***>

jcmvbkbc commented 2 weeks ago

I used the following command: parttool.py -b 2000000 write_partition --partition-name factory --input network_adapter.bin

arashsaifi commented 1 week ago

Sorry for my late response. Here is the output of the bootlogs after flashing that .bin file onto the board.

E (588) linux_ipc: esp_shmem_hw_queue_read, r = 0 w = 0
linux ptr = 0x42120000
vectors ptr = 0x4037c000
[    0.000000] Linux version 6.10.0-rc2-00056-gde20e17aa0f0-dirty (jcmvbkbc@octofox) (xtensa-generic-elf-gcc (GCC) 14.0.1 20240130 (experimental), GNU ld (GNU Binutils) 2.41) #1 PREEMPT Fri Jun 14 10:38:19 PDT 2024
[    0.000000] config ID: c2f0fffe:23090f1f
[    0.000000] earlycon: esp32s3acm0 at MMIO32 0x60038000 (options '')
[    0.000000] printk: legacy bootconsole [esp32s3acm0] enabled
[    0.000000] **********************************************************
[    0.000000] **   NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE   **
[    0.000000] **                                                      **
[    0.000000] ** This system shows unhashed kernel memory addresses   **
[    0.000000] ** via the console, logs, and other interfaces. This    **
[    0.000000] ** might reduce the security of your system.            **
[    0.000000] **                                                      **
[    0.000000] ** If you see this message and you are not debugging    **
[    0.000000] ** the kernel, report this immediately to your system   **
[    0.000000] ** administrator!                                       **
[    0.000000] **                                                      **
[    0.000000] **   NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE   **
[    0.000000] **********************************************************
[    0.000000] Zone ranges:
[    0.000000]   Normal   [mem 0x000000003d800000-0x000000003dffffff]
[    0.000000] Movable zone start for each node
[    0.000000] Early memory node ranges
[    0.000000]   node   0: [mem 0x000000003d800000-0x000000003dffffff]
[    0.000000] Initmem setup node 0 [mem 0x000000003d800000-0x000000003dffffff]
[    0.000000] pcpu-alloc: s0 r0 d32768 u32768 alloc=1*32768
[    0.000000] pcpu-alloc: [0] 0 
[    0.000000] Kernel command line: earlycon=esp32s3acm,mmio32,0x60038000 console=ttyGS3 debug rw root=mtd:rootfs no_hash_pointers
[    0.000000] 
[    0.000000] Dentry cache hash table entries: 1024 (order: 0, 4096 bytes, linear)
[    0.000000] Inode-cache hash table entries: 1024 (order: 0, 4096 bytes, linear)
[    0.000000] Built 1 zonelists, mobility grouping off.  Total pages: 2048
[    0.000000] mem auto-init: stack:off, heap alloc:off, heap free:off, mlocked free:off
    0.000000] virtual kernel memory layout:
[    0.000000]     lowmem  : 0x3d800000 - 0x3e000000  (    8 MB)
[    0.000000]     .text   : 0x42120000 - 0x4237dfa0  ( 2423 kB)
[    0.000000]     .rodata : 0x4237e000 - 0x423e3000  (  404 kB)
[    0.000000]     .data   : 0x3d800000 - 0x3d87e3e0  (  504 kB)
[    0.000000]     .init   : 0x3d87e3e0 - 0x3d8828c8  (   17 kB)
[    0.000000]     .bss    : 0x3d8828c8 - 0x3d8ba8e8  (  224 kB)
[    0.000000] Memory: 7308K/8192K available (2423K kernel code, 504K rwdata, 404K rodata, 98K init, 224K bss, 884K reserved, 0K cma-reserved)
[    0.000000] SLUB: HWalign=16, Order=0-3, MinObjects=0, CPUs=1, Nodes=1
[    0.000000] rcu: Preemptible hierarchical RCU implementation.
[    0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 10 jiffies.
[    0.000000] NR_IRQS: 33
[    0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.
[    0.000000] Calibrating CPU frequency 240.00 MHz
[    0.000000] clocksource: ccount: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7963585194 ns
[    0.000076] sched_clock: 32 bits at 240MHz, resolution 4ns, wraps every 8947848189ns
[    0.004396] Calibrating delay loop (skipped)... 240.00 BogoMIPS preset
[    0.005873] pid_max: default: 32768 minimum: 301
[    0.009262] Mount-cache hash table entries: 1024 (order: 0, 4096 bytes, linear)
[    0.012270] Mountpoint-cache hash table entries: 1024 (order: 0, 4096 bytes, linear)
[    0.065202] rcu: Hierarchical SRCU implementation.
[    0.066540] rcu:     Max phase no-delay instances is 1000.
[    0.087962] devtmpfs: initialized
[    0.121430] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604462750000 ns
[    0.123894] futex hash table entries: 256 (order: -1, 3072 bytes, linear)
[    0.126426] pinctrl core: initialized pinctrl subsystem
[    0.141935] NET: Registered PF_NETLINK/PF_ROUTE protocol family
[    0.223997] platform soc: Fixed dependency cycle(s) with /soc/intc@600c2000
[    0.354673] platform soc: Fixed dependency cycle(s) with /soc/intc@600c2000
[    0.418813] clocksource: Switched to clocksource ccount
[    0.481472] NET: Registered PF_INET protocol family
[    0.488295] IP idents hash table entries: 2048 (order: 2, 16384 bytes, linear)
[    0.512064] tcp_listen_portaddr_hash hash table entries: 1024 (order: 0, 4096 bytes, linear)
[    0.514307] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)
[    0.515987] TCP established hash table entries: 1024 (order: 0, 4096 bytes, linear)
[    0.517711] TCP bind hash table entries: 1024 (order: 1, 8192 bytes, linear)
[    0.523669] TCP: Hash tables configured (established 1024 bind 1024)
[    0.527920] UDP hash table entries: 256 (order: 0, 4096 bytes, linear)
[    0.531352] UDP-Lite hash table entries: 256 (order: 0, 4096 bytes, linear)
[    0.536384] NET: Registered PF_UNIX/PF_LOCAL protocol family
[    0.552014] RPC: Registered named UNIX socket transport module.
[    0.553441] RPC: Registered udp transport module.
[    0.554205] RPC: Registered tcp transport module.
[    0.555006] RPC: Registered tcp-with-tls transport module.
[    0.555898] RPC: Registered tcp NFSv4.1 backchannel transport module.
[    0.614889] Initialise system trusted keyrings
[    0.621524] workingset: timestamp_bits=30 max_order=11 bucket_order=0
[    0.633610] NFS: Registering the id_resolver key type
[    0.636209] Key type id_resolver registered
[    0.637418] Key type id_legacy registered
[    0.641437] jffs2: version 2.2. (NAND) © 2001-2006 Red Hat, Inc.
[    0.644239] Key type asymmetric registered
[    0.645432] Asymmetric key parser 'x509' registered
[    0.647630] io scheduler mq-deadline registered
[    0.650035] io scheduler kyber registered
[    0.707606] pinctrl-single 60004554.gpio_out_mux: 49 pins, size 196
[    0.722653] pinctrl-single 60009004.pinctrl: 49 pins, size 196
[    3.196637] 60000000.serial: ttyS0 at MMIO 0x60000000 (irq = 2, base_baud = 2500000) is a ESP32S3 UART
[    3.233594] 60038000.acm: ttyGS3 at MMIO 0x60038000 (irq = 3, base_baud = 0) is a ESP32S3 ACM
[    3.236333] printk: legacy console [ttyGS3] enabled
[    3.236333] printk: legacy console [ttyGS3] enabled
[    3.238341] printk: legacy bootconsole [esp32s3acm0] disabled
[    3.238341] printk: legacy bootconsole [esp32s3acm0] disabled
[    3.277594] esp32-ipc 600c0004.ipc: init_hw: regs = 600c0038, queues = 3fc9dea0
[    3.282292] random: crng init done
[    3.288254] esp32-ipc 600c0004.ipc: init_hw: queue 0: offset = -1024, mask = 3f
[    3.291894] esp32-ipc 600c0004.ipc: init_hw: queue 1: offset = -512, mask = 3f
[    3.325807] esp32_ipc_tx: q->read = 0, q->write = 0
jcmvbkbc commented 1 week ago

Here is the output of the bootlogs after flashing that .bin file onto the board.

Hmmm, puzzling. Could you please repeat it with the following network_adapter.bin.txt? It has some more logging in the IRQ delivery path and also a 1-second timeout in the IPC RX task.

arashsaifi commented 1 week ago

Yup! Here's the output after flashing that bin file:

Waiting for the device to reconnect.......
E (598) linux_ipc: esp_shmem_hw_queue_read, r = 0 w = 0

pp rom version: e7ae62f
net80211 rom version: e7ae62f
E (1608) linux_ipc: intenable = 0x1200320c, system_cpu_intr = 0x00000000, interrupt_core0_intr_status_2 = 0x00000000

E (1608) linux_ipc: esp_shmem_hw_queue_read, r = 0 w = 0

linux ptr = 0x42120000
vectors ptr = 0x4037c000
E (34138) linux_ipc: intenable = 0x1200320c, system_cpu_intr = 0x00000000, interrupt_core0_intr_status_2 = 0x00000000

[    0.000000] Linux version 6.10.0-rc2-00056-gde20e17aa0f0-dirty (jcmvbkbc@octofox) (xtensa-generic-elf-gcc (GCC) 14.0.1 20240130 (experimental), GNU ld (GNU Binutils) 2.41) #1 PREEMPT Fri Jun 14 10:38:19 PDT 2024
[    0.000000] config ID: c2f0fffe:23090f1f
[    0.000000] earlycon: esp32s3acm0 at MMIO32 0x60038000 (options '')
[    0.000000] printk: legacy bootconsole [esp32s3acm0] enabled
[    0.000000] **********************************************************
[    0.000000] **   NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE   **
[    0.000000] **                                                      **
[    0.000000] ** This system shows unhashed kernel memory addresses   **
[    0.000000] ** via the console, logs, and other interfaces. This    **
[    0.000000] ** might reduce the security of your system.            **
[    0.000000] **                                                      **
[    0.000000] ** If you see this message and you are not debugging    **
[    0.000000] ** the kernel, report this immediately to your system   **
[    0.000000] ** administrator!                                       **
[    0.000000] **                                                      **
[    0.000000] **   NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE   **
[    0.000000] **********************************************************
[    0.000000] Zone ranges:
[    0.000000]   Normal   [mem 0x000000003d800000-0x000000003dffffff]
[    0.000000] Movable zone start for each node
[    0.000000] Early memory node ranges
[    0.000000]   node   0: [mem 0x000000003d800000-0x000000003dffffff]
[    0.000000] Initmem setup node 0 [mem 0x000000003d800000-0x000000003dffffff]
[    0.000000] pcpu-alloc: s0 r0 d32768 u32768 alloc=1*32768
[    0.000000] pcpu-alloc: [0] 0 
[    0.000000] Kernel command line: earlycon=esp32s3acm,mmio32,0x60038000 console=ttyGS3 debug rw root=mtd:rootfs no_hash_pointers
[    0.000000] 
[    0.000000] Dentry cache hash table entries: 1024 (order: 0, 4096 bytes, linear)
[    0.000000] Inode-cache hash table entries: 1024 (order: 0, 4096 bytes, linear)
[    0.000000] Built 1 zonelists, mobility grouping off.  Total pages: 2048
[    0.000000] mem auto-init: stack:off, heap alloc:off, heap free:off, mlocked free:off
[    0.000000] virtual kernel memory layout:
[    0.000000]     lowmem  : 0x3d800000 - 0x3e000000  (    8 MB)
[    0.000000]     .text   : 0x42120000 - 0x4237dfa0  ( 2423 kB)
[    0.000000]     .rodata : 0x4237e000 - 0x423e3000  (  404 kB)
[    0.000000]     .data   : 0x3d800000 - 0x3d87e3e0  (  504 kB)
[    0.000000]     .init   : 0x3d87e3e0 - 0x3d8828c8  (   17 kB)
[    0.000000]     .bss    : 0x3d8828c8 - 0x3d8ba8e8  (  224 kB)
[    0.000000] Memory: 7308K/8192K available (2423K kernel code, 504K rwdata, 404K rodata, 98K init, 224K bss, 884K reserved, 0K cma-reserved)
[    0.000000] SLUB: HWalign=16, Order=0-3, MinObjects=0, CPUs=1, Nodes=1
[    0.000000] rcu: Preemptible hierarchical RCU implementation.
[    0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 10 jiffies.
[    0.000000] NR_IRQS: 33
[    0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.
[    0.000000] Calibrating CPU frequency 240.00 MHz
[    0.000000] clocksource: ccount: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7963585194 ns
[    0.000076] sched_clock: 32 bits at 240MHz, resolution 4ns, wraps every 8947848189ns
[    0.005214] Calibrating delay loop (skipped)... 240.00 BogoMIPS preset
[    0.007397] pid_max: default: 32768 minimum: 301
[    0.012112] Mount-cache hash table entries: 1024 (order: 0, 4096 bytes, linear)
[    0.014910] Mountpoint-cache hash table entries: 1024 (order: 0, 4096 bytes, linear)
[    0.068419] rcu: Hierarchical SRCU implementation.
[    0.070213] rcu:     Max phase no-delay instances is 1000.
[    0.091882] devtmpfs: initialized
[    0.124616] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604462750000 ns
[    0.129045] futex hash table entries: 256 (order: -1, 3072 bytes, linear)
[    0.132551] pinctrl core: initialized pinctrl subsystem
[    0.146956] NET: Registered PF_NETLINK/PF_ROUTE protocol family
[    0.230059] platform soc: Fixed dependency cycle(s) with /soc/intc@600c2000
[    0.360871] platform soc: Fixed dependency cycle(s) with /soc/intc@600c2000
[    0.426796] clocksource: Switched to clocksource ccount
[    0.490671] NET: Registered PF_INET protocol family
[    0.499735] IP idents hash table entries: 2048 (order: 2, 16384 bytes, linear)
[    0.523759] tcp_listen_portaddr_hash hash table entries: 1024 (order: 0, 4096 bytes, linear)
[    0.530142] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)
[    0.533123] TCP established hash table entries: 1024 (order: 0, 4096 bytes, linear)
[    0.535586] TCP bind hash table entries: 1024 (order: 1, 8192 bytes, linear)
[    0.538989] TCP: Hash tables configured (established 1024 bind 1024)
[    0.543737] UDP hash table entries: 256 (order: 0, 4096 bytes, linear)
[    0.546458] UDP-Lite hash table entries: 256 (order: 0, 4096 bytes, linear)
[    0.553378] NET: Registered PF_UNIX/PF_LOCAL protocol family
[    0.569586] RPC: Registered named UNIX socket transport module.
[    0.571734] RPC: Registered udp transport module.
[    0.572981] RPC: Registered tcp transport module.
[    0.574211] RPC: Registered tcp-with-tls transport module.
[    0.575724] RPC: Registered tcp NFSv4.1 backchannel transport module.
[    0.635816] Initialise system trusted keyrings
[    0.642550] workingset: timestamp_bits=30 max_order=11 bucket_order=0
[    0.655311] NFS: Registering the id_resolver key type
[    0.659442] Key type id_resolver registered
[    0.661153] Key type id_legacy registered
[    0.663950] jffs2: version 2.2. (NAND) © 2001-2006 Red Hat, Inc.
[    0.668386] Key type asymmetric registered
[    0.670095] Asymmetric key parser 'x509' registered
[    0.672830] io scheduler mq-deadline registered
[    0.674613] io scheduler kyber registered
[    0.725647] pinctrl-single 60004554.gpio_out_mux: 49 pins, size 196
[    0.749287] pinctrl-single 60009004.pinctrl: 49 pins, size 196
[    3.228819] 60000000.serial: ttyS0 at MMIO 0x60000000 (irq = 2, base_baud = 2500000) is a ESP32S3 UART
[    3.262484] 60038000.acm: ttyGS3 at MMIO 0x60038000 (irq = 3, base_baud = 0) is a ESP32S3 ACM
[    3.265548] printk: legacy console [ttyGS3] enabled
[    3.265548] printk: legacy console [ttyGS3] enabled
[    3.271068] printk: legacy bootconsole [esp32s3acm0] disabled
[    3.271068] printk: legacy bootconsole [esp32s3acm0] disabled
[    3.319701] random: crng init done
[    3.333905] esp32-ipc 600c0004.ipc: init_hw: regs = 600c0038, queues = 3fc9dea0
[    3.336444] esp32-ipc 600c0004.ipc: init_hw: queue 0: offset = -1024, mask = 3f
[    3.340131] esp32-ipc 600c0004.ipc: init_hw: queue 1: offset = -512, mask = 3f
[    3.373631] esp32_ipc_tx: q->read = 0, q->write = 0
jcmvbkbc commented 1 week ago

Hmmm… Even more puzzling. Just to make sure: nothing else appears if you wait a few more seconds after the line [ 3.373631] esp32_ipc_tx: q->read = 0, q->write = 0 is printed?

arashsaifi commented 1 week ago

Nope waited atleast 2 minutes.

On Thu, 20 Jun 2024, 7:23 am Max Filippov, @.***> wrote:

Hmmm… Even more puzzling. Just to make sure: nothing else appears if you wait a few more seconds after the line [ 3.373631] esp32_ipc_tx: q->read = 0, q->write = 0 is printed?

— Reply to this email directly, view it on GitHub https://github.com/jcmvbkbc/esp32-linux-build/issues/9#issuecomment-2179673935, or unsubscribe https://github.com/notifications/unsubscribe-auth/BH4ZI6TZZU5Y5DOQQM7AFRDZIIYYFAVCNFSM6AAAAABJE6DBSGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCNZZGY3TGOJTGU . You are receiving this because you authored the thread.Message ID: @.***>

jcmvbkbc commented 1 week ago

Is this output in chronological order, or is this a screen copy? E (34138) suggests that it happened 34 seconds after the boot, but there's no matching E (34138) linux_ipc: esp_shmem_hw_queue_read, r = ? w = ? line. Also a couple of lines at the very beginning is missing.

arashsaifi commented 1 week ago

Everything is in chronological order.

On Thu, 20 Jun 2024, 7:55 am Max Filippov, @.***> wrote:

Is this output in chronological order, or is this a screen copy? E (34138) suggests that it happened 34 seconds after the boot, but there's no matching E (34138) linux_ipc: esp_shmem_hw_queue_read, r = ? w = ? line. Also a couple of lines at the very beginning is missing.

— Reply to this email directly, view it on GitHub https://github.com/jcmvbkbc/esp32-linux-build/issues/9#issuecomment-2179695426, or unsubscribe https://github.com/notifications/unsubscribe-auth/BH4ZI6TPHYREQN65YHG4DC3ZII4QHAVCNFSM6AAAAABJE6DBSGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCNZZGY4TKNBSGY . You are receiving this because you authored the thread.Message ID: @.***>

arashsaifi commented 1 week ago

This is a screen copy (output copied after running IDF.py monitor)

On Thu, 20 Jun 2024, 8:12 am Arash Saifi, @.***> wrote:

Everything is in chronological order.

On Thu, 20 Jun 2024, 7:55 am Max Filippov, @.***> wrote:

Is this output in chronological order, or is this a screen copy? E (34138) suggests that it happened 34 seconds after the boot, but there's no matching E (34138) linux_ipc: esp_shmem_hw_queue_read, r = ? w = ? line. Also a couple of lines at the very beginning is missing.

— Reply to this email directly, view it on GitHub https://github.com/jcmvbkbc/esp32-linux-build/issues/9#issuecomment-2179695426, or unsubscribe https://github.com/notifications/unsubscribe-auth/BH4ZI6TPHYREQN65YHG4DC3ZII4QHAVCNFSM6AAAAABJE6DBSGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCNZZGY4TKNBSGY . You are receiving this because you authored the thread.Message ID: @.***>

arashsaifi commented 1 week ago

I tried reflashing it and then monitoring the output which was totally same as last time.

Let's try one thing, could you send me all your .bins, xipimage and whatever else is necessary (for the usb serial) that is to be flashed on the board maybe my build wasn't done properly.

On Thu, 20 Jun 2024, 8:13 am Arash Saifi, @.***> wrote:

This is a screen copy (output copied after running IDF.py monitor)

On Thu, 20 Jun 2024, 8:12 am Arash Saifi, @.***> wrote:

Everything is in chronological order.

On Thu, 20 Jun 2024, 7:55 am Max Filippov, @.***> wrote:

Is this output in chronological order, or is this a screen copy? E (34138) suggests that it happened 34 seconds after the boot, but there's no matching E (34138) linux_ipc: esp_shmem_hw_queue_read, r = ? w = ? line. Also a couple of lines at the very beginning is missing.

— Reply to this email directly, view it on GitHub https://github.com/jcmvbkbc/esp32-linux-build/issues/9#issuecomment-2179695426, or unsubscribe https://github.com/notifications/unsubscribe-auth/BH4ZI6TPHYREQN65YHG4DC3ZII4QHAVCNFSM6AAAAABJE6DBSGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCNZZGY4TKNBSGY . You are receiving this because you authored the thread.Message ID: @.***>

jcmvbkbc commented 1 week ago

could you send me all your .bins, xipimage and whatever else is necessary

you already got both xipImage and network_adapter.bin, which is combined linux bootloader and ipc server for FLASH and wifi. I'm still not sure what happens with the IPC server, perhaps I should make yet another version with printf-only logging to avoid any possibility of control codes messing up the output. Otherwise my impression is that it's a hardware issue. Maybe add capacitors to the power rails? Or maybe I should make a version that boots with CPU clock set to 80MHz?

jcmvbkbc commented 1 week ago

perhaps I should make yet another version with printf-only logging to avoid any possibility of control codes messing up the output

Oh, I think I've got a better idea: could you reflash the etc partition with an image without the cmdline file? That way we'd only get the firmware output in the monitor. And the following network_adapter.bin.txt boots with 80MHz CPU clock, could you please try it as well?

arashsaifi commented 1 week ago

Without the command line will it still show it's monitor output through the native usb UART ?

I'll give it a shot once I reach home and update you

On Sun, 23 Jun 2024, 6:11 pm Max Filippov, @.***> wrote:

perhaps I should make yet another version with printf-only logging to avoid any possibility of control codes messing up the output

Oh, I think I've got a better idea: could you reflash the etc partition with an image without the cmdline file? That way we'd only get the firmware output in the monitor. And the following network_adapter.bin.txt https://github.com/user-attachments/files/15944560/network_adapter.bin.txt boots with 80MHz CPU clock, could you please try it as well?

— Reply to this email directly, view it on GitHub https://github.com/jcmvbkbc/esp32-linux-build/issues/9#issuecomment-2184973884, or unsubscribe https://github.com/notifications/unsubscribe-auth/BH4ZI6X2SWAVLYPO465DTKTZI2667AVCNFSM6AAAAABJE6DBSGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCOBUHE3TGOBYGQ . You are receiving this because you authored the thread.Message ID: @.***>

arashsaifi commented 1 week ago

Okay, here's the output after doing the necessary steps.

E (583) linux_ipc: esp_ipc_init, ESP_SHMEM_IRQ_TO_HOST_REE (1643) linux_ipc: intenable = 0x1200320c, system_cpu_intr = 0x00000000, interrupt_core0_intr_status_2 = 0x00000000

E (1643) linux_ipc: esp_shmem_hw_queue_read, r = 0 w = 0

linux ptr = 0x42120000
vectors ptr = 0x4037c000
[    0.000000] Linux version 6.10.0-rc2-00056-gde20e17aa0f0-dirty (jcmvbkbc@octofox) (xtensa-generic-elf-gcc (GCC) 14.0.1 20240130 (experimental), GNU ld (GNU Binutils) 2.41) #1 PREEMPT Fri Jun 14 10:38:19 PDT 2024
[    0.000000] config ID: c2f0fffe:23090f1f
[    0.000000] earlycon: esp32s3acm0 at MMIO32 0x60038000 (options '')
[    0.000000] printk: legacy bootconsole [esp32s3acm0] enabled
[    0.000000] **********************************************************
[    0.000000] **   NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE   **
[    0.000000] **                                                      **
[    0.000000] ** This system shows unhashed kernel memory addresses   **
[    0.000000] ** via the console, logs, and other interfaces. This    **
[    0.000000] ** might reduce the security of your system.            **
[    0.000000] **                                                      **
[    0.000000] ** If you see this message and you are not debugging    **
[    0.000000] ** the kernel, report this immediately to your system   **
[    0.000000] ** administrator!                                       **
[    0.000000] **                                                      **
[    0.000000] **   NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE   **
[    0.000000] **********************************************************
[    0.000000] Zone ranges:
[    0.000000]   Normal   [mem 0x000000003d800000-0x000000003dffffff]
[    0.000000] Movable zone start for each node
[    0.000000] Early memory node ranges
[    0.000000]   node   0: [mem 0x000000003d800000-0x000000003dffffff]
[    0.000000] Initmem setup node 0 [mem 0x000000003d800000-0x000000003dffffff]
    0.000000] pcpu-alloc: s0 r0 d32768 u32768 alloc=1*32768
[    0.000000] pcpu-alloc: [0] 0 
[    0.000000] Kernel command line: earlycon=esp32s3acm,mmio32,0x60038000 console=ttyGS3 debug rw root=mtd:rootfs no_hash_pointers
[    0.000000] 
[    0.000000] Dentry cache hash table entries: 1024 (order: 0, 4096 bytes, linear)
[    0.000000] Inode-cache hash table entries: 1024 (order: 0, 4096 bytes, linear)
[    0.000000] Built 1 zonelists, mobility grouping off.  Total pages: 2048
[    0.000000] mem auto-init: stack:off, heap alloc:off, heap free:off, mlocked free:off
    0.000000] virtual kernel memory layout:
[    0.000000]     lowmem  : 0x3d800000 - 0x3e000000  (    8 MB)
[    0.000000]     .text   : 0x42120000 - 0x4237dfa0  ( 2423 kB)
[    0.000000]     .rodata : 0x4237e000 - 0x423e3000  (  404 kB)
[    0.000000]     .data   : 0x3d800000 - 0x3d87e3e0  (  504 kB)
[    0.000000]     .init   : 0x3d87e3e0 - 0x3d8828c8  (   17 kB)
[    0.000000]     .bss    : 0x3d8828c8 - 0x3d8ba8e8  (  224 kB)
[    0.000000] Memory: 7308K/8192K available (2423K kernel code, 504K rwdata, 404K rodata, 98K init, 224K bss, 884K reserved, 0K cma-reserved)
[    0.000000] SLUB: HWalign=16, Order=0-3, MinObjects=0, CPUs=1, Nodes=1
[    0.000000] rcu: Preemptible hierarchical RCU implementation.
[    0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 10 jiffies.
[    0.000000] NR_IRQS: 33
[    0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.
[    0.000000] Calibrating CPU frequency 80.00 MHz
[    0.000000] clocksource: ccount: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 23890755578 ns
[    0.000079] sched_clock: 32 bits at 80MHz, resolution 12ns, wraps every 26843545593ns
[    0.005305] Calibrating delay loop (skipped)... 80.00 BogoMIPS preset
[    0.007180] pid_max: default: 32768 minimum: 301
[    0.012272] Mount-cache hash table entries: 1024 (order: 0, 4096 bytes, linear)
[    0.014763] Mountpoint-cache hash table entries: 1024 (order: 0, 4096 bytes, linear)
[    0.078877] rcu: Hierarchical SRCU implementation.
[    0.080342] rcu:     Max phase no-delay instances is 1000.
    0.105232] devtmpfs: initialized
[    0.162950] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604462750000 ns
[    0.165978] futex hash table entries: 256 (order: -1, 3072 bytes, linear)
[    0.169171] pinctrl core: initialized pinctrl subsystem
[    0.191750] NET: Registered PF_NETLINK/PF_ROUTE protocol family
[    0.352328] platform soc: Fixed dependency cycle(s) with /soc/intc@600c2000
[    0.559325] platform soc: Fixed dependency cycle(s) with /soc/intc@600c2000
[    0.655369] clocksource: Switched to clocksource ccount
[    0.738946] NET: Registered PF_INET protocol family
[    0.752056] IP idents hash table entries: 2048 (order: 2, 16384 bytes, linear)
[    0.794408] tcp_listen_portaddr_hash hash table entries: 1024 (order: 0, 4096 bytes, linear)
[    0.800981] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)
[    0.803251] TCP established hash table entries: 1024 (order: 0, 4096 bytes, linear)
[    0.806609] TCP bind hash table entries: 1024 (order: 1, 8192 bytes, linear)
[    0.809104] TCP: Hash tables configured (established 1024 bind 1024)
[    0.813362] UDP hash table entries: 256 (order: 0, 4096 bytes, linear)
[    0.817122] UDP-Lite hash table entries: 256 (order: 0, 4096 bytes, linear)
[    0.823845] NET: Registered PF_UNIX/PF_LOCAL protocol family
[    0.842308] RPC: Registered named UNIX socket transport module.
[    0.844041] RPC: Registered udp transport module.
[    0.845102] RPC: Registered tcp transport module.
[    0.847407] RPC: Registered tcp-with-tls transport module.
[    0.848623] RPC: Registered tcp NFSv4.1 backchannel transport module.
[    0.971100] Initialise system trusted keyrings
[    0.977903] workingset: timestamp_bits=30 max_order=11 bucket_order=0
[    0.992173] NFS: Registering the id_resolver key type
[    0.995116] Key type id_resolver registered
[    0.997709] Key type id_legacy registered
[    1.001539] jffs2: version 2.2. (NAND) © 2001-2006 Red Hat, Inc.
[    1.004818] Key type asymmetric registered
[    1.007587] Asymmetric key parser 'x509' registered
[    1.010317] io scheduler mq-deadline registered
[    1.011828] io scheduler kyber registered
[    1.097542] pinctrl-single 60004554.gpio_out_mux: 49 pins, size 196
[    1.118773] pinctrl-single 60009004.pinctrl: 49 pins, size 196
    4.237845] 60000000.serial: ttyS0 at MMIO 0x60000000 (irq = 2, base_baud = 2500000) is a ESP32S3 UART
[    4.289858] 60038000.acm: ttyGS3 at MMIO 0x60038000 (irq = 3, base_baud = 0) is a ESP32S3 ACM
[    4.292730] printk: legacy console [ttyGS3] enabled
[    4.292730] printk: legacy console [ttyGS3] enabled
[    4.295063] printk: legacy bootconsole [esp32s3acm0] disabled
[    4.295063] printk: legacy bootconsole [esp32s3acm0] disabled
[    4.338769] random: crng init done
[    4.356544] esp32-ipc 600c0004.ipc: init_hw: regs = 600c0038, queues = 3fc9dea0
[    4.358783] esp32-ipc 600c0004.ipc: init_hw: queue 0: offset = -1024, mask = 3f
[    4.360641] esp32-ipc 600c0004.ipc: init_hw: queue 1: offset = -512, mask = 3f
[    4.411901] esp32_ipc_tx: q->read = 0, q->write = 0

I dont really think it is a hardware problem since I tried installing circuitpython to it and that worked flawlessly and could also run a few test programs like hello world to it.

arashsaifi commented 1 week ago

I ran the chip id command and it says the crystal is 40mhz and not 80mhz

esptool.py v4.6.2
Serial port COM18
Connecting...
Chip is ESP32-S3 (revision v0.2)
Features: WiFi, BLE
Crystal is 40MHz
MAC: f0:f5:bd:4b:f3:e4
jcmvbkbc commented 1 week ago

Without the command line will it still show it's monitor output through the native usb UART ?

It will not and that's what I want: we already know what the kernel prints and where it gets stuck.

here's the output

There's still the kernel output there. Anyway, what we have in the firmware right now would not answer the question where it gets stuck. Let me think how to debug that.

I dont really think it is a hardware problem

Ok.

the chip id command says the crystal is 40mhz and not 80mhz

80MHz is the lowest CPU clock frequency in case it runs off of PLL and that can be chosen in the idf menuconfig

arashsaifi commented 1 week ago

Oof this all is going over my head. What should I try next do tell me

On Mon, 24 Jun 2024, 12:00 pm Max Filippov, @.***> wrote:

Without the command line will it still show it's monitor output through the native usb UART ?

It will not and that's what I want: we already know what the kernel prints and where it gets stuck.

here's the output

There's still the kernel output there. Anyway, what we have in the firmware right now would not answer the question where it gets stuck. Let me think how to debug that.

I dont really think it is a hardware problem

Ok.

the chip id command says the crystal is 40mhz and not 80mhz

80MHz is the lowest CPU clock frequency in case it runs off of PLL and that can be chosen in the idf menuconfig

— Reply to this email directly, view it on GitHub https://github.com/jcmvbkbc/esp32-linux-build/issues/9#issuecomment-2185714102, or unsubscribe https://github.com/notifications/unsubscribe-auth/BH4ZI6QQ2WJFO436UIVEAOLZI64IDAVCNFSM6AAAAABJE6DBSGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCOBVG4YTIMJQGI . You are receiving this because you authored the thread.Message ID: @.***>

jcmvbkbc commented 6 days ago

Ok, can you flash the following network_adapter.bin.txt and post the output? It prints PRID of the linux and IPC cores and hangs the linux core to let the IPC core proceed.

arashsaifi commented 6 days ago

okay, so here is the output and after that it loops the last 2 lines forever.

E (589) linux_ipc: esp_ipc_init, ESP_SHMEM_IRQ_TO_HOST_REE (1649) linux_ipc: intenable = 0x1200320c, system_cpu_intr = 0x00000000, interrupt_core0_intr_status_2 = 0x00000000

E (1649) linux_ipc: esp_shmem_hw_queue_read, r = 0 w = 0

linux ptr = 0x42120000
vectors ptr = 0x4037c000
linux prid = 0x0000abab
E (2649) linux_ipc: intenable = 0x1200320c, system_cpu_intr = 0x00000000, interrupt_core0_intr_status_2 = 0x00000000

E (2649) linux_ipc: esp_shmem_hw_queue_read, r = 0 w = 0

E (3649) linux_ipc: intenable = 0x1200320c, system_cpu_intr = 0x00000000, interrupt_core0_intr_status_2 = 0x00000000

E (3649) linux_ipc: esp_shmem_hw_queue_read, r = 0 w = 0

E (4649) linux_ipc: intenable = 0x1200320c, system_cpu_intr = 0x00000000, interrupt_core0_intr_status_2 = 0x00000000

E (4649) linux_ipc: esp_shmem_hw_queue_read, r = 0 w = 0

E (5649) linux_ipc: intenable = 0x1200320c, system_cpu_intr = 0x00000000, interrupt_core0_intr_status_2 = 0x00000000

E (5649) linux_ipc: esp_shmem_hw_queue_read, r = 0 w = 0

E (6649) linux_ipc: intenable = 0x1200320c, system_cpu_intr = 0x00000000, interrupt_core0_intr_status_2 = 0x00000000

E (6649) linux_ipc: esp_shmem_hw_queue_read, r = 0 w = 0

E (7649) linux_ipc: intenable = 0x1200320c, system_cpu_intr = 0x00000000, interrupt_core0_intr_status_2 = 0x00000000

E (7649) linux_ipc: esp_shmem_hw_queue_read, r = 0 w = 0