labgrid-project / labgrid

embedded systems control library for development, testing and installation
https://labgrid.readthedocs.io/
Other
313 stars 162 forks source link

Missing serial output at start when using USB loader #1400

Open sjg20 opened 1 month ago

sjg20 commented 1 month ago

Using a sunxi board I find that the initial serial output does not appear with 'labgrid-client console' when sending U-Boot over USB.

I see this:

LG_CROSSBAR=ws://kea:20408/ws labgrid-client -vv -c /vid/software/devel/ubtest/lab/env_rpi_try.cfg -V do-bootstrap 1 -V do-build 1 -V do-send 1 -p pcduino3 -s start -a console 
...
INFO         StepLogger:   ← SunxiUSBDriver.execute() [0.129s]
INFO         StepLogger:  ← UBootWriterDriver.write() [4.261s]
INFO               root:  connecting to NetworkSerialPort(target=Target(name='pcduino3', env=Environment(config_file='/vid/software/devel/ubtest/lab/env_rpi_try.cfg'), var_dict=None), name='USBSerialPort', state=<BindingState.active: 2>, avail=True, host='kea', port=52691, speed=115200, protocol='rfc2217') calling microcom -s 115200 -t kea:52691
connected to 192.168.4.12 (port 52691)
Escape character: Ctrl-\
Type the escape character to get to the prompt.
Unknown monitor
Unknown monitor
In:    serial,usbkbd
Out:   serial,vidconsole
Err:   serial,vidconsole
Net:   eth0: ethernet@1c50000
starting USB...

But if I telnet to kea:52691 I see the full output:

U-Boot SPL 2024.04-00525-gfed663d1f43 (May 10 2024 - 09:20:55 -0600)
DRAM: 1024 MiB
CPU: 912000000Hz, AXI/AHB/APB: 3/2/2
Trying to boot from FEL

U-Boot 2024.04-00525-gfed663d1f43 (May 10 2024 - 09:20:55 -0600) Allwinner Technology

CPU:   Allwinner A20 (SUN7I)
Model: LinkSprite pcDuino3
DRAM:  1 GiB
Core:  58 devices, 24 uclasses, devicetree: separate
WDT:   Not starting watchdog@1c20c90
MMC:   mmc@1c0f000: 0
Loading Environment from FAT... Card did not respond to voltage select! : -110
** Bad device specification mmc 0 **
Unknown monitor
Unknown monitor
...

What could be dropping the initial output? I have the console device enabled before the USB loading starts.

sjg20 commented 1 month ago

Same problem on beaglebone black booting from uSD. This misses even the autoboot prompt, so cannot pass the 'U-Boot' test

U-Boot SPL 2020.04 (May 14 2024 - 07:57:58 -0600)
WDT:   Not found!
Trying to boot from MMC1
Loading Environment from FAT... ** No partition table - mmc 0 **
Loading Environment from MMC... *** Warning - bad CRC, using default environment

U-Boot 2020.04 (May 14 2024 - 07:57:58 -0600)

CPU  : AM335X-GP rev 2.1
Model: TI AM335x BeagleBone Black
DRAM:  512 MiB
MMC:   OMAP SD/MMC: 0, OMAP SD/MMC: 1
Loading Environment from FAT... MMC: no card present
Loading Environment from MMC... *** Warning - bad CRC, using default environment

<ethaddr> not set. Validating first E-fuse MAC
Net:   eth0: ethernet@4a100000
Warning: usb_ether MAC addresses don't match:
Address in ROM is          de:ad:be:ef:00:01
Address in environment is  34:84:e4:e7:f2:ef
, eth1: usb_ether
Press SPACE to abort autoboot in 2 seconds
jluebbe commented 1 month ago

labgrid-client console uses microcom as a telnet/RFC2217 client. That means, that the console is started after the strategy has reached to selected state (here: -s start). So any serial output from the board in the time between the completion of the state transition and microcom connecting is lost.

Usually, this is not a problem, as the strategy leaves the board in a "stable" state (at the bootloader promt, at the linux prompt, powered off, ...). So I'd suggest handling the bootloader autoboot interruption in the Strategy, by activating a UBootDriver with a config matching your UBoot setup.

Alternatively, you can try keeping a labgrid-client console open in second shell and running the setup from the first shell. Depending on your ser2net version, it might support multiple parallel connections.

In the longer term, we'd like to replace microcom with something built on top of the SerialDriver/ConsoleDriver, so that a connection activated earlier can be used and no data is lost. That would also be a prerequisite for triggering actions from labgrid-client console (e.g. cycle power).

sjg20 commented 1 month ago

Thanks for the info.

In fact it turns out that the extreme loss for bbb was due to a serial adaptor failure: [306094.679130] pl2303 ttyUSB14: pl2303_set_control_lines - failed: -32

Re the terminal, perhaps this terminal code from tbot would help?

https://github.com/Rahix/tbot/blob/master/tbot/machine/channel/channel.py#L1019

jluebbe commented 1 month ago

Re the terminal, perhaps this terminal code from tbot would help?

https://github.com/Rahix/tbot/blob/master/tbot/machine/channel/channel.py#L1019

While we can't use that code due to the license, we'll likely need something similar. Internally, it was not urgent enough to find time for this (compared to other things like the move away from crossbar/authbahn). :/

sjg20 commented 3 weeks ago

OK, I implemented a simple internal terminal, which fixes this problem. I will come up with a PR at some point

sjg20 commented 3 weeks ago

See here:

https://github.com/labgrid-project/labgrid/pull/1411/commits/3f6d7235a38888e2a34736c74be7a1ae266ac2a0