Open fmolinelli opened 3 years ago
Are you running from the eMMC or a NVMe SSD?
Hi.
booting from eMMC, have a NVMe only for data.
Thank you! Flavio
Il 12/10/2021 03:05, Jetsonhacks ha scritto:
Are you running from the eMMC or a NVMe SSD?
— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/jetsonhacks/buildJetsonXavierKernel/issues/9#issuecomment-940570417, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACCOLIQFJW7CLVRGFAUDJ6LUGOCV7ANCNFSM5FYQO6VQ. Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.
-- Flavio Molinelli
Networking & Security Consulting s.r.l. -www.nesecon.com- Tel. ++39.011.9904163
tel. +39 011 9904163 fax. +39 011 9903581 Corso Tassoni, 14 - 10143 Torino Capitale sociale 10.500,00 € i.v. Iscrizione REA Torino n.986529 Partita iva: IT08611070015
ATTENZIONE!! Il presente messaggio ha contenuto confidenziale, e la sua lettura, allegati compresi, e' riservata esclusivamente ai destinatari previsti. Nel caso riteniate di non essere uno dei destinatari previsti, siete pregati di distruggere il messaggio e di informarne il mittente. Il trattameno dei dati personali è gestito da NSC s.r.l. in conformità con la normativa vigente GDPR Regulation (EU) 2016/679 - Dlgs. 101/2018
What does the rest of your extlinux.conf look like? I would suggest that you take a look at the serial console and see where it is trying to load the Image. Also, check to make sure that the modules are in the correct place.
hi,
/What does the rest of your extlinux.conf look like?/
Pls see listings below
/I would suggest that you take a look at the serial console and see where it is trying to load the Image./
AFAIK the console should be the 4th usbserial device on the left microUSB plug, and should be alsways on, even when the AGX is off (but powered on... ) but I spent two days getting at it wuthout any success: - When I connect my ubuntu ( or macOS) desktop to the left micro-USB plug I don't see any device coming up in dmesg... so no way to monitor the console output from there. - I only get access to the console at the right USB-C (multifunction Flash/debug) as /dev/ttyACM0:115200:8N1, but this seems to be a serial TTY terminal, not a system console, and it is reset at every boot, so I cannot trace the pre-image boot operations. QUESTION: is there an early boot stpe's log written somewhere else?
/Also, check to make sure that the modules are in the correct place. /the modules seems to be in the right place, as they are loaded at boot, generating the error messages seen in dmesg boot sequence.
Also asked support from open-iscsi' maintainer Chris Leech, which told that 'The missing symbols should be enabled with CONFIG_BLK_DEV_BSGLIB'
In fact the booting original kernel Image has that option unset, while my compiled kernel does have it set. That could be the reason the modules aren't working with the original NVIDIA Image
QUESTION: is there another way to boot the compiled Image, like placing it on a USB drive, or microSD card? it would be fine in case of compil errors / mistakes in options settings / etc: simply plug out the device and you'd have your AGX back running with old kernel... ////
//
@.***:~$ cat /boot/extlinux/extlinux.conf TIMEOUT 30 DEFAULT primary
MENU TITLE L4T boot options
LABEL primary MENU LABEL primary kernel LINUX /boot/Image INITRD /boot/initrd APPEND ${cbootargs} quiet root=/dev/mmcblk0p1 rw rootwait rootfstype=ext4 console=ttyTCU0,115200n8 console=tty0 fbcon=map:0 net.ifnames=0 rootfstype=ext4
backup of
device can
#
#
#
#
LABEL backup MENU LABEL backup kernel LINUX /boot/Image.backup INITRD /boot/initrd APPEND ${cbootargs}
LABEL backup
MENU LABEL backup kernel
LINUX /boot/Image.backup
INITRD /boot/initrd
APPEND ${cbootargs}
and here's my /boot tree
@.***:~$ sudo tree /boot /boot ├── dtb │ ├── kernel_tegra194-p2888-0001-p2822-0000.dtb │ └── kernel_tegra194-p2888-0001-p2822-0000.dtb.sig ├── extlinux │ ├── extlinux.conf │ └── extlinux.conf.nv-update-extlinux-backup ├── grub │ └── grubenv ├── Image ├── Image.backup ├── Image.sig ├── Image.t19x ├── Image.t19x.sig ├── initrd ├── initrd.img -> initrd.img-4.9.201-tegra ├── initrd.img-4.9.201-tegra ├── initrd.sig ├── initrd.t19x ├── initrd.t19x.sig ├── tegra186-p3636-0001-p3509-0000-a01-adafruit-sph0645lm4h.dtbo ├── tegra186-p3636-0001-p3509-0000-a01-adafruit-uda1334a.dtbo ├── tegra186-p3636-0001-p3509-0000-a01.dtb ├── tegra186-p3636-0001-p3509-0000-a01-fe-pi-audio.dtbo ├── tegra186-p3636-0001-p3509-0000-a01-hdr40.dtbo ├── tegra186-p3636-0001-p3509-0000-a01-m2ke.dtbo ├── tegra186-p3636-0001-p3509-0000-a01-respeaker-4-mic-array.dtbo ├── tegra186-p3636-0001-p3509-0000-a01-respeaker-4-mic-lin-array.dtbo ├── tegra186-quill-p3310-1000-a00-00-base.dtb ├── tegra186-quill-p3310-1000-as-0888.dtb ├── tegra186-quill-p3310-1000-c03-00-base-adafruit-sph0645lm4h.dtbo ├── tegra186-quill-p3310-1000-c03-00-base-adafruit-uda1334a.dtbo ├── tegra186-quill-p3310-1000-c03-00-base-csi.dtbo ├── tegra186-quill-p3310-1000-c03-00-base.dtb ├── tegra186-quill-p3310-1000-c03-00-base-fe-pi-audio.dtbo ├── tegra186-quill-p3310-1000-c03-00-base-hdr30.dtbo ├── tegra186-quill-p3310-1000-c03-00-base-hdr40.dtbo ├── tegra186-quill-p3310-1000-c03-00-base-m2ke.dtbo ├── tegra186-quill-p3310-1000-c03-00-base-respeaker-4-mic-array.dtbo ├── tegra186-quill-p3310-1000-c03-00-base-respeaker-4-mic-lin-array.dtbo ├── tegra186-quill-p3310-1000-c03-00-dsi-hdmi-dp.dtb ├── tegra186-quill-p3489-0888-a00-00-base.dtb ├── tegra186-quill-p3489-1000-a00-00-ucm1.dtb ├── tegra186-quill-p3489-1000-a00-00-ucm2.dtb ├── tegra194-p2888-0001-p2822-0000-adafruit-sph0645lm4h.dtbo ├── tegra194-p2888-0001-p2822-0000-adafruit-uda1334a.dtbo ├── tegra194-p2888-0001-p2822-0000-csi.dtbo ├── tegra194-p2888-0001-p2822-0000.dtb ├── tegra194-p2888-0001-p2822-0000.dtb.sig ├── tegra194-p2888-0001-p2822-0000-fe-pi-audio.dtbo ├── tegra194-p2888-0001-p2822-0000-hdr40.dtbo ├── tegra194-p2888-0001-p2822-0000-m2ke.dtbo ├── tegra194-p2888-0001-p2822-0000-maxn.dtb ├── tegra194-p2888-0001-p2822-0000-maxn.dtb.sig ├── tegra194-p2888-0001-p2822-0000-respeaker-4-mic-array.dtbo ├── tegra194-p2888-0001-p2822-0000-respeaker-4-mic-lin-array.dtbo ├── tegra194-p2888-0004-e3900-0000-adafruit-sph0645lm4h.dtbo ├── tegra194-p2888-0004-e3900-0000-adafruit-uda1334a.dtbo ├── tegra194-p2888-0004-e3900-0000-csi.dtbo ├── tegra194-p2888-0004-e3900-0000.dtb ├── tegra194-p2888-0004-e3900-0000.dtb.sig ├── tegra194-p2888-0004-e3900-0000-dual-imx274.dtbo ├── tegra194-p2888-0004-e3900-0000-fe-pi-audio.dtbo ├── tegra194-p2888-0004-e3900-0000-hdr40.dtbo ├── tegra194-p2888-0004-e3900-0000-imx274.dtbo ├── tegra194-p2888-0004-e3900-0000-respeaker-4-mic-array.dtbo ├── tegra194-p2888-0004-e3900-0000-respeaker-4-mic-lin-array.dtbo ├── tegra194-p2888-0008-p2822-0000.dtb ├── tegra194-p2888-0008-p2822-0000.dtb.sig ├── tegra194-p2888-as-0006-p2822-0000.dtb ├── tegra194-p2888-as-0006-p2822-0000.dtb.sig ├── tegra194-p2888-as-p3668-p2822-0000.dtb ├── tegra194-p2888-as-p3668-p2822-0000.dtb.sig ├── tegra194-p3668-all-p3509-0000-adafruit-sph0645lm4h.dtbo ├── tegra194-p3668-all-p3509-0000-adafruit-uda1334a.dtbo ├── tegra194-p3668-all-p3509-0000-camera-imx219-dual.dtbo ├── tegra194-p3668-all-p3509-0000-camera-imx477-dual.dtbo ├── tegra194-p3668-all-p3509-0000-camera-imx477-imx219.dtbo ├── tegra194-p3668-all-p3509-0000.dtb ├── tegra194-p3668-all-p3509-0000.dtb.sig ├── tegra194-p3668-all-p3509-0000-fe-pi-audio.dtbo ├── tegra194-p3668-all-p3509-0000-hdr40.dtbo ├── tegra194-p3668-all-p3509-0000-m2ke.dtbo ├── tegra194-p3668-all-p3509-0000-respeaker-4-mic-array.dtbo └── tegra194-p3668-all-p3509-0000-respeaker-4-mic-lin-array.dtbo
3 directories, 81 files
Best Regards Flavio
Il 12/10/2021 19:45, Jetsonhacks ha scritto:
What does the rest of your extlinux.conf look like? I would suggest that you take a look at the serial console and see where it is trying to load the Image. Also, check to make sure that the modules are in the correct place.
— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/jetsonhacks/buildJetsonXavierKernel/issues/9#issuecomment-941232065, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACCOLIWTI4Y5CWM45B5I763UGRX2VANCNFSM5FYQO6VQ. Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.
-- Flavio Molinelli
Networking & Security Consulting s.r.l. -www.nesecon.com- Tel. ++39.011.9904163
tel. +39 011 9904163 fax. +39 011 9903581 Corso Tassoni, 14 - 10143 Torino Capitale sociale 10.500,00 € i.v. Iscrizione REA Torino n.986529 Partita iva: IT08611070015
ATTENZIONE!! Il presente messaggio ha contenuto confidenziale, e la sua lettura, allegati compresi, e' riservata esclusivamente ai destinatari previsti. Nel caso riteniate di non essere uno dei destinatari previsti, siete pregati di distruggere il messaggio e di informarne il mittente. Il trattameno dei dati personali è gestito da NSC s.r.l. in conformità con la normativa vigente GDPR Regulation (EU) 2016/679 - Dlgs. 101/2018
HI,
nope getting it working on Xavier AGX ... but...
it worked on Xavier NX !!!!!
I did exacly the same procedure on NX as I did on AGX, and voilà, NX booted my newly compiled kernel and open-iscsi was up and running:
$ uname -a Linux JXNX01 4.9.253-tegra #1 SMP PREEMPT Thu Oct 14 12:09:37 CEST 2021 aarch64 aarch64 aarch64 GNU/Linux $ systemctl status iscsid ● iscsid.service - iSCSI initiator daemon (iscsid) Loaded: loaded (/lib/systemd/system/iscsid.service; enabled; vendor preset: enabled) Active: active (running) since Thu 2021-10-14 14:34:10 CEST; 3min 16s ago Docs: man:iscsid(8) Main PID: 5651 (iscsid) Tasks: 2 (limit: 4915) CGroup: /system.slice/iscsid.service ├─5647 /sbin/iscsid └─5651 /sbin/iscsid
ott 14 14:34:09 JXNX01 systemd[1]: Starting iSCSI initiator daemon (iscsid)... ott 14 14:34:10 JXNX01 iscsid[5562]: iSCSI logger with pid=5647 started! ott 14 14:34:10 JXNX01 systemd[1]: Started iSCSI initiator daemon (iscsid). ott 14 14:34:11 JXNX01 iscsid[5647]: iSCSI daemon with pid=5651 started!
Now it seems to be an AGX booting related issue... But accordingly to NVIDIA jetson Developer Guide AGX & NX share the same booting flow....
do you have any idea about it?
(the difference between two boards is that AGX is booting from internal eMMC, while NX is booting from microSD ,,, would that matter?)
Hoping to get some hint from you!
Best Regards Flavio
PS Going ondoing the same with a Jetson Nano 2G: I'll report my results!
Il 12/10/2021 19:45, Jetsonhacks ha scritto:
What does the rest of your extlinux.conf look like? I would suggest that you take a look at the serial console and see where it is trying to load the Image. Also, check to make sure that the modules are in the correct place.
— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/jetsonhacks/buildJetsonXavierKernel/issues/9#issuecomment-941232065, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACCOLIWTI4Y5CWM45B5I763UGRX2VANCNFSM5FYQO6VQ. Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.
-- Flavio Molinelli
Networking & Security Consulting s.r.l. -www.nesecon.com- Tel. ++39.011.9904163
tel. +39 011 9904163 fax. +39 011 9903581 Corso Tassoni, 14 - 10143 Torino Capitale sociale 10.500,00 € i.v. Iscrizione REA Torino n.986529 Partita iva: IT08611070015
ATTENZIONE!! Il presente messaggio ha contenuto confidenziale, e la sua lettura, allegati compresi, e' riservata esclusivamente ai destinatari previsti. Nel caso riteniate di non essere uno dei destinatari previsti, siete pregati di distruggere il messaggio e di informarne il mittente. Il trattameno dei dati personali è gestito da NSC s.r.l. in conformità con la normativa vigente GDPR Regulation (EU) 2016/679 - Dlgs. 101/2018
Hi
eventually I found a way to solve this issue: it could be a workaround, it could be a feature.... at least for develpment phase.
That's my last story:
solved the issue with the AGX microUSB console. It seems that internal FDTI chip was suck... while getting it into Force Recovery mode it revived! Now I'm able to get console interaction from my Host PC (seeing it as /dev/ttyUSB3, 115200, 8N1)
Looking at the debug output I noticed some things: 1) C-Boot is looking for boot information at - External SD, - External USB - NVME - Internal eMMC : partitions kernel and kernel_b (dunno why, my AGS is set to boot from _b partitions..)
2) It never look at eMMC partition 1 (APP, where /boot resides)
3) When going to SD, USB, NVME, it looks for an active partition with EXT2/4 and /boot and /boot/extlinux content...
First of all I tried to flash the new kernel from SDK Host
- moved the kernel into
Then got back to my findings and ... cloned the /boot tree from eMMC to nvme as it was (Image and extlinux config uncluded) ... reset AGX, and my iscsi kernel was there!
@.:~$ uname -a Linux BL-AGX01 4.9.253-tegra #1 SMP PREEMPT Mon Oct 11 12:35:16 CEST 2021 aarch64 aarch64 aarch64 GNU/Linux @.:~$ systemctl status iscsid ● iscsid.service - iSCSI initiator daemon (iscsid) Loaded: loaded (/lib/systemd/system/iscsid.service; enabled; vendor preset: enabled) Active: active (running) since Thu 2021-10-14 18:52:56 CEST; 3min 14s ago Docs: man:iscsid(8) Main PID: 6195 (iscsid) Tasks: 2 (limit: 4915) CGroup: /system.slice/iscsid.service ├─6194 /sbin/iscsid └─6195 /sbin/iscsid
ott 14 18:52:56 BL-AGX01 systemd[1]: Starting iSCSI initiator daemon (iscsid)... ott 14 18:52:56 BL-AGX01 iscsid[6169]: iSCSI logger with pid=6194 started! ott 14 18:52:56 BL-AGX01 systemd[1]: Started iSCSI initiator daemon (iscsid). ott 14 18:52:57 BL-AGX01 iscsid[6194]: iSCSI daemon with pid=6195 started! @.***:~$
Conclusion:
- AGX board no more boot from mmcblk0p1/boot, but only from signed images residing into 'kernel' or 'kernel_b' partitions (p29 and p30, could be) - AGX booting partitions can only be flashed from SDK host (if the imgae building tree is OK), to AGX in recovery mode - /boot/Image can be retrieved from microSD, USB, NVME devices, along with extlinux config: no signing is required, if 'security fuse' hasn't been burned yet: this allows for remote update, in place of OTA update, and for testing new kernels without burning them on flash (put a USB drive with /boot dir on it and go testing: if it doesn't work... remove and you have your AGX back working with old kernel...
Just a thought: AGX L4T online update do work completely in place: no SDK host is required... so there's a way to burn the 'kernel' partition with the 'right' image... there's only a need to understand how to do that... here's some reference could help finding it: - https://forums.developer.nvidia.com/t/flashing-a-new-kernel-natively-on-xavier/67559/6 (where are the scripts...?) - https://forums.developer.nvidia.com/t/custom-kernel-without-uboot-solved/65075 (why eMMC /boot is used no more...)
Hoping this would help other developers
Best regards Flavio
--
Flavio Molinelli
Networking & Security Consulting s.r.l. -www.nesecon.com- Tel. ++39.011.9904163
tel. +39 011 9904163 fax. +39 011 9903581 Corso Tassoni, 14 - 10143 Torino Capitale sociale 10.500,00 € i.v. Iscrizione REA Torino n.986529 Partita iva: IT08611070015
ATTENZIONE!! Il presente messaggio ha contenuto confidenziale, e la sua lettura, allegati compresi, e' riservata esclusivamente ai destinatari previsti. Nel caso riteniate di non essere uno dei destinatari previsti, siete pregati di distruggere il messaggio e di informarne il mittente. Il trattameno dei dati personali è gestito da NSC s.r.l. in conformità con la normativa vigente GDPR Regulation (EU) 2016/679 - Dlgs. 101/2018
@fmolinelli I am glad you got something to work.
Without the serial debug kernel, it would have been very difficult indeed to figure out what was going on. I'll leave your issue open so that others may benefit.
Hi, thank you so much for your great work.
Currently I'm trying to enable open-iscsi on NVIDIA Jetson Xavier AGX module, in order to use on it rancher/longhorn distributed storage.
The standard kernel isn't built with open-iscsi support, so I rebuilt it using the following kernel options: CONFIG_CRYPTO=y CONFIG_CRYPTO_CRC32C=y CONFIG_ISCSI_TCP=m CONFIG_SCSI_ISCSI_ATTRS=m CONFIG_SCSI_LOWLEVEL=y CONFIG_BLK_DEV=y CONFIG_INET=y
I thoroughly followed your README and didn't see anything wrong
Kernel and modules build correctly, the board reboots fine with the new kernel, at least it seems so (see later...) but 'modprobe iscsi_tcp' fails: blueadmin@BL-AGX01:/wdir/kernel/buildJetsonXavierKernel$ sudo modprobe iscsi_tcp [sudo] password for blueadmin: modprobe: ERROR: could not insert 'iscsi_tcp': Unknown symbol in module, or unknown parameter (see dmesg): blueadmin@BL-AGX01:/wdir/kernel/buildJetsonXavierKernel$ dmesg | tail [ 4021.062113] scsi_transport_iscsi: Unknown symbol bsg_request_fn (err 0) [ 4423.947586] scsi_transport_iscsi: Unknown symbol bsg_setup_queue (err 0) [ 4423.947789] scsi_transport_iscsi: Unknown symbol bsg_job_done (err 0) bsglib should be included... blueadmin@BL-AGX01:~/wdir/kernel/buildJetsonXavierKernel$ grep BSG /usr/src/kernel/kernel-4.9/.config CONFIG_BLK_DEV_BSG=y CONFIG_BLK_DEV_BSGLIB=y
Here's my Xavier AGX version: blueadmin@BL-AGX01:~$ ./jetsonInfo.py NVIDIA Jetson AGX Xavier [16GB] L4T 32.6.1 [ JetPack 4.6 ] Ubuntu 18.04.6 LTS Kernel Version: 4.9.253-tegra CUDA 10.2.300 CUDA Architecture: 7.2 OpenCV version: 4.1.1 OpenCV Cuda: NO CUDNN: 8.2.1.32 TensorRT: 8.0.1.6 Vision Works: 1.6.0.501 VPI: ii libnvvpi1 1.1.12 arm64 NVIDIA Vision Programming Interface library Vulcan: 1.2.70
Here are the available scsi related kernel modules: blueadmin@BL-AGX01:~$ find /lib/modules/$(uname -r) -type f -name scs /lib/modules/4.9.253-tegra/kernel/drivers/scsi/iscsi_tcp.ko /lib/modules/4.9.253-tegra/kernel/drivers/scsi/libiscsi_tcp.ko /lib/modules/4.9.253-tegra/kernel/drivers/scsi/scsi_transport_iscsi.ko /lib/modules/4.9.253-tegra/kernel/drivers/scsi/scsi_transport_srp.ko /lib/modules/4.9.253-tegra/kernel/drivers/scsi/libiscsi.ko
and here's the iscsi_tcp modinfo: blueadmin@BL-AGX01:~$ modinfo iscsi_tcp filename: /lib/modules/4.9.253-tegra/kernel/drivers/scsi/iscsi_tcp.ko license: GPL description: iSCSI/TCP data-path author: Mike Christie michaelc@cs.wisc.edu, Dmitry Yusupov dmitry_yus@yahoo.com, Alex Aizman itn780@yahoo.com depends: libiscsi,libiscsi_tcp,scsi_transport_iscsi intree: Y vermagic: 4.9.253-tegra SMP preempt mod_unload modversions aarch64 parm: max_lun:uint parm: debug_iscsi_tcp:Turn on debugging for iscsi_tcp module Set to 1 to turn on, and zero to turn off. Default is off. (int)
Thinking I missed some kernel options... or that I failed to install it on the boot dir:
When testing a custom kernel, it is recommended that you create a backup of
the original kernel and add a new entry to this file so that the device can
fallback to the original kernel. To do this:
#
1, Make a backup of the original kernel
sudo cp /boot/Image /boot/Image.backup
#
2, Copy your custom kernel into /boot/Image
#
3, Uncomment below menu setting lines for the original kernel
#
4, Reboot
LABEL backup MENU LABEL backup kernel LINUX /boot/Image.backup INITRD /boot/initrd APPEND ${cbootargs}
my new kernel's compil date should be semothing like MON Oct 11 12:xx:xx .....
Pls can you help me getting the new kernel booted in place of the older one?
Best regards Flavio
PS as per NVIDIA Kernel Customization Guide, the boot loader should load the given Image even if not signed ....