intel / ipu6-drivers

GNU General Public License v2.0
177 stars 53 forks source link

OV2740 camera/webcam sensor on X1 Nano Gen2 #55

Open polair opened 2 years ago

polair commented 2 years ago

Hi all. Thank you for supporting this hardware on an open source OS. It's beyond appreciated. I've attempted to build your software/DKMS drivers on Arch linux for the Alder Lake IPU6 with the OV2740 sensor. Using the most recently available git packages, I'm unable to load the intel-ipu6-psys and intel-ipu6-isys modules. modprobe for intel-ipu6 itself loads the module and after this I'm able to load the ov2740 module but not with the psys/isys modules.

❯ sudo dmesg | grep -i -e 'ipu|mei|ov2|vsc|v4l' | egrep -v audit                            Tue 18 Oct 2022 12:36:55 PM MDT
egrep: warning: egrep is obsolescent; using grep -E
[    0.011252] ACPI: SSDT 0x000000008D828000 000150 (v02 LENOVO IpuSsdt  00001000 INTL 20200717)
[   10.671738] v4l2loopback driver version 0.12.7 loaded
[   11.812411] mei_me 0000:00:16.0: enabling device (0000 -> 0002)
[   13.856510] mei_hdcp 0000:00:16.0-b638ab7e-94e2-4ea2-a552-d1c54b627f04: bound 0000:00:02.0 (ops i915_hdcp_component_ops [i915])
[   13.864941] mei_pxp 0000:00:16.0-fbf6fcf1-96cf-4e2e-a6a6-1bab8cbe36b1: bound 0000:00:02.0 (ops i915_pxp_tee_component_ops [i915])
[ 4543.903121] intel-ipu6 intel-ipu: enabling device (0000 -> 0002)
[ 4543.903288] intel-ipu6 intel-ipu: Device 0x465d (rev: 0x2)
[ 4543.903317] intel-ipu6 intel-ipu: physical base address 0x603c000000
[ 4543.903318] intel-ipu6 intel-ipu: mapped as: 0x00000000abd82859
[ 4543.903365] intel-ipu6 intel-ipu: IPU in secure mode
[ 4543.903366] intel-ipu6 intel-ipu: IPU secure touch = 0x0
[ 4543.903367] intel-ipu6 intel-ipu: IPU camera mask = 0xff
[ 4543.921040] intel-ipu6 intel-ipu: IPC reset done
[ 4543.921042] intel-ipu6 intel-ipu: cpd file name: intel/ipu6ep_fw.bin
[ 4543.922677] intel-ipu6 intel-ipu: FW version: 20220510
[ 4543.922896] RIP: 0010:ipu_pci_probe.cold+0x3fc/0x930 [intel_ipu6]
[ 4543.923008]  ipu_init+0x3f/0x1000 [intel_ipu6 cb9a573451b991ba9515d26d3199af1054f2a906]
[ 4543.923070] Modules linked in: intel_ipu6(OE+) videobuf2_dma_contig videobuf2_memops videobuf2_v4l2 videobuf2_common uhid rfcomm cmac algif_hash algif_skcipher af_alg bnep snd_seq_dummy snd_hrtimer snd_seq snd_seq_device bridge stp llc wireguard curve25519_x86_64 libchacha20poly1305 chacha_x86_64 poly1305_x86_64 libcurve25519_generic libchacha ip6_udp_tunnel udp_tunnel uinput snd_ctl_led snd_soc_skl_hda_dsp snd_soc_intel_hda_dsp_common spi_ljca(OE) i2c_ljca(OE) snd_soc_hdac_hdmi snd_sof_probes gpio_ljca(OE) spi_nor snd_soc_dmic joydev mousedev mtd iTCO_wdt intel_pmc_bxt iTCO_vendor_support mei_pxp mei_hdcp mei_wdt pmt_telemetry pmt_class intel_rapl_msr intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp coretemp kvm_intel kvm irqbypass crct10dif_pclmul crc32_pclmul crc32c_intel polyval_clmulni polyval_generic gf128mul ghash_clmulni_intel aesni_intel crypto_simd cryptd rapl intel_cstate intel_uncore psmouse pcspkr think_lmi firmware_attributes_class wmi_bmof btusb btrtl btbcm btintel
[ 4543.923140]  btmtk bluetooth ecdh_generic crc16 ljca(OE) snd_hda_codec_hdmi snd_hda_codec_realtek snd_hda_codec_generic snd_sof_pci_intel_tgl snd_sof_intel_hda_common soundwire_intel soundwire_generic_allocation soundwire_cadence snd_sof_intel_hda iwlmvm snd_sof_pci snd_sof_xtensa_dsp snd_sof snd_sof_utils snd_soc_hdac_hda mac80211 snd_hda_ext_core snd_soc_acpi_intel_match snd_soc_acpi soundwire_bus snd_soc_core snd_compress ac97_bus snd_pcm_dmaengine libarc4 snd_hda_intel snd_intel_dspcfg snd_intel_sdw_acpi snd_hda_codec iwlwifi snd_hda_core snd_hwdep snd_pcm cfg80211 spi_intel_pci i2c_i801 spi_intel intel_lpss_pci i2c_smbus snd_timer intel_lpss mei_me idma64 processor_thermal_device_pci processor_thermal_device thunderbolt processor_thermal_rfim ucsi_acpi processor_thermal_mbox typec_ucsi processor_thermal_rapl intel_vsec thinkpad_acpi intel_rapl_common typec igen6_edac roles ledtrig_audio platform_profile rfkill snd mei_vsc(OE) mei soundcore int3403_thermal soc_button_array
[ 4543.923188]  int340x_thermal_zone mac_hid ov2740(OE) v4l2_fwnode tpm_crb v4l2_async intel_skl_int3472_tps68470 tps68470_regulator tpm_tis tpm_tis_core clk_tps68470 tpm intel_skl_int3472_discrete intel_hid rng_core sparse_keymap int3400_thermal acpi_thermal_rel acpi_tad acpi_pad vboxnetflt(OE) vboxnetadp(OE) vboxdrv(OE) v4l2loopback(OE) videodev mc dm_multipath dm_mod sg crypto_user fuse acpi_call(OE) bpf_preload ip_tables x_tables serio_raw atkbd xhci_pci xhci_pci_renesas libps2 wmi vivaldi_fmap i8042 serio zram i915 drm_buddy video drm_display_helper cec ttm intel_agp intel_gtt zfs(POE) zunicode(POE) zzstd(OE) zlua(OE) zavl(POE) icp(POE) zcommon(POE) znvpair(POE) spl(OE) nvme nvme_core nvme_common
[ 4543.923464] RIP: 0010:ipu_pci_probe.cold+0x3fc/0x930 [intel_ipu6]
[ 5370.637306] v4l2loopback driver version 0.12.7 loaded

I'm uncertain what other feedback to provide as with the module hang I've been unable to get further debug information. I would be very happy to provide any feedback you need to troubleshoot.

Also, are you able to comment on the "intel_iommu=off" kernel option? Is this necessary?

Take care.

polair commented 2 years ago

Further updates. I wrote an AUR package and made it available in gitlab that compiles the kernel using your patches and the method described in the repo readme here. FYI, I don't completely understand the current division between this repo and the ivsc-driver, I understand the ivsc code was merged into this repo? The kernel build does not currently apply the changes in that repo.

Booting from this modified kernel and attempting to load the correct modules as described above results in the same hang and the following stack trace can be found in the dmesg logs:

[   10.110640] CPU: 4 PID: 917 Comm: systemd-udevd Tainted: P     U     OE      6.0.2-1-ck-ipu6 #1 06a7f912c08f7a6e2e1745711475640e9abd9eda                                                                                                                                  
[   10.110643] Hardware name: LENOVO 21E8xnano2/21E8xnano2, BIOS N3IET36W (1.16 ) 09/15/2022                                                                                                                                                                                 
[   10.110644] RIP: 0010:ipu                                                                                                                                                                                                                                                 
_pci_probe.cold+0x3fc/0x930 [intel_ipu6]                                                                                                                                                                                                                                     
[   10.110654] Code: 48 85 ff 48 89 fa 0f 84 dd 00 00 00 48 8b bf 10 03 00 00 48 85 ff 0f 84 2d 01 00 00 e8 59 b2 ff ff 48 8b 43 18 48 85 c0 74 11 <48> 8b b8 10 03 00 00 48 85 ff 74 05 e8 3f b2 ff ff 48 8b 53 20 48                                                       
[   10.110655] RSP: 0018:ffffac41411fbb78 EFLAGS: 00010282                                                                                                                                                                                                                   
[   10.110657] RAX: fffffffffffffff4 RBX: ffff910398d05428 RCX: 0000000000000000                                                                                                                                                                                             
[   10.110657] RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000000                                                                                                                                                                                             
[   10.110658] RBP: ffff910382549000 R08: ffff910382549378 R09: 0000000000000024                                                                                                                                                                                             
[   10.110659] R10: 00000000000000b3 R11: 00000000000000b3 R12: ffff9103825490d0                                                                                                                                                                                             
[   10.110659] R13: ffffac4161000000 R14: 00000000fffffff4 R15: ffffac4161000000                                                                                                                                                                                             
[   10.110660] FS:  00007feb9ec58200(0000) GS:ffff910abf300000(0000) knlGS:0000000000000000                                                                                                                                                                                  
[   10.110661] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033                                                                                                                                                                                                             
[   10.110662] CR2: 0000000000000304 CR3: 000000011ad96002 CR4: 0000000000f70ee0                                                                                                                                                                                             
[   10.110663] PKRU: 55555554                                                                                                                                                                                                                                                
[   10.110666] Call Trace:                                                                                                                                                                                                                                                   
[   10.110668]  <TASK>                                                                                                                                                                                                                                                       
[   10.110670]  local_pci_probe+0x42/0x80                                                                                                                                                                                                                                    
[   10.110674]  pci_device_probe+0xc1/0x220                                                                                                                                                                                                                                  
[   10.110676]  ? sysfs_do_create_link_sd+0x6e/0xe0                                                                                                                                                                                                                          
[   10.110679]  really_probe+0xdb/0x380                                                                                                                                                                                                                                      
[   10.110681]  ? pm_runtime_barrier+0x54/0x90                                                                                                                                                                                                                               
[   10.110683]  __driver_probe_device+0x78/0x170                                                                                                                                                                                                                             
[   10.110684]  driver_probe_device+0x1f/0x90                                                                                                                                                                                                                                
[   10.110685]  __driver_attach+0xd5/0x1d0                                                                                                                                                                                                                                   
[   10.110686]  ? __device_attach_driver+0x110/0x110                                                                                                                                                                                                                         
[   10.110687]  bus_for_each_dev+0x88/0xd0                                                                                                                                                                                                                                   
[   10.110690]  bus_add_driver+0x1b2/0x200                                                                                                                                                                                                                                   
[   10.110691]  driver_register+0x8d/0xe0                                                                                                                                                                                                                                    
[   10.110692]  ? 0xffffffffc1128000                                                                                                                                                                                                                                         
[   10.110694]  ipu_init+0x3f/0x1000 [intel_ipu6 7e912da99b00e1af5cbe883c75ad55692ac520d8]                                                                                                                                                                                   
[   10.110700]  do_one_initcall+0x8c/0x250                                                                                                                                                                                                                                   
[   10.110703]  do_init_module+0x4a/0x1e0                                                                                                                                                                                                                                    
[   10.110705]  __do_sys_init_module+0x17e/0x1b0                                                                                                                                                                                                                             
[   10.110706]  do_syscall_64+0x5c/0x90                                                                                                                                                                                                                                      
[   10.110709]  ? syscall_exit_to_user_mode+0x1b/0x40                                                                                                                                                                                                                        
[   10.110711]  ? exc_page_fault+0x74/0x170                                                                                                                                                                                                                                  
[   10.110712]  entry_SYSCALL_64_after_hwframe+0x63/0xcd                                                                                                                                                                                                                     
[   10.110714] RIP: 0033:0x7feb9f5f2eae                                                                                                                                                                                                                                      
[   10.110715] Code: 48 8b 0d dd ee 0c 00 f7 d8 64 89 01 48 83 c8 ff c3 66 2e 0f 1f 84 00 00 00 00 00 90 f3 0f 1e fa 49 89 ca b8 af 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d aa ee 0c 00 f7 d8 64 89 01 48                                                       
[   10.110716] RSP: 002b:00007ffce69c4fc8 EFLAGS: 00000246 ORIG_RAX: 00000000000000af                                                                                                                                                                                        
[   10.110718] RAX: ffffffffffffffda RBX: 000055b3d8c15fc0 RCX: 00007feb9f5f2eae                                                                                                                                                                                             
[   10.110718] RDX: 00007feb9f73e343 RSI: 0000000000052c5f RDI: 000055b3d8d9df90                                                                                                                                                                                             
[   10.110719] RBP: 00007feb9f73e343 R08: 27d4eb2f165667c5 R09: 85ebca77c2b2ae63                                                                                                                                                                                             
[   10.110719] R10: 000000000000c121 R11: 0000000000000246 R12: 0000000000020000                                                                                                                                                                                             
[   10.110720] R13: 000055b3d8cdbd30 R14: 000055b3d8c15fc0 R15: 000055b3d8cdbd90                                                                                                                                                                                             
[   10.110721]  </TASK>                                                                                                                                                                                                                                                      
[   10.110722] Modules linked in: processor_thermal_device_pci processor_thermal_device thunderbolt(+) intel_ipu6(+) fjes(-) acpi_cpufreq(-) processor_thermal_rfim processor_thermal_mbox ucsi_acpi intel_vsec processor_thermal_rapl typec_ucsi intel_rapl_common igen6_ed\
ac typec roles thinkpad_acpi ledtrig_audio platform_profile rfkill i2c_hid_acpi snd i2c_hid soundcore int3403_thermal int340x_thermal_zone soc_button_array ov2740 tpm_crb mac_hid v4l2_fwnode v4l2_async intel_skl_int3472_tps68470 tpm_tis tps68470_regulator tpm_tis_core\
 clk_tps68470 tpm intel_skl_int3472_discrete rng_core intel_hid sparse_keymap int3400_thermal acpi_thermal_rel acpi_tad acpi_pad vboxnetflt(OE) vboxnetadp(OE) vboxdrv(OE) v4l2loopback(OE) videodev mc dm_multipath dm_mod sg crypto_user acpi_call(OE) fuse ip_tables x_ta\
bles xhci_pci serio_raw xhci_pci_renesas atkbd wmi libps2 vivaldi_fmap i8042 serio zram i915 drm_buddy video drm_display_helper cec ttm intel_agp intel_gtt zfs(POE) zunicode(POE) zzstd(OE) zlua(OE)                                                                        
[   10.110750]  zavl(POE) icp(POE) zcommon(POE) znvpair(POE) spl(OE) nvme nvme_core nvme_common                                                                                                                                                                              
[   10.110753] Unloaded tainted modules: pcc_cpufreq():1 fjes():1 acpi_cpufreq():1 pcc_cpufreq():1 acpi_cpufreq():1 fjes():1 pcc_cpufreq():1 acpi_cpufreq():1 fjes():1 acpi_cpufreq():1 fjes():1 pcc_cpufreq():1 pcc_cpufreq():1 acpi_cpufreq():1 fjes():1 pcc_cpufreq():1 a\
cpi_cpufreq():1 pcc_cpufreq():1 acpi_cpufreq():1 acpi_cpufreq():1 pcc_cpufreq():1 pcc_cpufreq():1 acpi_cpufreq():1 pcc_cpufreq():1 acpi_cpufreq():1 pcc_cpufreq():1 acpi_cpufreq():1 pcc_cpufreq():1 acpi_cpufreq():1 pcc_cpufreq():1 acpi_cpufreq():1 acpi_cpufreq():1 pcc_\
cpufreq():1 pcc_cpufreq():1 acpi_cpufreq():1 pcc_cpufreq():1 acpi_cpufreq():1 acpi_cpufreq():1 pcc_cpufreq():1 pcc_cpufreq():1 acpi_cpufreq():1 acpi_cpufreq():1                                                                                                             
[   10.110769] CR2: 0000000000000304                                                                                                                                                                                                                                         
[   10.110771] ---[ end trace 0000000000000000 ]--- 

This appears very similar to the issue described in #48, with the difference being that the IOMMU/gpio patches have been applied.

jwrdegoede commented 2 years ago

I have been working on making the ipu6-driver stack work on a Lenovo Thinkpad X1 Yoga gen 7 (alderlake) with a 6.0 kernel and using the INT3472 mainline kernel driver for power-control.

I have submitted the following fixes for this, I believe these should also fix things not working on the X1 Nano Gen2:

https://github.com/intel/ipu6-drivers/pull/58/ https://github.com/intel/ivsc-driver/pull/24 https://lore.kernel.org/linux-acpi/20221022150722.31787-1-hdegoede@redhat.com/

Note this relies on the mainline kernel itself also being build with the 2 patches from ipu6-drivers/patch. And if you look at the int3472-support-independent-clock-and-LED-gpios-5.17+.patch you will see it adds an entry for the Yoga Gen 7 to the int3472_sensor_configs[] table. You may need to add a copy of that entry with the sensor-module-name used for the main camera on the X1 nano gen2. Note the IR camera is not supported yet.

polair commented 2 years ago

@jwrdegoede

Thanks a lot for the information and reply. I tried to get the camera operational by building the kernel with the patches (applying a custom patch via AUR/PKGBUILD) and then I built your branch as DKMS modules, thinking I was being clever. I did it this way since I ran into issues building the kernel manually with ipu6/ivsc built in, surely a problem with my build script.

That said, compiling your branches as DKMS modules and then loading the patched kernel... I still get a hang on module load. It seems to be when the module tries to load the firmware from the binary blobs. Here is the stack trace:

203   │ [  269.003562] intel-ipu6 intel-ipu: enabling device (0000 -> 0002)
 204   │ [  269.003799] intel-ipu6 intel-ipu: Device 0x465d (rev: 0x2)
 205   │ [  269.003818] intel-ipu6 intel-ipu: physical base address 0x603c000000
 206   │ [  269.003819] intel-ipu6 intel-ipu: mapped as: 0x00000000c02099fe
 207   │ [  269.003908] intel-ipu6 intel-ipu: IPU in secure mode
 208   │ [  269.003909] intel-ipu6 intel-ipu: IPU secure touch = 0x0
 209   │ [  269.003909] intel-ipu6 intel-ipu: IPU camera mask = 0xff
 210   │ [  269.021078] intel-ipu6 intel-ipu: IPC reset done
 211   │ [  269.021079] intel-ipu6 intel-ipu: cpd file name: intel/ipu6ep_fw.bin
 212   │ [  269.022436] intel-ipu6 intel-ipu: FW version: 20220510
 213   │ [  269.022602] BUG: kernel NULL pointer dereference, address: 0000000000000304
 214   │ [  269.022603] #PF: supervisor read access in kernel mode
 215   │ [  269.022604] #PF: error_code(0x0000) - not-present page
 216   │ [  269.022605] PGD 0 P4D 0 
 217   │ [  269.022607] Oops: 0000 [#1] PREEMPT SMP NOPTI
 218   │ [  269.022609] CPU: 0 PID: 4091 Comm: modprobe Tainted: P S   U     OE      6.0.2-1-ck-ipu6 #1 23ce19dd0dc8b7c00e34bf1573e242d25abc113b
 219   │ [  269.022611] Hardware name: LENOVO 21E8xnano2/21E8xnano2, BIOS N3IET36W (1.16 ) 09/15/2022
 220   │ [  269.022611] RIP: 0010:ipu_pci_probe.cold+0x3fc/0x930 [intel_ipu6]
 221   │ [  269.022618] Code: 48 85 ff 48 89 fa 0f 84 dd 00 00 00 48 8b bf 10 03 00 00 48 85 ff 0f 84 2d 01 00 00 e8 59 b2 ff ff 48 8b 43 18 48 85 c0 74 11 <48> 8b b8 10 03 00 00 48 85 ff 74 05 e8 3f b2 ff ff 48 8b 53 20 48
 222   │ [  269.022619] RSP: 0018:ffffade4c0febaf0 EFLAGS: 00010282
 223   │ [  269.022620] RAX: fffffffffffffff4 RBX: ffff9d115c044828 RCX: 0000000000000000
 224   │ [  269.022621] RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000000
 225   │ [  269.022622] RBP: ffff9d114250d000 R08: ffff9d114250d378 R09: 0000000000000024
 226   │ [  269.022622] R10: 0000000000000350 R11: 0000000000000350 R12: ffff9d114250d0d0
 227   │ [  269.022623] R13: ffffade4e7000000 R14: 00000000fffffff4 R15: ffffade4e7000000
 228   │ [  269.022624] FS:  00007f7492e71740(0000) GS:ffff9d187f200000(0000) knlGS:0000000000000000
 229   │ [  269.022625] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
 230   │ [  269.022625] CR2: 0000000000000304 CR3: 000000012156a002 CR4: 0000000000f70ef0
 231   │ [  269.022626] PKRU: 55555554
 232   │ [  269.022626] Call Trace:
 233   │ [  269.022628]  <TASK>
 234   │ [  269.022629]  local_pci_probe+0x42/0x80
 235   │ [  269.022635]  pci_device_probe+0xc1/0x220
 236   │ [  269.022637]  ? sysfs_do_create_link_sd+0x6e/0xe0
 237   │ [  269.022641]  really_probe+0xdb/0x380
 238   │ [  269.022643]  ? pm_runtime_barrier+0x54/0x90
 239   │ [  269.022645]  __driver_probe_device+0x78/0x170
 240   │ [  269.022647]  driver_probe_device+0x1f/0x90
 241   │ [  269.022647]  __driver_attach+0xd5/0x1d0
 242   │ [  269.022649]  ? __device_attach_driver+0x110/0x110
 243   │ [  269.022649]  bus_for_each_dev+0x88/0xd0
 244   │ [  269.022652]  bus_add_driver+0x1b2/0x200
 245   │ [  269.022653]  driver_register+0x8d/0xe0
 246   │ [  269.022654]  ? 0xffffffffc1242000
 247   │ [  269.022655]  ipu_init+0x3f/0x1000 [intel_ipu6 0c4ea451ed18f4e72586c4bb0708b9519a737879]
 248   │ [  269.022659]  do_one_initcall+0x8c/0x250
 249   │ [  269.022663]  do_init_module+0x4a/0x1e0
 250   │ [  269.022666]  __do_sys_init_module+0x17e/0x1b0
 251   │ [  269.022668]  do_syscall_64+0x5c/0x90
 252   │ [  269.022671]  ? syscall_exit_to_user_mode+0x1b/0x40
 253   │ [  269.022673]  ? do_syscall_64+0x6b/0x90
 254   │ [  269.022674]  ? syscall_exit_to_user_mode+0x1b/0x40
 255   │ [  269.022675]  ? do_syscall_64+0x6b/0x90
 256   │ [  269.022676]  ? exc_page_fault+0x74/0x170
 257   │ [  269.022677]  entry_SYSCALL_64_after_hwframe+0x63/0xcd
 258   │ [  269.022680] RIP: 0033:0x7f7492f7eeae
 259   │ [  269.022681] Code: 48 8b 0d dd ee 0c 00 f7 d8 64 89 01 48 83 c8 ff c3 66 2e 0f 1f 84 00 00 00 00 00 90 f3 0f 1e fa 49 89 ca b8 af 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d aa ee 0c 00 f7 d8 64 89 01 48
 260   │ [  269.022681] RSP: 002b:00007ffe16035e28 EFLAGS: 00000246 ORIG_RAX: 00000000000000af
 261   │ [  269.022682] RAX: ffffffffffffffda RBX: 000055ed9c8daee0 RCX: 00007f7492f7eeae
 262   │ [  269.022683] RDX: 000055ed9bb41cb2 RSI: 000000000005028d RDI: 00007f7492a27010
 263   │ [  269.022684] RBP: 000055ed9bb41cb2 R08: 0000000000071000 R09: 85ebca77c2b2ae63
 264   │ [  269.022684] R10: 0000000000004191 R11: 0000000000000246 R12: 0000000000040000
 265   │ [  269.022685] R13: 000055ed9c8dae20 R14: 0000000000000000 R15: 000055ed9c8db260
 266   │ [  269.022686]  </TASK>
 267   │ [  269.022686] Modules linked in: intel_ipu6(OE+) mei_csi(OE) mei_ace(OE) intel_vsc(OE) ccm ip6table_filter xt_comment ip6_tables bridge stp llc uinput spi_ljca(OE) snd_ctl_led snd_soc_skl_hda_dsp joydev snd_soc_intel_hda_dsp_com
       │ mon mousedev i2c_ljca(OE) gpio_ljca(OE) snd_sof_probes snd_soc_hdac_hdmi snd_soc_dmic spi_nor iTCO_wdt intel_pmc_bxt mtd iTCO_vendor_support hid_multitouch mei_hdcp mei_pxp mei_wdt pmt_telemetry pmt_class intel_rapl_msr intel_tcc
       │ _cooling x86_pkg_temp_thermal intel_powerclamp coretemp kvm_intel kvm irqbypass crct10dif_pclmul crc32_pclmul crc32c_intel polyval_clmulni polyval_generic gf128mul ghash_clmulni_intel aesni_intel crypto_simd cryptd rapl intel_cst
       │ ate intel_uncore pcspkr psmouse think_lmi firmware_attributes_class wmi_bmof btusb btrtl btbcm btintel btmtk bluetooth ecdh_generic crc16 ljca(OE) snd_hda_codec_hdmi snd_hda_codec_realtek snd_hda_codec_generic snd_sof_pci_intel_t
       │ gl snd_sof_intel_hda_common soundwire_intel soundwire_generic_allocation
 268   │ [  269.022710]  soundwire_cadence snd_sof_intel_hda snd_sof_pci snd_sof_xtensa_dsp snd_sof snd_sof_utils snd_soc_hdac_hda snd_hda_ext_core snd_soc_acpi_intel_match snd_soc_acpi soundwire_bus snd_soc_core snd_compress ac97_bus snd
       │ _pcm_dmaengine iwlmvm snd_hda_intel snd_intel_dspcfg snd_intel_sdw_acpi snd_hda_codec mac80211 snd_hda_core i2c_i801 spi_intel_pci snd_hwdep libarc4 iwlwifi snd_pcm spi_intel i2c_smbus intel_lpss_pci snd_timer mei_me intel_lpss i
       │ dma64 cfg80211 processor_thermal_device_pci processor_thermal_device processor_thermal_rfim thunderbolt processor_thermal_mbox ucsi_acpi processor_thermal_rapl intel_vsec typec_ucsi intel_rapl_common typec igen6_edac roles thinkp
       │ ad_acpi ledtrig_audio platform_profile rfkill mei_vsc(OE) snd mei i2c_hid_acpi i2c_hid soundcore int3403_thermal int340x_thermal_zone soc_button_array tpm_crb mac_hid ov2740(OE) v4l2_fwnode v4l2_async intel_skl_int3472_tps68470 t
       │ ps68470_regulator tpm_tis clk_tps68470 tpm_tis_core intel_skl_int3472_discrete tpm
 269   │ [  269.022730]  intel_hid rng_core sparse_keymap int3400_thermal acpi_thermal_rel acpi_tad acpi_pad vboxnetflt(OE) vboxnetadp(OE) vboxdrv(OE) v4l2loopback(OE) videodev mc dm_multipath dm_mod i2c_dev sg crypto_user acpi_call(OE) f
       │ use ip_tables x_tables xhci_pci serio_raw xhci_pci_renesas atkbd wmi libps2 vivaldi_fmap i8042 serio zram i915 drm_buddy video drm_display_helper cec ttm intel_agp intel_gtt zfs(POE) zunicode(POE) zzstd(OE) zlua(OE) zavl(POE) icp
       │ (POE) zcommon(POE) znvpair(POE) spl(OE) nvme nvme_core nvme_common
 270   │ [  269.022744] Unloaded tainted modules: pcc_cpufreq():1 acpi_cpufreq():1 pcc_cpufreq():1 pcc_cpufreq():1 acpi_cpufreq():1 acpi_cpufreq():1 acpi_cpufreq():1 acpi_cpufreq():1 acpi_cpufreq():1 pcc_cpufreq():1 acpi_cpufreq():1 acpi_
       │ cpufreq():1 pcc_cpufreq():1 pcc_cpufreq():1 acpi_cpufreq():1 pcc_cpufreq():1 acpi_cpufreq():1 pcc_cpufreq():1 acpi_cpufreq():1 acpi_cpufreq():1 pcc_cpufreq():1 acpi_cpufreq():1 pcc_cpufreq():1 pcc_cpufreq():1 acpi_cpufreq():1 acp
       │ i_cpufreq():1 pcc_cpufreq():1 pcc_cpufreq():1 acpi_cpufreq():1 acpi_cpufreq():1 pcc_cpufreq():1 acpi_cpufreq():1 acpi_cpufreq():1 pcc_cpufreq():1 acpi_cpufreq():1 acpi_cpufreq():1 pcc_cpufreq():1 acpi_cpufreq():1 pcc_cpufreq():1 
       │ pcc_cpufreq():1 acpi_cpufreq():1 acpi_cpufreq():1 acpi_cpufreq():1 acpi_cpufreq():1 pcc_cpufreq():1 acpi_cpufreq():1 pcc_cpufreq():1 acpi_cpufreq():1 pcc_cpufreq():1 acpi_cpufreq():1 pcc_cpufreq():1 acpi_cpufreq():1 pcc_cpufreq()
       │ :1 acpi_cpufreq():1 pcc_cpufreq():1 acpi_cpufreq():1 acpi_cpufreq():1 pcc_cpufreq():1
 271   │ [  269.022769]  acpi_cpufreq():1 acpi_cpufreq():1 acpi_cpufreq():1 pcc_cpufreq():1 acpi_cpufreq():1 pcc_cpufreq():1 acpi_cpufreq():1 pcc_cpufreq():1 acpi_cpufreq():1 acpi_cpufreq():1 acpi_cpufreq():1 acpi_cpufreq():1 pcc_cpufreq(
       │ ):1 pcc_cpufreq():1 acpi_cpufreq():1 acpi_cpufreq():1 pcc_cpufreq():1 pcc_cpufreq():1 acpi_cpufreq():1 acpi_cpufreq():1 pcc_cpufreq():1 acpi_cpufreq():1 acpi_cpufreq():1 pcc_cpufreq():1 acpi_cpufreq():1 acpi_cpufreq():1 acpi_cpuf
       │ req():1 pcc_cpufreq():1 acpi_cpufreq():1 pcc_cpufreq():1 acpi_cpufreq():1 pcc_cpufreq():1 acpi_cpufreq():1 acpi_cpufreq():1 pcc_cpufreq():1 acpi_cpufreq():1 pcc_cpufreq():1 pcc_cpufreq():1 acpi_cpufreq():1 acpi_cpufreq():1 pcc_cp
       │ ufreq():1 pcc_cpufreq():1 acpi_cpufreq():1 pcc_cpufreq():1 acpi_cpufreq():1 pcc_cpufreq():1 acpi_cpufreq():1 pcc_cpufreq():1 acpi_cpufreq():1 pcc_cpufreq():1 acpi_cpufreq():1 pcc_cpufreq():1 acpi_cpufreq():1 acpi_cpufreq():1 pcc_
       │ cpufreq():1 acpi_cpufreq():1 pcc_cpufreq():1 pcc_cpufreq():1 acpi_cpufreq():1
 272   │ [  269.022790]  acpi_cpufreq():1 pcc_cpufreq():1 pcc_cpufreq():1 acpi_cpufreq():1 pcc_cpufreq():1 acpi_cpufreq():1 pcc_cpufreq():1 acpi_cpufreq():1 pcc_cpufreq():1 acpi_cpufreq():1 acpi_cpufreq():1 pcc_cpufreq():1 acpi_cpufreq():
       │ 1 pcc_cpufreq():1 acpi_cpufreq():1 pcc_cpufreq():1 acpi_cpufreq():1 pcc_cpufreq():1 acpi_cpufreq():1 pcc_cpufreq():1 acpi_cpufreq():1 pcc_cpufreq():1 acpi_cpufreq():1 pcc_cpufreq():1 pcc_cpufreq():1 acpi_cpufreq():1 acpi_cpufreq(
       │ ):1 pcc_cpufreq():1 acpi_cpufreq():1 pcc_cpufreq():1 pcc_cpufreq():1 acpi_cpufreq():1 acpi_cpufreq():1 pcc_cpufreq():1 acpi_cpufreq():1 pcc_cpufreq():1 acpi_cpufreq():1 pcc_cpufreq():1 pcc_cpufreq():1 acpi_cpufreq():1 acpi_cpufre
       │ q():1 pcc_cpufreq():1 pcc_cpufreq():1 acpi_cpufreq():1 acpi_cpufreq():1 pcc_cpufreq():1 acpi_cpufreq():1 pcc_cpufreq():1 acpi_cpufreq():1 pcc_cpufreq():1 acpi_cpufreq():1 pcc_cpufreq():1 acpi_cpufreq():1 pcc_cpufreq():1 pcc_cpufr
       │ eq():1 acpi_cpufreq():1 acpi_cpufreq():1 pcc_cpufreq():1 acpi_cpufreq():1
 273   │ [  269.022811]  pcc_cpufreq():1 acpi_cpufreq():1 pcc_cpufreq():1 acpi_cpufreq():1 pcc_cpufreq():1 acpi_cpufreq():1 pcc_cpufreq():1 pcc_cpufreq():1 acpi_cpufreq():1 pcc_cpufreq():1 acpi_cpufreq():1 pcc_cpufreq():1 acpi_cpufreq():1
       │  acpi_cpufreq():1 pcc_cpufreq():1 pcc_cpufreq():1 acpi_cpufreq():1 acpi_cpufreq():1 pcc_cpufreq():1 fjes():1 acpi_cpufreq():1 fjes():1 acpi_cpufreq():1 pcc_cpufreq():1 fjes():1 pcc_cpufreq():1 acpi_cpufreq():1 fjes():1 pcc_cpufre
       │ q():1 acpi_cpufreq():1 fjes():1 acpi_cpufreq():1 pcc_cpufreq():1 acpi_cpufreq():1 fjes():1 pcc_cpufreq():1 acpi_cpufreq():1 fjes():1 acpi_cpufreq():1 pcc_cpufreq():1 fjes():1 pcc_cpufreq():1 acpi_cpufreq():1 pcc_cpufreq():1 acpi_
       │ cpufreq():1 acpi_cpufreq():1 pcc_cpufreq():1 pcc_cpufreq():1 acpi_cpufreq():1 acpi_cpufreq():1 pcc_cpufreq():1 pcc_cpufreq():1 acpi_cpufreq():1 pcc_cpufreq():1 acpi_cpufreq():1 pcc_cpufreq():1 acpi_cpufreq():1 pcc_cpufreq():1 acp
       │ i_cpufreq():1 acpi_cpufreq():1 pcc_cpufreq():1 acpi_cpufreq():1 pcc_cpufreq():1
 274   │ [  269.022832]  acpi_cpufreq():1 pcc_cpufreq():1 acpi_cpufreq():1 pcc_cpufreq():1 acpi_cpufreq():1
 275   │ [  269.022834] CR2: 0000000000000304
 276   │ [  269.022836] ---[ end trace 0000000000000000 ]---
 277   │ [  269.022837] RIP: 0010:ipu_pci_probe.cold+0x3fc/0x930 [intel_ipu6]
 278   │ [  269.022842] Code: 48 85 ff 48 89 fa 0f 84 dd 00 00 00 48 8b bf 10 03 00 00 48 85 ff 0f 84 2d 01 00 00 e8 59 b2 ff ff 48 8b 43 18 48 85 c0 74 11 <48> 8b b8 10 03 00 00 48 85 ff 74 05 e8 3f b2 ff ff 48 8b 53 20 48
 279   │ [  269.022842] RSP: 0018:ffffade4c0febaf0 EFLAGS: 00010282
 280   │ [  269.022843] RAX: fffffffffffffff4 RBX: ffff9d115c044828 RCX: 0000000000000000
 281   │ [  269.022844] RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000000
 282   │ [  269.022844] RBP: ffff9d114250d000 R08: ffff9d114250d378 R09: 0000000000000024
 283   │ [  269.022845] R10: 0000000000000350 R11: 0000000000000350 R12: ffff9d114250d0d0
 284   │ [  269.022845] R13: ffffade4e7000000 R14: 00000000fffffff4 R15: ffffade4e7000000
 285   │ [  269.022846] FS:  00007f7492e71740(0000) GS:ffff9d187f200000(0000) knlGS:0000000000000000
 286   │ [  269.022846] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
 287   │ [  269.022847] CR2: 0000000000000304 CR3: 000000012156a002 CR4: 0000000000f70ef0
 288   │ [  269.022847] PKRU: 55555554
jwrdegoede commented 2 years ago

@polair, those errors look like your build of the ipu6-drivers does not include my patches from #58 .

polair commented 2 years ago

@jwrdegoede Thanks for taking the time to reply. I understand you're definitely not Intel webcam support. I had made a late night mistake in my build script.

I can confirm that your patches allow me to load the modules. Now if only I could figure out my issue with v4I2 opening the camera.. I get the following error when using gst-launch:

 ~/.ca/pa/cl/icamerasrc-git   *...  sudo -E LANG=C gst-launch-1.0 icamerasrc device-name=ov2740-uf ! autovideoconvert ! waylandsink
Setting pipeline to PAUSED ...
[10-25 23:49:04.207] CamHAL[INF] aiqb file name ov13b10.aiqb
[10-25 23:49:04.207] CamHAL[ERR] there is no aiqb file:ov13b10
[10-25 23:49:04.207] CamHAL[INF] aiqb file name ov13b10.aiqb
[10-25 23:49:04.207] CamHAL[ERR] there is no aiqb file:ov13b10
[10-25 23:49:04.207] CamHAL[INF] aiqb file name ov8856.aiqb
[10-25 23:49:04.207] CamHAL[ERR] there is no aiqb file:ov8856
[10-25 23:49:04.207] CamHAL[INF] aiqb file name ov8856.aiqb
[10-25 23:49:04.207] CamHAL[ERR] there is no aiqb file:ov8856
[10-25 23:49:04.207] CamHAL[INF] aiqb file name ov01a10.aiqb
[10-25 23:49:04.207] CamHAL[INF] aiqb file name ov01a10.aiqb
[10-25 23:49:04.207] CamHAL[INF] aiqb file name ov01a10.aiqb
[10-25 23:49:04.207] CamHAL[INF] aiqb file name ov01a10.aiqb
[10-25 23:49:04.207] CamHAL[INF] aiqb file name OV02C10_1BG203N3_ADL.aiqb
[10-25 23:49:04.208] CamHAL[INF] aiqb file name OV02C10_1BG203N3_ADL.aiqb
[10-25 23:49:04.208] CamHAL[INF] aiqb file name OV2740_CJFLE23_ADL.aiqb
[10-25 23:49:04.208] CamHAL[INF] aiqb file name OV2740_CJFLE23_ADL.aiqb
[10-25 23:49:04.208] CamHAL[INF] aiqb file name HM2170_1SG205N3_ADL.aiqb
[10-25 23:49:04.208] CamHAL[INF] aiqb file name HM2170_1SG205N3_ADL.aiqb
[10-25 23:49:04.208] CamHAL[INF] aiqb file name hi556.aiqb
[10-25 23:49:04.208] CamHAL[INF] aiqb file name hi556.aiqb
[10-25 23:49:04.208] CamHAL[INF] aiqb file name ov01a1s.aiqb
[10-25 23:49:04.208] CamHAL[INF] aiqb file name ov01a1s.aiqb
[10-25 23:49:04.208] CamHAL[ERR] Failed to find DevName for cameraId: 7, get video node: ov2740 , devname: /dev/v4l-subdev1
[10-25 23:49:04.210] CamHAL[ERR] MediaControl init failed
Failed to set pipeline to PAUSED.
Setting pipeline to NULL ...
Freeing pipeline ...
0:00:00.255988871 21472 0x562782181e00 ERROR             icamerasrc gstcamerasrc.cpp:2845:gst_camerasrc_start: CameraId=7 failed to open libcamhal device.
0:00:00.293548908 21472 0x562782181e00 WARN              CamBaseSrc gstcambasesrc.cpp:4192:gst_cam_base_src_activate_push:<camerasrc0> Failed to start in push mode
0:00:00.293562272 21472 0x562782181e00 INFO                GST_PADS gstpad.c:1275:activate_mode_internal:<camerasrc0:src> failed to activate in push mode

Which implies that the hal package isn't somehow providing the correct something? I have the following files provided by that package:

» pwd                                                                           Tue 25 Oct 2022 11:52:47 PM MDT
/usr/share/defaults/etc/camera
ace at nanotux in /u/s/d/e/camera
» fd -i ov2740                                                                  Tue 25 Oct 2022 11:52:47 PM MDT
OV2740_CJFLE23_ADL.aiqb
gcss/graph_settings_OV2740_CJFLE23_ADL.xml
sensors/ov2740-uf.xml

When I try to run it with v4l2-relayd I get a different error that I haven't been able to debug, which also implies there is something going on with the gstreamer->hal communication:

Oct 25 23:59:06 nanotux v4l2-relayd[144062]: g_source_remove: assertion 'tag > 0' failed
Oct 25 23:59:06 nanotux v4l2-relayd[144062]: g_source_remove: assertion 'tag > 0' failed
Oct 25 23:59:06 nanotux v4l2-relayd[144062]: gst_element_set_state: assertion 'GST_IS_ELEMENT (element)' failed
Oct 25 23:59:06 nanotux v4l2-relayd[144062]: gst_object_unref: assertion 'object != NULL' failed
Oct 25 23:59:06 nanotux v4l2-relayd[144062]: gst_element_set_state: assertion 'GST_IS_ELEMENT (element)' failed
Oct 25 23:59:06 nanotux v4l2-relayd[144062]: gst_object_unref: assertion 'object != NULL' failed
Oct 25 23:59:06 nanotux systemd[1]: v4l2-relayd.service: Deactivated successfully
jwrdegoede commented 2 years ago

The following works for me (as a regular user):

sudo chmod 777 /dev/ipu-psys0
gst-launch-1.0 icamerasrc buffer-count=7 ! video/x-raw,format=NV12,width=1280,height=720 ! videoconvert ! xvimagesink
gst-launch-1.0 icamerasrc buffer-count=7 ! video/x-raw,format=NV12,width=1920,height=1080 ! videoconvert ! xvimagesink

Note that I do not pass a device-name= argument to icamerasrc.

polair commented 2 years ago

@jwrdegoede

Thanks for continuing to reply! I don't have the device /dev/ipu-psys0 or any other ipu labeled device. The modules are all installed and loaded correctly as far as I can tell:

⏎    lsmod | grep ipu                                                                        25.2s  Wed Oct 26 at 0:23 MDT -06
intel_ipu6_psys       110592  0
intel_ipu6_isys       155648  0
videobuf2_dma_contig    28672  1 intel_ipu6_isys
videobuf2_v4l2         40960  1 intel_ipu6_isys
videobuf2_common       86016  4 videobuf2_dma_contig,videobuf2_v4l2,intel_ipu6_isys,videobuf2_memops
intel_ipu6            135168  2 intel_ipu6_isys,intel_ipu6_psys
v4l2_fwnode            32768  2 intel_ipu6_isys,ov2740
v4l2_async             32768  3 v4l2_fwnode,intel_ipu6_isys,ov2740
videodev              315392  6 v4l2_async,videobuf2_v4l2,v4l2loopback,intel_ipu6_isys,videobuf2_common,ov2740
mc                     73728  7 v4l2_async,videodev,snd_usb_audio,videobuf2_v4l2,intel_ipu6_isys,videobuf2_common,ov2740

Looking at dmesg and grepping for output with "vsc" doesn't show anything. It seems that this is something to do with the vsc modules/firmware from ivsc-driver. I'm both doing the merge described in the ipu6-driver README as well as installing the ivsc-driver package itself. Maybe this creates a conflict?

~ >>> lsmod | grep -i vsc                                                                                                 13:04:21
intel_vsc              16384  2 mei_ace,mei_csi
mei_vsc                49152  0
mei                   172032  11 mei_ace,mei_ace_debug,mei_csi,mei_wdt,mei_hdcp,mei_pxp,mei_vsc,mei_me
jwrdegoede commented 2 years ago

Did you rebuild your main kernel with the IOMMU + int3472-gpio + https://lore.kernel.org/linux-acpi/20221022150722.31787-1-hdegoede@redhat.com/ patches ?

You need the int3472-gpio + my patch to get the sensors to show up in ls /sys/bus/i2c/devices. What is the output of ls /sys/bus/i2c/devices ?

polair commented 2 years ago

@jwrdegoede OK. It works! Deleting previous comment. The key was your final one line patch. I can confirm that your pull request fixes operational issues with the OV2740 sensor on the X1 nano. My procedure:

Build kernel with the three patches, IOMMU, GPIO, and your latt2021. Clone ipu6-drivers with your pull request and apply ivsc patchset as described in README to then build as DKMS, clone ivsc-driver with your pull request and build as DKMS.

Packages required otherwise. intel-ivsc-firmware, intel-ipu6-camera-bin, v4l2-relayd, and icamerasrc.

I'm going to write up the specifics on the Arch linux forum/wiki when I get some time, it's not exactly straightforward. Hopefully more users will mean some other bugs get ironed out, because...

It's definitely not smooth sailing from there, there are issues with the icamerasrc/hal and v4l2-relayd crashed on me twice in an hour. But the camera is on and streaming, which is satisfying. It would be great if I didn't have to carry another laptop for video calls. :|

edit: ok, I figured this out. The udev rules for the hal package wreak havoc on various libraries from the camera-bin, icamerasrc, and hal packages. It seems that there are some build flags necessary but the Arch packages work fine if you just don't install the udev rules.

Interestingly (and maybe I shouldn't dare to provide compliments given the Linux drivers for year+ old hardware are in such a sad state), the quality of the camera feed is better than anything I've seen in a laptop. So kudos to Intel there, from a photographer. The contrast, resolution, and clarity are all very very nice. If only they cared more about Linux/FOSS.

jwrdegoede commented 2 years ago

Interesting that v4l2-relayd works for you, because I have not got that working yet (under Fedora) may I ask how you have installed v4l2-relayed? And can you share your /etc/default/v4l2-relayd file ?

polair commented 2 years ago

@jwrdegoede I installed it via custom PKGBUILD. I can provide it here if it would help. It took some tweaking and understanding of the options:

1) Your v4l2loopback module needs to be modprobed with a matching label and card number. I have the following options in modprobe.d or you can apply them manually:

│ File: /etc/modprobe.d/v4l2loopback.conf
│ options video_nr=1 card_label=Camera exclusive_caps=1

2) My relayd conf file matches this options and looks like:

~/Co/x1/deps  command cat /etc/default/v4l2-relayd                                                  Fri Oct 28 at 10:01 MDT -06
# GStreamer source element name:
VIDEOSRC="icamerasrc"
#SPLASHSRC="filesrc location=/.../splash.png ! pngdec ! imagefreeze num-buffers=4 ! videoscale ! videoconvert"

# Output format, width, height, and frame rate:
FORMAT=NV12
WIDTH=1920
HEIGHT=1080
FRAMERATE=30/1

# Virtual video device name:
CARD_LABEL="Camera"
# Extra options to pass to v4l2-relayd:
#EXTRA_OPTS=-d
polair commented 2 years ago

@jwrdegoede I'm not done here though. I wasn't able to fix the sleep issue. Does suspend work fine for you? It totally borks the state of the ipu modules, even if I unload everything before sleep. If I unload and try to reload them after wake, I get a stack trace similar to the one before I tried your patches.

Weirdly, the only way I can repeat the sleep behaviour but without sleeping to try to debug, is to rmmod the i2c/gpio/ljca modules and then modprobe the full stack starting there up to ipu6/ov2740. Then ipu6_isys hangs and I have to reboot. I spent way too much time rmmodding/modprobing various related modules to see if I could figure out the ordering issue to no avail. I would love to understand better the way kernel/hardware initialization works so I could debug it. But my day job is radically far away from this.

After sleep, the state of the i2/devices all looks the same. /dev/ipu-psys0 is there. But the camera can't be detected.

polair commented 2 years ago

OK. I was finally able to get a stack trace from the sleep issue and then the reloading of the modules. There seems to be some relevant information. Maybe this weekend I'll consider this a hobby and dig through the code a bit to try to understand it:

[ 2720.712975] intel-ipu6-isys intel-ipu6-isys0: removed
[ 2720.896681] intel-ipu6-psys intel-ipu6-psys0: removed
[ 2722.661236] intel-ipu6 intel-ipu: Device 0x465d (rev: 0x2)
[ 2722.661291] intel-ipu6 intel-ipu: physical base address 0x603c000000
[ 2722.661294] intel-ipu6 intel-ipu: mapped as: 0x00000000eda50765
[ 2722.661428] intel-ipu6 intel-ipu: IPU in secure mode
[ 2722.661430] intel-ipu6 intel-ipu: IPU secure touch = 0x0
[ 2722.661432] intel-ipu6 intel-ipu: IPU camera mask = 0xff
[ 2722.680617] intel-ipu6 intel-ipu: IPC reset done
[ 2722.680634] intel-ipu6 intel-ipu: cpd file name: intel/ipu6ep_fw.bin
[ 2722.684287] intel-ipu6 intel-ipu: FW version: 20220510
[ 2722.686681] intel-ipu6 intel-ipu: IPU6-v3 driver version 1.0
[ 2722.777963] general protection fault, probably for non-canonical address 0x7ebd63b7e6d86ca1: 0000 [#1] PREEMPT SMP NOPTI
[ 2722.777971] CPU: 4 PID: 41236 Comm: modprobe Tainted: P S   U     OE      6.0.6-1-ck-ipu6 #2 027f034b4b9e314841f80ea01c4d4d4137c26a5e
[ 2722.777977] Hardware name: LENOVO 21E8xnano2/21E8xnano2, BIOS N3IET36W (1.16 ) 09/15/2022
[ 2722.777980] RIP: 0010:software_node_graph_get_port_parent+0x3e/0xb0
[ 2722.777991] Code: ff ff 77 0a 48 81 7f 08 80 60 58 b3 74 6e 31 ff 48 8b 9f c0 00 00 00 48 85 db 74 42 48 8b 83 80 00 00 00 48 c7 c6 95 8d 8f b3 <48> 8b 38 e8 ca dc e0 ff 85 c0 74 31 48 89 df 48 83 c7 40 74 4e 48
[ 2722.777993] RSP: 0018:ffffbc00c479bad0 EFLAGS: 00010286
[ 2722.777997] RAX: 7ebd63b7e6d86ca1 RBX: ffffa0a10c960500 RCX: 0000000000000001
[ 2722.778000] RDX: ffffffffb3586080 RSI: ffffffffb38f8d95 RDI: ffffa0a10c961600
[ 2722.778001] RBP: ffffbc00c479bbc0 R08: ffffbc00c479bac8 R09: 0000000000000040
[ 2722.778003] R10: ffffffffc115f4e0 R11: ffffbc00c479b878 R12: ffffa09faad8a740
[ 2722.778004] R13: ffffa09f812c70d0 R14: ffffbc00c479bb30 R15: 0000000000000001
[ 2722.778006] FS:  00007fc319b33740(0000) GS:ffffa0a6bf300000(0000) knlGS:0000000000000000
[ 2722.778009] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 2722.778011] CR2: 00007fc319736000 CR3: 000000022d60e004 CR4: 0000000000f70ee0
[ 2722.778013] PKRU: 55555554
[ 2722.778015] Call Trace:
[ 2722.778018]  <TASK>
[ 2722.778020]  fwnode_graph_get_port_parent+0x65/0xb0
[ 2722.778025]  fwnode_graph_get_remote_port_parent+0x41/0x90
[ 2722.778030]  v4l2_async_nf_parse_fwnode_endpoints+0x115/0x2ef [v4l2_fwnode 249a5ff996bbd5017720200450701a93aaae250c]
[ 2722.778040]  ? 0xffffffffc114d000
[ 2722.778046]  isys_probe+0x6ab/0x880 [intel_ipu6_isys 243b0c07cab5d2b8251fc8e8432f3612b730ec7e]
[ 2722.778067]  ipu_bus_probe+0x48/0xe0 [intel_ipu6 6e6f4570154ac967bc7f464be53d0742f2a7abec]
[ 2722.778081]  really_probe+0xdb/0x380
[ 2722.778086]  ? pm_runtime_barrier+0x54/0x90
[ 2722.778090]  __driver_probe_device+0x78/0x170
[ 2722.778093]  driver_probe_device+0x1f/0x90
[ 2722.778096]  __driver_attach+0xd5/0x1d0
[ 2722.778100]  ? __device_attach_driver+0x110/0x110
[ 2722.778103]  bus_for_each_dev+0x88/0xd0
[ 2722.778110]  bus_add_driver+0x1b2/0x200
[ 2722.778113]  driver_register+0x8d/0xe0
[ 2722.778117]  ? 0xffffffffc0f9b000
[ 2722.778119]  do_one_initcall+0x8c/0x250
[ 2722.778127]  do_init_module+0x4a/0x1e0
[ 2722.778132]  __do_sys_init_module+0x17e/0x1b0
[ 2722.778137]  do_syscall_64+0x5c/0x90
[ 2722.778146]  ? syscall_exit_to_user_mode+0x1b/0x40
[ 2722.778149]  ? do_syscall_64+0x6b/0x90
[ 2722.778154]  ? exc_page_fault+0x74/0x170
[ 2722.778157]  entry_SYSCALL_64_after_hwframe+0x63/0xcd
[ 2722.778163] RIP: 0033:0x7fc319c40eae
[ 2722.778166] Code: 48 8b 0d dd ee 0c 00 f7 d8 64 89 01 48 83 c8 ff c3 66 2e 0f 1f 84 00 00 00 00 00 90 f3 0f 1e fa 49 89 ca b8 af 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d aa ee 0c 00 f7 d8 64 89 01 48
[ 2722.778169] RSP: 002b:00007ffec19b5738 EFLAGS: 00000246 ORIG_RAX: 00000000000000af
[ 2722.778172] RAX: ffffffffffffffda RBX: 0000556e45e672c0 RCX: 00007fc319c40eae
[ 2722.778174] RDX: 0000556e44b5ccb2 RSI: 000000000006abd5 RDI: 00007fc3196cc010
[ 2722.778175] RBP: 0000556e44b5ccb2 R08: 000000000008b000 R09: 85ebca77c2b2ae63
[ 2722.778177] R10: 00000000000176e1 R11: 0000000000000246 R12: 0000000000040000
[ 2722.778179] R13: 0000556e45e67070 R14: 0000000000000000 R15: 0000556e45e68690
[ 2722.778182]  </TASK>
[ 2722.778183] Modules linked in: intel_ipu6_isys(OE+) intel_ipu6(OE) mei_csi(OE) mei_ace(OE) intel_vsc(OE) v4l2loopback(OE) videobuf2_dma_contig v4l2_fwnode v4l2_async videobuf2_memops videobuf2_v4l2 videobuf2_common videodev mc wireguard curve25519_x86_64 libchacha20poly1305 chacha_x86_64 poly1305_x86_64 libcurve25519_generic libchacha ip6_udp_tunnel udp_tunnel rfcomm uhid cmac algif_hash algif_skcipher af_alg bnep snd_seq_dummy snd_hrtimer snd_seq snd_seq_device ccm bridge stp llc uinput joydev snd_ctl_led mousedev snd_soc_skl_hda_dsp snd_soc_intel_hda_dsp_common snd_sof_probes snd_soc_hdac_hdmi i2c_ljca(OE) spi_ljca(OE) gpio_ljca(OE) snd_soc_dmic spi_nor hid_multitouch mtd iTCO_wdt intel_pmc_bxt iTCO_vendor_support mei_pxp mei_hdcp mei_wdt pmt_telemetry intel_rapl_msr pmt_class intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp coretemp kvm_intel kvm irqbypass crct10dif_pclmul crc32_pclmul crc32c_intel polyval_clmulni polyval_generic gf128mul ghash_clmulni_intel aesni_intel
[ 2722.778251]  crypto_simd cryptd rapl intel_cstate intel_uncore pcspkr psmouse think_lmi wmi_bmof firmware_attributes_class btusb btrtl btbcm btintel btmtk bluetooth ecdh_generic ljca(OE) crc16 snd_hda_codec_hdmi snd_hda_codec_realtek snd_hda_codec_generic snd_sof_pci_intel_tgl snd_sof_intel_hda_common soundwire_intel soundwire_generic_allocation soundwire_cadence snd_sof_intel_hda snd_sof_pci snd_sof_xtensa_dsp snd_sof snd_sof_utils snd_soc_hdac_hda snd_hda_ext_core snd_soc_acpi_intel_match snd_soc_acpi soundwire_bus snd_soc_core snd_compress ac97_bus snd_pcm_dmaengine iwlmvm snd_hda_intel snd_intel_dspcfg mac80211 snd_intel_sdw_acpi snd_hda_codec libarc4 snd_hda_core snd_hwdep iwlwifi spi_intel_pci snd_pcm i2c_i801 mei_me spi_intel intel_lpss_pci i2c_smbus snd_timer mei intel_lpss idma64 cfg80211 processor_thermal_device_pci processor_thermal_device ucsi_acpi processor_thermal_rfim thunderbolt typec_ucsi processor_thermal_mbox processor_thermal_rapl typec intel_vsec intel_rapl_common
[ 2722.778312]  igen6_edac roles thinkpad_acpi ledtrig_audio platform_profile rfkill i2c_hid_acpi i2c_hid snd soundcore int3403_thermal int340x_thermal_zone soc_button_array mac_hid intel_skl_int3472_tps68470 tpm_crb tps68470_regulator clk_tps68470 tpm_tis tpm_tis_core intel_skl_int3472_discrete tpm rng_core intel_hid sparse_keymap int3400_thermal acpi_thermal_rel acpi_tad acpi_pad vboxnetflt(OE) vboxnetadp(OE) vboxdrv(OE) dm_multipath dm_mod i2c_dev sg crypto_user acpi_call(OE) fuse ip_tables x_tables xhci_pci serio_raw xhci_pci_renesas wmi atkbd libps2 vivaldi_fmap i8042 serio zram i915 drm_buddy video drm_display_helper cec ttm intel_agp intel_gtt zfs(POE) zunicode(POE) zzstd(OE) zlua(OE) zavl(POE) icp(POE) zcommon(POE) znvpair(POE) spl(OE) nvme nvme_core nvme_common
[ 2722.778372] Unloaded tainted modules: intel_vsc(OE):1 mei_csi(OE):1 mei_ace(OE):1 intel_ipu6(OE):1 intel_ipu6_psys(OE):1 intel_ipu6_isys(OE):1 ov2740(OE):1 v4l2loopback(OE):1 [last unloaded: intel_vsc(OE)]
[ 2722.778388] ---[ end trace 0000000000000000 ]---
[ 2722.778390] RIP: 0010:software_node_graph_get_port_parent+0x3e/0xb0
[ 2722.778395] Code: ff ff 77 0a 48 81 7f 08 80 60 58 b3 74 6e 31 ff 48 8b 9f c0 00 00 00 48 85 db 74 42 48 8b 83 80 00 00 00 48 c7 c6 95 8d 8f b3 <48> 8b 38 e8 ca dc e0 ff 85 c0 74 31 48 89 df 48 83 c7 40 74 4e 48
[ 2722.778397] RSP: 0018:ffffbc00c479bad0 EFLAGS: 00010286
[ 2722.778400] RAX: 7ebd63b7e6d86ca1 RBX: ffffa0a10c960500 RCX: 0000000000000001
[ 2722.778401] RDX: ffffffffb3586080 RSI: ffffffffb38f8d95 RDI: ffffa0a10c961600
[ 2722.778403] RBP: ffffbc00c479bbc0 R08: ffffbc00c479bac8 R09: 0000000000000040
[ 2722.778404] R10: ffffffffc115f4e0 R11: ffffbc00c479b878 R12: ffffa09faad8a740
[ 2722.778406] R13: ffffa09f812c70d0 R14: ffffbc00c479bb30 R15: 0000000000000001
[ 2722.778407] FS:  00007fc319b33740(0000) GS:ffffa0a6bf300000(0000) knlGS:0000000000000000
[ 2722.778409] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 2722.778411] CR2: 00007fc319736000 CR3: 000000022d60e004 CR4: 0000000000f70ee0
[ 2722.778413] PKRU: 55555554
[ 2722.963981] intel-ipu6-psys intel-ipu6-psys0: pkg_dir entry count:8
[ 2722.964443] intel-ipu6-psys intel-ipu6-psys0: psys probe minor: 0
RoXXoR commented 1 year ago

I'm now in the same boat (X1 Nano Gen2 without working camera). I'm currently trying with 6.2.2 kernel,

But gst-launch-1.0 icamerasrc buffer-count=7 ! video/x-raw,format=NV12,width=1280,height=720 ! videoconvert ! waylandsink can't find a proper source device (CamHAL[ERR] Failed to find DevName for cameraId: 0, get video node: ov13b10 , devname: /dev/v4l-subdev1).

Now here comes now a new error message I was not able to find anything about in this context yet. When I load the ov2740 module I get:

[ 2903.745031] ov2740 i2c-INT3474:01: error -EINVAL: no link frequencies defined [ 2903.745040] ov2740 i2c-INT3474:01: error -EINVAL: failed to check HW configuration [ 2903.745141] ov2740: probe of i2c-INT3474:01 failed with error -22

Any hint in which direction I could further loot at?

Thanks in advance

RoXXoR commented 1 year ago

Found the issue. I was loading the ov2740 from the mainline kernel, and not from the ipu6 driver (installed via dkms). Removed the ov2740.ko from the mainline kernel and now I can at least see a video stream via gst-launch

polair commented 1 year ago

@RoXXoR I made the attempt to update to the 6.2 kernel recently and couldn't get it to work. Can you describe exactly how you 'removed' ov2740.ko' from the mainline kernel in more detail? Thanks much.

RoXXoR commented 1 year ago

@polair initially brute force renamed /lib/modules/6.2.2-ipu6mod/kernel/drivers/media/i2c/ov2740.ko to ov2740.ko.old and thus modprobe loaded ov2740.ko from /lib/modules/6.2.3-ipu6mod/updates/drivers/media/i2c/ (dkms build modules). A simple modinfo ov2740 should have show me from the beginning that it was loading the wrong .ko Now I removed the module from the kernel config (not setting CONFIG_VIDEO_OV2740). I'm now running on 6.2.3 have gst-launch showing the video but need to figure out what the issue with v4l2-relayd is.

Cheers.

polair commented 1 year ago

@RoXXoR great, thanks much!

jwrdegoede commented 1 year ago

need to figure out what the issue with v4l2-relayd is.

v4l2-relayd needs a patched v4l2-loopback which send events when the other end of the loopback-device is opened/closed.

@smallorange has been working on getting the necessary changes merged into v4l2-loopback upstream (https://github.com/umlaeute/v4l2loopback).

The changes are upstream now but the latest v4l2-loopback code introduces some new issues with v4l2-relayd. According to: https://github.com/rpmfusion/v4l2loopback-kmod/pull/2

https://github.com/umlaeute/v4l2loopback/commit/a66968647843f57448b59cf98d0318f1e98e072c is the v4l2-loopback commit to use to get a working v4l2-relayd. At least until the issues with the latest latest v4l2-loopback changes are resolved.

almereyda commented 1 year ago

Is there a generic way around the observed relayd issue? I'm seeing the same issue on a ThinkPad X1 Gen 10 with a custom v4l2 loopback module built from the commit above:

g_source_remove: assertion 'tag > 0' failed
g_source_remove: assertion 'tag > 0' failed
gst_element_set_state: assertion 'GST_IS_ELEMENT (element)' failed
gst_object_unref: assertion 'object != NULL' failed
gst_element_set_state: assertion 'GST_IS_ELEMENT (element)' failed
gst_object_unref: assertion 'object != NULL' failed

The commit a669686 that was referenced registers itself to DKMS as version 0.12.7 when doing sudo dkms add . from the checked out source tree. sudo dkms autoinstall v4l2loopback/0.12.7 then works and modinfo -F srcversion v4l2loopback shows 16E224DE4ABFAEC5AE8E4CC. What is it that might be missing here?

This command starts, after indicating a GST plugin path with export GST_PLUGIN_PATH=/usr/lib/gstreamer-1.0/, but cannot find frames at first:

gst-launch-1.0 icamerasrc buffer-count=7 device-name=ov2740-uf ! video/x-raw,format=NV12,width=1280,height=720 ! v4l2sink device=/dev/video1
eitung wird auf PAUSIERT gesetzt ...
[06-12 03:57:35.236] CamHAL[INF] aiqb file name OV2740_CJFLE23_ADL.aiqb
[06-12 03:57:35.237] CamHAL[INF] aiqb file name OV2740_CJFLE23_ADL.aiqb
Leitung ist aktiv und erfordert keinen VORLAUF …
Leitung ist vorgelaufen …
Leitung wird auf ABSPIELEN gesetzt ...
New clock: GstSystemClock

(gst-launch-1.0:17813): GStreamer-CRITICAL **: 03:57:35.297: Registering meta implementation 'GstCamerasrcMeta' without init function
[06-12 03:57:40.477] CamHAL[WAR] <id0>@waitFrame, time out happens, wait recovery
[06-12 03:57:45.477] CamHAL[WAR] <id0>@waitFrame, time out happens, wait recovery
^Chandling interrupt.
Interrupt: Leitung wird gestoppt ...
Execution ended after 0:00:12.908696791
Leitung wird auf NULL gesetzt ...
[06-12 03:57:50.477] CamHAL[WAR] <id0>@waitFrame, time out happens, wait recovery
[06-12 03:57:55.478] CamHAL[WAR] <id0>@waitFrame, time out happens, wait recovery
[06-12 03:58:00.478] CamHAL[WAR] <id0>@waitFrame, time out happens, wait recovery
[06-12 03:58:05.478] CamHAL[WAR] <id0>@waitFrame, time out happens, wait recovery
[06-12 03:58:10.478] CamHAL[WAR] <id0>@waitFrame, time out happens, wait recovery
[06-12 03:58:15.478] CamHAL[WAR] <id0>@waitFrame, time out happens, wait recovery
[06-12 03:58:20.479] CamHAL[WAR] <id0>@waitFrame, time out happens, wait recovery
^C

Subsequent runs show a different error on quitting:

Leitung wird auf PAUSIERT gesetzt ...
[06-12 04:06:55.635] CamHAL[INF] aiqb file name OV2740_CJFLE23_ADL.aiqb
[06-12 04:06:55.641] CamHAL[INF] aiqb file name OV2740_CJFLE23_ADL.aiqb
Leitung ist aktiv und erfordert keinen VORLAUF …
Leitung ist vorgelaufen …
Leitung wird auf ABSPIELEN gesetzt ...
New clock: GstSystemClock

(gst-launch-1.0:39905): GStreamer-CRITICAL **: 04:06:55.686: Registering meta implementation 'GstCamerasrcMeta' without init function
Verzögerung neu verteilen …
^Chandling interrupt.
Interrupt: Leitung wird gestoppt ...
Execution ended after 0:00:09.019229900
Leitung wird auf NULL gesetzt ...
Enter ia_aiq_get_aiqd_data()
Out-aiqd data size: 41480
[06-12 04:07:04.826] CamHAL[WAR] Failed to open file /run/camera/ov2740-uf_VIDEO.aiqd, error Datei oder Verzeichnis nicht gefunden
Leitung wird geleert ...

The AIQD file names are first recognised, and then not anymore.

With relayd not working, this has no chance of working in Cheese or Firefox.

Maybe relayd also needs an equivalent for export GST_PLUGIN_PATH=/usr/lib/gstreamer-1.0/ to tell it where to look for the GStreamer plugins we built along the way? The end of the announcement in https://hansdegoede.livejournal.com/27276.html suggests something like that. gst-launch-1.0 icamerasrc device-name=ov2740-uf buffer-count=7 ! video/x-raw,format=NV12,width=1280,height=720 ! videoconvert ! ximagesink with an X image sink is able to show the camera feed.