Open rolandjitsu opened 5 years ago
Ok, so after trying a few other things, I've discovered that I had a service running which is using I2C (wiringPiI2C) on address 0x51
and 0x50
to get some info from some peripheral and after stopping that, the raspistill cmd seems to work just fine (at least for the amount of time I tried it).
NOTE: I'm also using SPI, but I don't think that should affect the camera.
You're using I2C on which bus? i2c0 or i2c1? i2c0 is the same one that the camera (and DSI display) use, and you can not use it safely from Linux if you are using either of those two functions. There is no inter-processor arbitration for the peripheral. Use of i2c1 should be perfectly safe.
Please add start_debug=1
to config.txt, and then run sudo vcdbg log assert
and sudo vcdbg log msg
after you have had the lockup.
Is this on a custom CM baseboard, or the CMIO? How long are the camera cables? If you get corruption or other issues on the I2C bus to the camera then I can see the GPU waiting for a long time before failing the capture request.
Furthermore, a reboot does not seem to fix the issue, I have to power cycle to restore it to a working state.
That would imply a hardware issue. No software state is retained over a reboot, therefore have you got something funny going on in the hardware with the camera shutdown line or similar?
You're using I2C on which bus? i2c0 or i2c1?
We're using both. 0 for camera and 1 for another peripheral.
i2c0 is the same one that the camera (and DSI display) use, and you can not use it safely from Linux if you are using either of those two functions. There is no inter-processor arbitration for the peripheral.
I2C0 is only used for camera.
Use of i2c1 should be perfectly safe.
Ok.
Please add
start_debug=1
to config.txt, and then runsudo vcdbg log assert
andsudo vcdbg log msg
after you have had the lockup.
I will try that.
Is this on a custom CM baseboard, or the CMIO? How long are the camera cables? If you get corruption or other issues on the I2C bus to the camera then I can see the GPU waiting for a long time before failing the capture request.
Yes, it's a custom board. Standard length (~15cm).
Actually, if we collect data over I2C1 from the other peripheral at the same time we also run the camera, then the raspistill cmd gets stuck. If I reboot and run only the camera, it works just fine.
Furthermore, a reboot does not seem to fix the issue, I have to power cycle to restore it to a working state.
That would imply a hardware issue. No software state is retained over a reboot, therefore have you got something funny going on in the hardware with the camera shutdown line or similar?
Well, it only happens when we cannot power off/reboot properly after the raspistill cmd gets stuck.
Here's our gpio setup:
BANK0 (GPIO 0 to 27):
GPIO 0: level=1 fsel=4 alt=0 func=SDA0
GPIO 1: level=1 fsel=4 alt=0 func=SCL0
GPIO 2: level=0 fsel=1 func=OUTPUT
GPIO 3: level=0 fsel=1 func=OUTPUT
GPIO 4: level=0 fsel=0 func=INPUT
GPIO 5: level=1 fsel=0 func=INPUT
GPIO 6: level=1 fsel=0 func=INPUT
GPIO 7: level=1 fsel=1 func=OUTPUT
GPIO 8: level=1 fsel=1 func=OUTPUT
GPIO 9: level=0 fsel=4 alt=0 func=SPI0_MISO
GPIO 10: level=0 fsel=4 alt=0 func=SPI0_MOSI
GPIO 11: level=0 fsel=4 alt=0 func=SPI0_SCLK
GPIO 12: level=1 fsel=0 func=INPUT
GPIO 13: level=0 fsel=0 func=INPUT
GPIO 14: level=1 fsel=4 alt=0 func=TXD0
GPIO 15: level=1 fsel=4 alt=0 func=RXD0
GPIO 16: level=1 fsel=0 func=INPUT
GPIO 17: level=1 fsel=1 func=OUTPUT
GPIO 18: level=0 fsel=0 func=INPUT
GPIO 19: level=0 fsel=0 func=INPUT
GPIO 20: level=0 fsel=0 func=INPUT
GPIO 21: level=0 fsel=0 func=INPUT
GPIO 22: level=1 fsel=0 func=INPUT
GPIO 23: level=0 fsel=0 func=INPUT
GPIO 24: level=0 fsel=0 func=INPUT
GPIO 25: level=0 fsel=0 func=INPUT
GPIO 26: level=0 fsel=0 func=INPUT
GPIO 27: level=0 fsel=1 func=OUTPUT
BANK1 (GPIO 28 to 45):
GPIO 28: level=0 fsel=0 func=INPUT
GPIO 29: level=0 fsel=0 func=INPUT
GPIO 30: level=0 fsel=0 func=INPUT
GPIO 31: level=0 fsel=0 func=INPUT
GPIO 32: level=0 fsel=0 func=INPUT
GPIO 33: level=0 fsel=0 func=INPUT
GPIO 34: level=1 fsel=0 func=INPUT
GPIO 35: level=1 fsel=0 func=INPUT
GPIO 36: level=1 fsel=0 func=INPUT
GPIO 37: level=0 fsel=0 func=INPUT
GPIO 38: level=0 fsel=0 func=INPUT
GPIO 39: level=0 fsel=0 func=INPUT
GPIO 40: level=0 fsel=0 func=INPUT
GPIO 41: level=0 fsel=0 func=INPUT
GPIO 42: level=0 fsel=0 func=INPUT
GPIO 43: level=0 fsel=0 func=INPUT
GPIO 44: level=1 fsel=6 alt=2 func=SDA1
GPIO 45: level=1 fsel=6 alt=2 func=SCL1
BANK2 (GPIO 46 to 53):
GPIO 46: level=1 fsel=0 func=INPUT
GPIO 47: level=1 fsel=1 func=OUTPUT
GPIO 48: level=1 fsel=4 alt=0 func=SD0_CLK
GPIO 49: level=1 fsel=4 alt=0 func=SD0_CMD
GPIO 50: level=0 fsel=4 alt=0 func=SD0_DAT0
GPIO 51: level=0 fsel=4 alt=0 func=SD0_DAT1
GPIO 52: level=0 fsel=4 alt=0 func=SD0_DAT2
GPIO 53: level=0 fsel=4 alt=0 func=SD0_DAT3
Same issue here! Didn't see this, so I posted on the Raspberry SO first. https://raspberrypi.stackexchange.com/questions/104545/raspberry-camera-v1-2-results-in-kernel-panic
The difference is that I only use the camera. Nothing else. Running the latest raspbian lite on the Pi Zero W, with Cam v1.2 connected and the Bluetoothd service turned off (though that shouldn't be a problem).
Please add
start_debug=1
to config.txt, and then runsudo vcdbg log assert
andsudo vcdbg log msg
after you have had the lockup.
@6by9 getting the following:
519430.890: assert( pass == ISP_TUNER_PASS_NONE ) failed; ../../../../../middleware/ISP/tuner/isp_tuner_brcm_focus.c::master_focus_post_isp_update line 1912 rev 0e6daa5
vcdbg_ctx_get_dump_stack: dump_stack failed
----------------
525715.823: assert( temps->last_tone_mapping_params_valid ) failed; ../../../../../middleware/ISP/tuner/isp_tuner_brcm_agc.c::isp_tuner_brcm_agc_switch_mode line 2745 rev 0e6daa5
vcdbg_ctx_get_dump_stack: dump_stack failed
----------------
525796.896: assert( pass == ISP_TUNER_PASS_NONE ) failed; ../../../../../middleware/ISP/tuner/isp_tuner_brcm_focus.c::master_focus_post_isp_update line 1912 rev 0e6daa5
vcdbg_ctx_get_dump_stack: dump_stack failed
----------------
526009.383: assert( pass == ISP_TUNER_PASS_NONE ) failed; ../../../../../middleware/ISP/tuner/isp_tuner_brcm_focus.c::master_focus_post_isp_update line 1912 rev 0e6daa5
vcdbg_ctx_get_dump_stack: dump_stack failed
----------------
541157.195: assert( temps->last_tone_mapping_params_valid ) failed; ../../../../../middleware/ISP/tuner/isp_tuner_brcm_agc.c::isp_tuner_brcm_agc_switch_mode line 2745 rev 0e6daa5
vcdbg_ctx_get_dump_stack: dump_stack failed
----------------
541244.364: assert( pass == ISP_TUNER_PASS_NONE ) failed; ../../../../../middleware/ISP/tuner/isp_tuner_brcm_focus.c::master_focus_post_isp_update line 1912 rev 0e6daa5
vcdbg_ctx_get_dump_stack: dump_stack failed
----------------
541478.645: assert( pass == ISP_TUNER_PASS_NONE ) failed; ../../../../../middleware/ISP/tuner/isp_tuner_brcm_focus.c::master_focus_post_isp_update line 1912 rev 0e6daa5
vcdbg_ctx_get_dump_stack: dump_stack failed
----------------
588009.923: assert( temps->last_tone_mapping_params_valid ) failed; ../../../../../middleware/ISP/tuner/isp_tuner_brcm_agc.c::isp_tuner_brcm_agc_switch_mode line 2745 rev 0e6daa5
vcdbg_ctx_get_dump_stack: dump_stack failed
----------------
588104.091: assert( pass == ISP_TUNER_PASS_NONE ) failed; ../../../../../middleware/ISP/tuner/isp_tuner_brcm_focus.c::master_focus_post_isp_update line 1912 rev 0e6daa5
vcdbg_ctx_get_dump_stack: dump_stack failed
----------------
588352.472: assert( pass == ISP_TUNER_PASS_NONE ) failed; ../../../../../middleware/ISP/tuner/isp_tuner_brcm_focus.c::master_focus_post_isp_update line 1912 rev 0e6daa5
vcdbg_ctx_get_dump_stack: dump_stack failed
----------------
596598.232: assert( Timed out waiting to destroy ril.camera ) failed; ../../../../../interface/mmal/vc/ril/mmal_ril.c::fail_destroy line 1408 rev 0e6daa5
vcdbg_ctx_get_dump_stack: dump_stack failed
----------------
596808.362: assert( (interrupt_save & (1<<30)) || ((int)_tx_thread_system_state < 0) ) failed; ../../../../../vcfw/rtos/threadx/rtos_threadx_latch.c::rtos_latch_get_real line 99 rev 0e6daa5
vcdbg_ctx_get_dump_stack: dump_stack failed
----------------
and sometimes:
009943.067: assert( edid_state[hdmi_core].i2c_driver ) failed; ../../../../../middleware/hdmi/edid.c::hdmi_edid_shutdown line 736 rev 0e6daa5
vcdbg_ctx_get_dump_stack: dump_stack failed
----------------
024893.679: assert( camera_subsystem_state.power_enable_ref_count[port] > 0 ) failed; ../../../../../vcfw/drivers/device/camera_subsystem/camera_subsystem.c::camera_disable_ctrl line 1141 rev 0e6daa5
vcdbg_ctx_get_dump_stack: dump_stack failed
----------------
002696.160: brfs: File read: /mfs/sd/config.txt
002697.483: brfs: File read: 2019 bytes
002698.513: brfs: File read: /mfs/sd/dt-blob.bin
003061.909: HDMI:EDID error reading EDID block 0 attempt 0
003423.927: HDMI:EDID error reading EDID block 0 attempt 1
003785.942: HDMI:EDID error reading EDID block 0 attempt 2
004147.959: HDMI:EDID error reading EDID block 0 attempt 3
004509.974: HDMI:EDID error reading EDID block 0 attempt 4
004871.991: HDMI:EDID error reading EDID block 0 attempt 5
005234.007: HDMI:EDID error reading EDID block 0 attempt 6
005596.024: HDMI:EDID error reading EDID block 0 attempt 7
005958.039: HDMI:EDID error reading EDID block 0 attempt 8
006320.055: HDMI:EDID error reading EDID block 0 attempt 9
006321.112: HDMI:EDID giving up on reading EDID block 0
006682.197: HDMI:EDID error reading EDID block 0 attempt 0
007044.218: HDMI:EDID error reading EDID block 0 attempt 1
007406.239: HDMI:EDID error reading EDID block 0 attempt 2
007768.259: HDMI:EDID error reading EDID block 0 attempt 3
008130.279: HDMI:EDID error reading EDID block 0 attempt 4
008492.299: HDMI:EDID error reading EDID block 0 attempt 5
008854.319: HDMI:EDID error reading EDID block 0 attempt 6
009216.339: HDMI:EDID error reading EDID block 0 attempt 7
009578.359: HDMI:EDID error reading EDID block 0 attempt 8
009940.379: HDMI:EDID error reading EDID block 0 attempt 9
009941.441: HDMI:EDID giving up on reading EDID block 0
009945.259: brfs: File read: 7150 bytes
009947.498: brfs: File read: /mfs/sd/config.txt
009948.807: gpioman: gpioman_get_pin_num: pin DISPLAY_SDA not defined
009948.878: gpioman: gpioman_get_pin_num: pin LEDS_PWR_OK not defined
009950.462: gpioman: gpioman_get_pin_num: pin BT_ON not defined
009950.521: gpioman: gpioman_get_pin_num: pin WL_ON not defined
009985.562: gpioman: gpioman_get_pin_num: pin LEDS_PWR_OK not defined
009985.658: *** Restart logging
009985.722: brfs: File read: 2019 bytes
010347.354: hdmi: HDMI:EDID error reading EDID block 0 attempt 0
010709.380: hdmi: HDMI:EDID error reading EDID block 0 attempt 1
011071.405: hdmi: HDMI:EDID error reading EDID block 0 attempt 2
011433.430: hdmi: HDMI:EDID error reading EDID block 0 attempt 3
011795.455: hdmi: HDMI:EDID error reading EDID block 0 attempt 4
012157.480: hdmi: HDMI:EDID error reading EDID block 0 attempt 5
012519.505: hdmi: HDMI:EDID error reading EDID block 0 attempt 6
012881.530: hdmi: HDMI:EDID error reading EDID block 0 attempt 7
013243.555: hdmi: HDMI:EDID error reading EDID block 0 attempt 8
013605.580: hdmi: HDMI:EDID error reading EDID block 0 attempt 9
013606.648: hdmi: HDMI:EDID giving up on reading EDID block 0
013967.753: hdmi: HDMI:EDID error reading EDID block 0 attempt 0
014329.775: hdmi: HDMI:EDID error reading EDID block 0 attempt 1
014691.796: hdmi: HDMI:EDID error reading EDID block 0 attempt 2
015053.817: hdmi: HDMI:EDID error reading EDID block 0 attempt 3
015415.840: hdmi: HDMI:EDID error reading EDID block 0 attempt 4
015777.861: hdmi: HDMI:EDID error reading EDID block 0 attempt 5
016139.883: hdmi: HDMI:EDID error reading EDID block 0 attempt 6
016501.905: hdmi: HDMI:EDID error reading EDID block 0 attempt 7
016863.926: hdmi: HDMI:EDID error reading EDID block 0 attempt 8
017225.948: hdmi: HDMI:EDID error reading EDID block 0 attempt 9
017227.011: hdmi: HDMI:EDID giving up on reading EDID block 0
017230.674: hdmi: HDMI:hdmi_get_state is deprecated, use hdmi_get_display_state instead
017231.902: brfs: File read: /mfs/sd/cmdline.txt
017232.003: Read command line from file 'cmdline.txt':
017232.064: 'dwc_otg.lpm_enable=0 console=tty1 root=PARTUUID=d81c1b8f-02 rootfstype=ext4 elevator=deadline fsck.repair=yes rootwait quiet splash plymouth.ignore-serial-consoles'
017232.409: brfs: File read: 164 bytes
017716.781: brfs: File read: /mfs/sd/kernel7.img
017716.849: Loading 'kernel7.img' to 0x8000 size 0x510950
017716.945: No kernel trailer - assuming DT-capable
017717.029: brfs: File read: 5310800 bytes
017721.285: brfs: File read: /mfs/sd/bcm2710-rpi-cm3.dtb
017721.352: Loading 'bcm2710-rpi-cm3.dtb' to 0x518950 size 0x62ad
017858.625: brfs: File read: 25261 bytes
017860.745: brfs: File read: /mfs/sd/config.txt
017861.364: brfs: File read: 2019 bytes
017866.911: brfs: File read: /mfs/sd/overlays/i2c1-bcm2708.dtbo
017882.818: Loaded overlay 'i2c1-bcm2708'
017882.876: dtparam: sda1_pin=44
017883.744: dtparam: scl1_pin=45
017884.525: dtparam: pin_func=6
017885.439: dtparam: i2c_vc=on
017900.071: dtparam: spi=on
017914.517: dtparam: audio=on
017959.277: brfs: File read: 1283 bytes
017963.917: brfs: File read: /mfs/sd/overlays/pi3-disable-bt.dtbo
017978.802: dterror: can't find symbol 'uart0_pins'
017978.817: Failed to resolve overlay 'pi3-disable-bt'
017978.961: dtparam: watchdog=on
017988.918: brfs: File read: 954 bytes
017992.492: brfs: File read: /mfs/sd/overlays/i2c-rtc.dtbo
018015.122: Loaded overlay 'i2c-rtc'
018015.177: dtparam: pcf8523=true
020192.316: Device tree loaded to 0x2eff9800 (size 0x6742)
020194.301: gpioman: gpioman_get_pin_num: pin SDCARD_CONTROL_POWER not defined
021453.908: vchiq_core: vchiq_init_state: slot_zero = 0xef980000, is_master = 1
021458.352: hdmi: HDMI:hdmi_get_state is deprecated, use hdmi_get_display_state instead
021463.798: TV service:host side not connected, dropping notification 0x00000002, 0x00000002, 0x00000055
024905.996: camsubs: Looking for camera 0: i2c_port = 0, led gpio = 2, power enable gpio = 3
025206.780: camsubs: Camera not found
025206.896: camsubs: Looking for camera 0: i2c_port = 0, led gpio = 2, power enable gpio = 3
025507.664: camsubs: Camera not found
025507.771: camsubs: Looking for camera 0: i2c_port = 0, led gpio = 2, power enable gpio = 3
025509.458: camsubs: Camera found OK
025511.569: gpioman: gpioman_get_pin_num: pin FLASH_0_ENABLE not defined
025511.644: gpioman: gpioman_get_pin_num: pin FLASH_0_INDICATOR not defined
025511.743: gpioman: gpioman_get_pin_num: pin FLASH_0_ENABLE not defined
025511.813: gpioman: gpioman_get_pin_num: pin FLASH_0_INDICATOR not defined
120042.611: gpioman: gpioman_get_pin_num: pin FLASH_0_ENABLE not defined
120042.662: gpioman: gpioman_get_pin_num: pin FLASH_0_INDICATOR not defined
120154.846: Using RPI AWB
Another strange thing that I see is that on the i2c 0, where the camera is, there's random values showing up:
0 1 2 3 4 5 6 7 8 9 a b c d e f
00: -- -- -- -- -- -- -- -- -- -- -- -- 0f
10: -- -- -- -- -- -- -- -- -- 19 -- -- -- 1d -- --
20: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
30: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
40: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
50: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
60: -- -- -- -- 64 -- -- -- -- -- -- -- -- -- -- --
70: -- -- -- -- -- -- -- --
The above seems to happen whenever I use the i2c 1. If I stop the service/bin that uses it, then the i2c 0 is stable and raspistill works without issues.
After running a few times the
raspistill -v -o test.jpg -t 1000
cmd just hangs at:This is the log from a prev run:
Here's the drivers I have:
I'm doing this on a CM3 with the following kernel and os:
Furthermore, a reboot does not seem to fix the issue, I have to power cycle to restore it to a working state.