indi_v4l2_ccd error

[borisbme]

Hi!

I have been using for some time indi_v4l2_ccd lib on a raspberry with an SQ11 cam for guiding with no problem, but the indi lib was old, and outdated. Yesterday I updated it (compiled from the source) to the latest version, and since than I cannot do a still photo. Live video still works, but when I click on Preview, the driver quits with an error. The error is the following:

2018-07-01T06:06:00: [WARNING] Manual/auto exposure control is not possible on the device! 2018-07-01T06:06:00: [WARNING] Absolute exposure duration control is not possible on the device! 2018-07-01T06:06:00: [INFO] Found intial Input "Camera 1", Format "Motion-JPEG", Size 1280x720, Frame interval 1/30s 2018-07-01T06:06:00: [INFO] V4L2 CCD Device is online. Initializing properties.

2018-07-01T06:06:39: [WARNING] No exposure or streaming in progress 2018-07-01T06:06:39: [WARNING] Failed 1.000-second manual exposure, no adequate control is registered. 2018-07-01T06:06:39: [ERROR] Failed exposing, the absolute exposure duration control is undefined 2018-07-01T06:06:39: [WARNING] Failed 1.000-second manual exposure, no adequate control is registered. 2018-07-01T06:06:39: [ERROR] Failed exposing, the absolute exposure duration control is undefined 2018-07-01T06:06:39: [WARNING] Failed 1.000-second manual exposure, no adequate control is registered.

I also compiled the latest indi drivert on my main pc which is Arch linux based, but the phenomenon is the exact same.

And since this exact same webcan (Sq11) was working with a previous version, I think my setup should be ok, for even the latest version.

log: log.txt

UPDATE:

2018. July 2nd: Just tried another web camera, it produces the same error.

[TallFurryMan]

This situation occurs when the INDI driver cannot locate the V4L2 controls used for exposure. The supported absolute exposure control is either "Exposure (Absolute)" or "Exposure time, Absolute", and the supported manual exposure control is "Exposure, Auto". Your camera probably publishes a different name for this control, but I can't see it in the logs. I'll double check where it should appear.

[TallFurryMan]

If I can still read that part, it's possible that your log have the DEBUG level disabled. Could you capture your scenario again with this enabled? If you used another webcam, same log can be interesting too. It should include the line "Enumerating V4L2 controls".

[borisbme]

Hi!

I got the logs for 3 different cameras(all behaving the way I described it above

). I tried to enable debug on the INDI interface in kstars, but as soon as I clicked on enable the debug was instantly disabled...

2018-07-03T21:47:26: [INFO] Debug is disabled. 2018-07-03T21:47:26: [INFO] Debug is enabled. 2018-07-03T21:47:24: [INFO] Debug is disabled. 2018-07-03T21:47:24: [INFO] Debug is enabled.

https://youtu.be/JreY7Pr_hQg

(or if there is another way please tell me, I have small experience in debugging indi libs)

log_cam01.txt log_cam02.txt log_cam03.txt

[TallFurryMan]

When the driver is controlled by Ekos, the debug level is managed by the dialog which appears when you click "logs" on the main (first) tab.

[borisbme]

Ok, getting there. I hope I get it right this time. Debugging enabled everywhere I could, both kstars and indi logs are attached.

indi_v4l2_ccd_19:50:07.log log_21-49-36.txt

[TallFurryMan]

Nearly, it seems the driver is started with no debug output, then Ekos toggles the right level but the interesting logs have already passed by.

@knro sorry that I can't help right now, how should @borisbme proceed to get the debug enumerations to appear in the log? It seems the debug level is not active right when the driver is connected.

[borisbme]

Any progress on this guys? I had a fully working system 9 days ago that I have upgraded, and now I have a non-functional system. I would really like to help solve this bug, but I'm thinking on downgrading, so I can use my system again.

[TallFurryMan]

My opinion is that you should always have a rollback solution when you upgrade this way. But this opinion is indeed useless for you. My setup is on the other side of the Earth right now, so I can't offer much. I need to see the enumeration of the V4L2 controls in the log, because I suspect the name of the control is different than other cameras. But to determine this, the verbose logs must be enabled in the driver when it is connected, and this is apparently not the case in your logs...

[borisbme]

I think this one has the camera enumeration

If not I'll make a video how I created it and you can aid me to the proper setup.

2018-07-24.txt

[TallFurryMan]

Yes I can see there are zero V4L2 controls enumerated, which is interesting. That may not a dead-end, but there is only MJPG format available and I don't know if that is parseable. Let me check if we can restore what functionality you had in the past.

A few questions: When it was working, what exposure time were you using with that camera? And what was the effective exposure time? Would you, by chance, have old logs of a session? What is the version of your kernel? I'll assume you did not update the system when upgrading indi.

[borisbme]

Hi,

When it was working I was using 1s exposure time.

I have made logs with the latest indi with other two cameras, just to be sure it is not the fault of the camera.

Also I have rolled back to v1.4 and created a debug log. in v1.4 the same camera is working flawlessly.

kernel: 4.14.52-v7+ #1123 SMP Wed Jun 27 17:35:49 BST 2018 armv7l GNU/Linux

Tell me if you need any more info

2018-07-27_latest_cam02.txt 2018-07-27_latest_cam03.txt 2018-07-27_v1.4_cam03.txt

[TallFurryMan]

This is very interesting, thanks a lot. It seems the enumerated controls are not the same in 1.4 and 1.7. I parsed quickly, but I can see both manual and auto exposure controls in your 1.4 log. That should be a great help finding the regression.

[TallFurryMan]

Let me rephrase : I can see the control for cam03, and it has a different name compared the generic one, so I think cam03 could be managed with a patch. Control name appears in 1.4 and 1.7.

Do you have the same log for cam02 in 1.4? Or did I miss something?

[borisbme]

Hi,

No I just made the log with v1.4 with the 3rd cam, I thought it will be sufficient. anyway made logs with the other two cams.

2018-07-27_v1.4_cam01.txt 2018-07-27_v1.4_cam02.txt

Interesting, the 1st cam cannot make a picture, however EKOS not sending any error, just waiting...

As I recall with v1.2 it was working okay.

Let me go back even further, and I'll post a log also with v1.2 for the first cam.

[borisbme]

Yepp, with v1.2 even the 1st cam works okay.

2018-07-27_v1.2_cam01.txt

[TallFurryMan]

Well, "works" is a bit quick here :) with 1.2 on cam01, you end up with 1/30s frames. That's not exactly what you want to achieve eh?

[TallFurryMan]

From what I see in the logs: -- cam01 and cam02 have no exposure control, 1.7 can use them for streaming but not for exposure. 1.2 recovers a 1/30s frame, and 1.4 can't recover anything because V4L2 doesn't advertise MJPG is a compressed format. -- cam03 has exposure control, but the name is different. 1.7 misses the name, thus is limited to streaming. 1.4 recovers a frame in streaming mode, it's not possible to tell the exposure from the log, less than 2s.

Cam03 I can try to patch to get manual exposure working.

Cam01/cam02 I can try to work out something I had in my mind for some time, stacking streamed frames on MJPG cameras, but that's a bit more involved. I know this would be interesting to many, but I don't want to make easy promises.

[TallFurryMan]

@borisbme Could you test PR #662? This may possibly solve your issue with cam03.

For cam01/cam01 I need more time to come up with a solution that doesn't just output a 1/30s frame.

[TallFurryMan]

I think I found something that looks like a regression, that would explain what you observe with cam01/cam02. I'll be reinstating the algorithm which was stacking monochrome frames when no exposure control was available. That got lost during 1.4. I need a bit of time to test this before creating a PR.

[borisbme]

ok, "works" was a quick statement :) My goal is to use that cam guiding, and since cam01 is SQ11, it can be mounted on the finderscope for guiding due its small size. cam03 is a cam I got from a friend for testing this bug, so in some time I have to give it back. For me and the ones using a smaller and/or cheaper cam to guide your cam01/cam02 method would be the best.

And as allways, if you need some testing to be done on those cams, just send me the files and instruction :)

[TallFurryMan]

@borisbme #662 was merged, so if you update your "latest" branch, cam03 might work. Could you test that? And don't forget to provide logs :)

[TallFurryMan]

@borisbme Please test PR #664, that should get your cam01/cam02 back into the game!

Configure the "stack" property to "ADDITIVE" and try an exposure, that will stack frames up. "MEAN" is probably not useful for those non-absolute cameras, except to reduce noise in high-light frames. You can then test "TAKE DARK" to auto-substract a dark to each subsequent frame. You may reset that dark frame with "RESET DARK". You may save that setting in the configuration for automatic use, but the dark is completely manual anyway. Code is non-optimized, and I'm not the original author. I'm just finalizing what I wanted to do last year, and integrating the feature with the rest of the changes. @knro Please wait for @borisbme, and/or test yourself, I think you'll like that PR.

[knro]

@TallFurryMan Great, we'll wait for results. Great work!

[TallFurryMan]

We can probably support Rapsberry Pi too with this, would require tweaking a bit maybe. Idea is that any camera that has limited exposure time could benefit. Also, we could mean frames directly at the driver level by voluntarily limiting available exposure time. But well, no need for this now. Tested with a MJPEG webcam in a low-light environment, very cool results. Single frame had 8 gray levels only, additive stacked for 10 seconds had pretty neat contrast. Not tested on stars though, and the webcam had relatively low noise.

[borisbme]

Hi guys!

I comiled the latest master banch and tried cam03. Unfortunately it did not succeed to make an image. As I saw in the log the reason was: V4L2 CCD: [WARNING] Failed 1.000-second manual exposure, out of device bounds [0.000,0.025].

Shall I proceed on moving to cam01/02 test, or we should short this one out and then move on?

2018-07-29_latest_cam03.txt

[TallFurryMan]

Actually, you should request 0.025s with cam03 because that's indeed what the driver reports as maximum exposure duration.

Please test #664 afterwards with cam01/cam02. Tell me if you need help pulling that PR.

[borisbme]

Yeah, cam03 worked with 0.02s exposure.

Then I switched to PR 664 and compiled indi and done a test with cam01,cam02 it worked okay!

2018-07-29_PR664_cam01.txt 2018-07-29_PR664_cam02.txt

[TallFurryMan]

OK thanks. No it's not working as intended. The actual frame duration is measured with the time it takes to be recovered, not by the time it actually exposes. Let me rework that.

[TallFurryMan]

Updated #664. @borisbme please test with your three cameras if you can. Don't hesitate to try long exposures!

[borisbme]

Hi,

I've done the tests with all the three cameras. Is 10 s expo long enough?

2018-08-06_cam01.txt 2018-08-06_cam02.txt 2018-08-06_cam03.txt

[TallFurryMan]

Thanks! Technically it works, but there's still an incoherence between the exposure requested and the number of frames retrieved: the driver will consider the time it took to gather the frames instead of computing the total exposure time by adding each frame exposure time.

Thus you will theoretically get better results by disabling logs when you expose. There's room for improvement.

And in terms of usability, does the driver now answer your need?

[borisbme]

Hi, for the test made inside I think it works for me. I think this PR can be merged to the main. At the weekend I'll have more time to do a real test with real stars :)

So thanks for the help and support.

[TallFurryMan]

If you are using this camera for guiding, be cautious with the exposure time then. Try to determine whether the value vs exposure curve is linear, and if so what the slope is. Because the driver is not using the duration of each frame to determine how long it actually exposed, there could be surprises.

[TallFurryMan]

Jasem I think we can close this issue.