search

AllskyTeam / allsky

A Raspberry Pi operated Wireless Allsky Camera
MIT License
1.19k stars 181 forks source link

RasPi HQ camera night capture gaps #890

Closed jimjackii closed 2 years ago

jimjackii commented 2 years ago

I have problems with night capture with a RasPi HQ camera.

There are gaps of up to 10s between the images. During this time meteors cannot be detected. I also have another system running, and I have already noticed that meteors are being missed.

EricClaeys commented 2 years ago

@jimjackii What's your nighttime delay set to? It determines the gap between images. Older versions of Allsky also had a delay when taking images with the HQ camera. Try upgrading - we just released version 0.8.3 today. See this page.

jimjackii commented 2 years ago

Hi Eric,

thank's for this hint.

I will try thie new version this day/night.

In the older allsky version (v0.8.1) i had a nighttime delay = 10.

EricClaeys commented 2 years ago

@jimjackii Let me know how it goes. Please be careful to follow the instructions at the link I provided. The new version should help. A nighttime delay of 10 is 10 milliseconds - very short. I'm 99% sure your 10 seconds between pictures is due to the artificial delay in the old version,

jimjackii commented 2 years ago

Hi,

I have now tested the new version V0.8.3.

First on DAY with Autoexposure and AutoGain. This makes the pictures a little too dark. Autoexposure is adjusted, but Autogain remains at 1.

Then by NIGHT, but now without auto exposure and without auto gain. Autoexposure and Autogain for DAY are also switched off.

But it looks like the capture is done with Autoexposure. The image brightness fluctuates extremely.

Enclosed are the config files and the log file with debug level 4.

https://jimjackii.de/files/allsky/20220110/settings_RPiHQ.json https://jimjackii.de/files/allsky/20220110/config.sh https://jimjackii.de/files/allsky/20220110/allsky.log

And there still seems to be the gap of 10...12s between the individual images with NIGHT capture.

Images are created with a distance of 60s and the time stamps of the individual images are always 60 + 12s apart.

jimjackii commented 2 years ago

Hi,

I found the problem with the auto exposure.

It was the parameter

CAPTURE_EXTRA_PARAMETERS = "- mean-value 0.5 -autoexposure 1"

in config.sh. I used this because it was recommended.

This activates the autoexposure for DAY and NIGHT, regardless of the settings under "Camera Settings". For me this is misleading and not easy to find.

jimjackii commented 2 years ago

Hi,

I ran the new version last night.

The gap between the individual images of 10 ... 12s is still there. See the time stamp of the individual images.

https://jimjackii.de/files/allsky/20220111/screenshot_allsky_cam_images.jpg

I have another question about autogain in NIGHT mode.

With Autogain = ON I have set the gain = 16. The gain was not changed during the night. Does Autoexposure have to be activated for this? I only had problems with Autoexposure - the brightness fluctuated extremely between the individual images.

I also found that despite Gain = 16 and Autogain = ON, the picture was much darker than with fixed Gain = 6 and Autogain = OFF.

EricClaeys commented 2 years ago

@jimjackii @AndreasLMeg We obviously still have some work with the auto-exposure / auto-gain. Sorry it's confusing. The "-autoexposure 1" turns on night autoexposure. Try removing it from CAPTURE_EXTRA_PARAMETERS and see if it helps.

I just looked at the code. If the "-mean-value" is set, the time between exposures is 1 MS plus whatever your delay is. If "-mean-value" is NOT set but auto-exposure is on, the time between exposure is 10 seconds plus whatever your delay is. That 10 seconds is to give the camera's auto-exposure algorithm time to try and determine the correct exposure. With "-mean-value", we implemented our own auto-exposure algorithm so don't need the extra delay.

jimjackii commented 2 years ago

@EricClaeys Last night I had the following setting in config.sh CAPTURE_EXTRA_PARAMETERS = "- mean-value 0.8" I had already removed the parameter "-autoexposure 1".

In the web settings under "Camera Settings" I had Autoexposure = OFF and Autogain = OFF.

So are still gaps of approx. 11s between the images.

Delay is 10. Is it possible that the Delay time is seconds and not not milliseconds?

AndreasLMeg commented 2 years ago

@jimjackii: if you use "-mean-value" please use AUTOGAIN AND AUTOEXPOSURE

I know i have one issue in the software

jimjackii commented 2 years ago

@AndreasLMeg With parameter "-mean 0.3" or "-mean 0.5" and AUTOGAIN = ON and AUTO EXPOSURE = ON, the pictures were too dark.

With the "-mean 0.8" parameter and AUTOGAIN = OFF and AUTOEXPOSURE = OFF, the images are acceptably bright.

AndreasLMeg commented 2 years ago

@jimjackii: But unfortunately this is my implementation. And I only considered the common use of mean-value with AUTOGAIN and AUTOEXPUSURE. (a different useage was not a usecase for me)

For me only the night pictures were important. and the day pictures were always OK.

Please send me an optimal day picture and the settings for it. As a comparison, a picture with AUTO and meanvalue = 0.8

Please read also: https://github.com/thomasjacquin/allsky/discussions/793

jimjackii commented 2 years ago

@AndreasLMeg Here the files

Manual files with 1ms / 2ms exposure. http://jimjackii.de/files/allsky/20220112/image__manu__1ms.jpg http://jimjackii.de/files/allsky/20220112/image__manu__2ms.jpg

Auto file with mean = 0.8 http://jimjackii.de/files/allsky/20220112/image__auto_mean_08.jpg

AndreasLMeg commented 2 years ago

@jimjackii: Thank you What do you think: is 1ms or 2ms better ? and please make the AUTO Test also with 0.5 and 0.3

Then we can discuse the images and can find a change in the implementation.

jimjackii commented 2 years ago

@AndreasLMeg Here the other files.

Auto file with mean = 0.5 http://jimjackii.de/files/allsky/20220112/image__auto_mean_05.jpg Auto file with mean = 0.3 http://jimjackii.de/files/allsky/20220112/image__auto_mean_03.jpg

I think the best manu file ist the 1ms file.

For the auto files, I think 0.8 is best at the moment.

Unfortunately the sky is mostly cloudy in the last few days. I would repeat the tests again on a sunny day.

jimjackii commented 2 years ago

@AndreasLMeg If I enter the mean value, but leave the other settings off (Autoexposure, Autogain), then these values ​​will be adjusted during the recordings - they both seem to be set to Auto.

AndreasLMeg commented 2 years ago

@jimjackii Before my change (mean algo), only fixed values ​​were possible with the RaspiHQ camera. This led to overexposure in the twilight. There were also problems with the moon and extraneous light from cars.

I wanted to find a solution for this without having to constantly change the parameters. (https://github.com/thomasjacquin/allsky/discussions/793)

And I think it has become an optimal solution for the night. During the day the pictures are a bit too dark, but that didn't bother me as I'm not interested in day shots. Older Allsky versions also had restrictions on the settings (Day/Night, AutoExposure, AutoGain,...)

Hence the recommendation: "-mean-value 0.3 -autoexposure 1" This is also the reason why the settings in config.sh are overwritten. If you don't want this delete "-autoexposure 1"

The feature currently requires autogain and autoexposure to be enabled, if not there are known side effects If you don't want that you have to remove the extra parameters completely

My next development is to support autoGainOnly, AutoexposureOnly and NoAuto If you want this, you have to wait a short time

After this I will look to make the day images better or to switch to "autoexposure" of raspistill/libcamera If you want this, you have to wait a little bit longer

I am sorry that not all wishes will be fulfilled and that it may take some time before changes are made. But this is a free open source project and unfortunately we cannot guarantee everything. Suggestions and error reports are welcome, but for me it's just a hobby and I can only work on it in my free time. But I much prefer to do other things (stargazing, hiking, skiing)

jimjackii commented 2 years ago

@AndreasLMeg I think it's wonderful that you support the project with programming work. And of course it is clear to me that this is a hobby project.

I would also like to help, but I lack the experience for programming. But I can help with tests, as already done.

AndreasLMeg commented 2 years ago

@jimjackii Thank you for your support und testing. Excuse me, but I'm not used to working on open projects and I've burned my fingers a few times. Unfortunately, there is always more confusion after my answers (English is not my mother tongue). I always try to discuss first and then create bugs. I also think Allsky is a great project, but I will leave the administration, planning and support to others.

EricClaeys commented 2 years ago

@jimjackii Helping test is VERY much appreciated. Things might work well for @AndreasLMeg and me (otherwise we wouldn't release an update), but not work for others. Sometimes this is user error, but that tells us we need to make the software more error proof or improve the documentation. That's where testers are useful.

EricClaeys commented 2 years ago

@jimjackii We haven't forgotten about you. We are working on improvements to the exposure algorithm and should have something you can test soon.

@AndreasLMeg FYI

jimjackii commented 2 years ago

@EricClaeys @AndreasLMeg No problem - I can see that a lot of work is being done elsewhere.

EricClaeys commented 2 years ago

@jimjackii Steffen, Are you using the newest Allsky? The daytime and nighttime Mean Target Mean settings in the WebUI replace CAPTURE_EXTRA_PARAMETERS in config.sh so you can set that to "". Auto gain and Auto exposure should both be turned on for best results for day and night. The "max gain" and "max exposure" values aren't implemented yet, but will be in the next release of Allsky.

jimjackii commented 2 years ago

@EricClaeys Thanks for the hint.

Unfortunately, I can't operate the camera at the moment, as work on the roof/chimney can probably take place.

When I can operate the camera again I will test the new software.

EricClaeys commented 2 years ago

Closing - this request was added to the Projects list - see the Projects link at the top of any Allsky GitHub page. Project Title: Decrease RPiHQ gaps between exposures #890 The next version of Allsky (after v2022.03.01) will have a 1 ms delay (the shortest allowed) when the Target Mean is set, and 2 seconds when it's not set during the day and 10 seconds at night. The 2 and 10 are to give the camera driver's auto exposure time to work. We suggest people always use the Target Mean auto exposure/gain method. Note also that there's a non-trivial delay between pictures due to having to invoke libcamera-still or raspistill for every picture.