Starlight Xpress Oculus all-sky camera

[bamiesking]

Hi,

I'd like to get indi-allsky working with a Starlight Xpress Oculus all-sky camera - is there a driver in indi-allsky that would work with this device? Any help greatly appreciated.

Thanks.

[aaronwmorris]

I should be able to get it working, I just need to know the gain setting name that is used for the camera. If you could connect the camera to indi and run indi_getprop on the CLI and send me the output, I can get the gain setting coded in. It only takes a few minutes.

[aaronwmorris]

Just let me know if you are still interested. I do not have a Starlight Xpress camera to test with and I am very interested in making this work.

It really does only take a few minutes to add the correct parameter to the code for the camera to function.

[bamiesking]

Hi Aaron,

Yes I'm still interested. I'll get back to you on this next week. Sorry for the delay!

[aaronwmorris]

No problem! I just did not want to scare you off thinking this was a huge effort.

[bamiesking]

Hi Aaron, just letting you know I am going to come back to this. Various issues (not related to INDI or indi-allsky) have got in the way. Feel free to close the issue and I will reopen it when I can give you the information you need. Thanks.

[aaronwmorris]

No worries. I will be here when you have time!

[rkilgard]

Hi,

I'd love to reopen this issue. I'm running Astroberry on a Pi 3 and can take images with a Starlight Xpress Oculus through KStars/Ekos, so the Indi drivers are definitely working. Below is what I get from indi_getprop.

Cheers, Roy

indi_getprop.txt

[aaronwmorris]

I have merged #189 . Let me know if that works for you.

I did not see any gain settings reported in the indi_getprop output, does that sound correct to you?

[rkilgard]

Yes, it doesn't seem to report gain. I'm not sure what that's about but INDI doesn't seem to care. The manufacturer says the gain is 0.3 e-/ADU.

The new version dies in a different place so that's progress. Here's what I get if I attempt to start indi-allsky manually. Maybe it needs the gain hard-coded?

(indi-allsky) astroberry@astroberry:~/indi-allsky $ systemctl --user start indiserver (indi-allsky) astroberry@astroberry:~/indi-allsky $ source virtualenv/indi-allsky/bin/activate (indi-allsky) astroberry@astroberry:~/indi-allsky $ ./allsky.py run /home/astroberry/indi-allsky/virtualenv/indi-allsky/lib/python3.7/site-packages/paramiko/transport.py:236: CryptographyDeprecationWarning: Blowfish has been deprecated "class": algorithms.Blowfish, 2022-05-04 18:17:56,717 [INFO] MainProcess indi.init() #90: creating an instance of IndiClient 2022-05-04 18:17:56,717 [INFO] MainProcess allsky._initialize() #291: Connecting to indiserver INDI::BaseClient::connectServer: creating new connection... INDI::BaseClient::connectServer: Already connected. 2022-05-04 18:17:56,728 [INFO] MainProcess indi.newDevice() #119: new device SX CCD SuperStar 2022-05-04 18:17:56,720 [INFO] MainProcess indi.serverConnected() #172: Server connected (localhost:7624) 2022-05-04 18:17:56,732 [INFO] MainProcess indi.newDevice() #119: new device Telescope Simulator 2022-05-04 18:17:56,739 [INFO] MainProcess indi.newText() #163: new Text ACTIVE_DEVICES for device Telescope Simulator 2022-05-04 18:17:56,740 [INFO] MainProcess indi.newText() #163: new Text ACTIVE_DEVICES for device SX CCD SuperStar 2022-05-04 18:17:56,741 [INFO] MainProcess indi.newText() #163: new Text SCOPE_CONFIG_NAME for device Telescope Simulator 2022-05-04 18:17:56,741 [INFO] MainProcess indi.newSwitch() #156: new Switch MOUNT_TYPE for device Telescope Simulator 2022-05-04 18:17:56,772 [INFO] MainProcess indi.newSwitch() #156: new Switch SIM_PIER_SIDE for device Telescope Simulator 2022-05-04 18:18:04,739 [INFO] MainProcess indi.findCcds() #307: Found device SX CCD SuperStar 2022-05-04 18:18:04,740 [INFO] MainProcess indi.findCcds() #312: Detected ccd 2022-05-04 18:18:04,740 [INFO] MainProcess indi.findCcds() #312: Detected guider 2022-05-04 18:18:04,741 [INFO] MainProcess indi.findCcds() #307: Found device Telescope Simulator 2022-05-04 18:18:04,741 [INFO] MainProcess indi.findCcds() #312: Detected telescope 2022-05-04 18:18:04,742 [INFO] MainProcess indi.findCcds() #312: Detected guider 2022-05-04 18:18:04,742 [INFO] MainProcess allsky._initialize() #308: Found 1 CCDs 2022-05-04 18:18:04,743 [WARNING] MainProcess allsky._initialize() #311: Connecting to device SX CCD SuperStar 2022-05-04 18:18:04,756 [INFO] MainProcess indi.newText() #163: new Text DRIVER_INFO for device SX CCD SuperStar 2022-05-04 18:18:04,757 [INFO] MainProcess indi.newSwitch() #156: new Switch CONNECTION for device SX CCD SuperStar 2022-05-04 18:18:04,979 [INFO] MainProcess miscDb.addCamera() #43: Camera DB ID: 1 2022-05-04 18:18:04,979 [INFO] MainProcess indi.configureDevice() #327: Setting switch DEBUG 2022-05-04 18:18:05,082 [INFO] MainProcess indi.newSwitch() #156: new Switch DEBUG for device SX CCD SuperStar 2022-05-04 18:18:06,083 [INFO] MainProcess indi.updateCcdBlobMode() #179: Set BLOB mode 2022-05-04 18:18:07,085 [INFO] MainProcess indi.configureDevice() #327: Setting switch CCD_FRAME_TYPE 2022-05-04 18:18:07,188 [INFO] MainProcess indi.newSwitch() #156: new Switch CCD_FRAME_TYPE for device SX CCD SuperStar 2022-05-04 18:18:08,390 [INFO] MainProcess indi.getCcdInfo() #249: Detecting bayer pattern 2022-05-04 18:18:13,405 [WARNING] MainProcess indi.getCcdInfo() #258: CCD_CFA fetch timeout, assuming monochrome camera 2022-05-04 18:18:13,608 [WARNING] MainProcess indi.getCcdGain() #389: indi_sx_ccd does not support gain settings 2022-05-04 18:18:13,608 [INFO] MainProcess allsky._initialize() #356: Minimum CCD exposure: 0.00010001 2022-05-04 18:18:13,609 [INFO] MainProcess allsky._initialize() #371: Default CCD exposure: 0.00010001 2022-05-04 18:18:13,610 [INFO] MainProcess allsky._initialize() #378: CCD CFA: None 2022-05-04 18:18:13,610 [ERROR] MainProcess allsky.unhandled_exception() #36: An uncaught exception occurred: 2022-05-04 18:18:13,610 [ERROR] MainProcess allsky.unhandled_exception() #37: Type: <class 'KeyError'> 2022-05-04 18:18:13,611 [ERROR] MainProcess allsky.unhandled_exception() #38: Value: 'min' 2022-05-04 18:18:13,615 [ERROR] MainProcess allsky.unhandled_exception() #43: ' File "./allsky.py", line 128, in \n action_func(*args_list, **kwargs_dict)\n' 2022-05-04 18:18:13,616 [ERROR] MainProcess allsky.unhandled_exception() #43: ' File "/home/astroberry/indi-allsky/indi_allsky/allsky.py", line 552, in run\n self._initialize()\n' 2022-05-04 18:18:13,616 [ERROR] MainProcess allsky.unhandled_exception() #43: ' File "/home/astroberry/indi-allsky/indi_allsky/allsky.py", line 382, in _initialize\n ccd_min_gain = self.config[\'CCD_INFO\'][\'GAIN_INFO\'][\'min\']\n'

[aaronwmorris]

Looks like I neglected the code paths for cameras that do not support gain. Merged #190 to fix.

[rkilgard]

Awesome! It's working! But all my images look like darks. It appears to be taking 0.0001 second images. Everything is set to the default values. I tried playing with the night gain, target ADU, and default and min exposure settings, but I can't get it to do anything different. I'm sure I'm missing something obvious here.

Update: never mind, forgot to HUP the process. I think it's working.

[aaronwmorris]

You may have to give it a minute or two for the exposure algorithm to normalize. It starts at the minimum exposure and increases from there.

If it seems like it gets stuck on the minimum exposure, taking darks can sometimes help with that situation.

[rkilgard]

It did a great job and ran all night! It didn't generate any of the charts/timelapses though, and clicking on Images produces an Internal Server Error. Is there an action I need to take to enable those processes?

Thanks! This software is amazing. I've been fighting with USB/ethernet extenders for over a year now (and losing), but your software let me put a Pi near the camera and removed a workload from my telescope control computer.

[aaronwmorris]

Interesting... check the log at /var/log/indi-allsky/webapp-indi-allsky.log and let me know what exceptions are generated for those views.

[rkilgard]

It's pretty chatty. Here's the line in the log from the loop view. Looks like it's maybe a SQL error? The others are similar. I'll put the images one below as well.

May 5 10:50:59 astroberry [ERROR] MainProcess app.log_exception() #1456: Exception on /indi-allsky/js/loop [GET]#012Traceback (most recent call last):#012 File "/home/astroberry/indi-allsky/virtualenv/indi-allsky/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1820, in _execute_context#012 cursor, statement, parameters, context#012 File "/home/astroberry/indi-allsky/virtualenv/indi-allsky/lib/python3.7/site-packages/sqlalchemy/engine/default.py", line 732, in do_execute#012 cursor.execute(statement, parameters)#012sqlite3.OperationalError: no such column: image.exp_elapsed#012#012The above exception was the direct cause of the following exception:#012#012Traceback (most recent call last):#012 File "/home/astroberry/indi-allsky/virtualenv/indi-allsky/lib/python3.7/site-packages/flask/app.py", line 2077, in wsgi_app#012 response = self.full_dispatch_request()#012 File "/home/astroberry/indi-allsky/virtualenv/indi-allsky/lib/python3.7/site-packages/flask/app.py", line 1525, in full_dispatch_request#012 rv = self.handle_user_exception(e)#012 File "/home/astroberry/indi-allsky/virtualenv/indi-allsky/lib/python3.7/site-packages/flask/app.py", line 1523, in full_dispatch_request#012 rv = self.dispatch_request()#012 File "/home/astroberry/indi-allsky/virtualenv/indi-allsky/lib/python3.7/site-packages/flask/app.py", line 1509, in dispatch_request#012 return self.ensure_sync(self.view_functions[rule.endpoint])(*req.view_args)#012 File "/home/astroberry/indi-allsky/virtualenv/indi-allsky/lib/python3.7/site-packages/flask/views.py", line 84, in view#012 return current_app.ensure_sync(self.dispatch_request)(args, **kwargs)#012 File "/home/astroberry/indi-allsky/indi_allsky/flask/views.py", line 277, in dispatch_request#012 json_data = self.get_objects()#012 File "/home/astroberry/indi-allsky/indi_allsky/flask/views.py", line 388, in get_objects#012 'image_list' : self.getLatestImages(history_seconds),#012 File "/home/astroberry/indi-allsky/indi_allsky/flask/views.py", line 408, in getLatestImages#012 for i in latest_images:#012 File "/home/astroberry/indi-allsky/virtualenv/indi-allsky/lib/python3.7/site-packages/sqlalchemy/orm/query.py", line 2896, in iter#012 return self._iter().iter()#012 File "/home/astroberry/indi-allsky/virtualenv/indi-allsky/lib/python3.7/site-packages/sqlalchemy/orm/query.py", line 2906, in _iter#012 execution_options={"_sa_orm_load_options": self.load_options},#012 File "/home/astroberry/indi-allsky/virtualenv/indi-allsky/lib/python3.7/site-packages/sqlalchemy/orm/session.py", line 1696, in execute#012 result = conn._execute_20(statement, params or {}, execution_options)#012 File "/home/astroberry/indi-allsky/virtualenv/indi-allsky/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20#012 return meth(self, args_10style, kwargs_10style, execution_options)#012 File "/home/astroberry/indi-allsky/virtualenv/indi-allsky/lib/python3.7/site-packages/sqlalchemy/sql/elements.py", line 326, in _execute_on_connection#012 self, multiparams, params, execution_options#012 File "/home/astroberry/indi-allsky/virtualenv/indi-allsky/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1508, in _execute_clauseelement#012 cache_hit=cache_hit,#012 File "/home/astroberry/indi-allsky/virtualenv/indi-allsky/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1863, in _execute_context#012 e, statement, parameters, cursor, context#012 File "/home/astroberry/indi-allsky/virtualenv/indi-allsky/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 2044, in _handle_dbapi_exception#012 sqlalchemy_exception, with_traceback=excinfo[2], from=e#012 File "/home/astroberry/indi-allsky/virtualenv/indi-allsky/lib/python3.7/site-packages/sqlalchemy/util/compat.py", line 207, in raise_#012 raise exception#012 File "/home/astroberry/indi-allsky/virtualenv/indi-allsky/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1820, in _execute_context#012 cursor, statement, parameters, context#012 File "/home/astroberry/indi-allsky/virtualenv/indi-allsky/lib/python3.7/site-packages/sqlalchemy/engine/default.py", line 732, in do_execute#012 cursor.execute(statement, parameters)#012sqlalchemy.exc.OperationalError: (sqlite3.OperationalError) no such column: image.exp_elapsed#012[SQL: SELECT image.id AS image_id, image.filename AS image_filename, image."createDate" AS "image_createDate", image."dayDate" AS "image_dayDate", image.exposure AS image_exposure, image.exp_elapsed AS image_exp_elapsed, image.gain AS image_gain, image.binmode AS image_binmode, image."temp" AS image_temp, image.night AS image_night, image.adu AS image_adu, image.stable AS image_stable, image.moonmode AS image_moonmode, image.moonphase AS image_moonphase, image.adu_roi AS image_adu_roi, image.sqm AS image_sqm, image.stars AS image_stars, image.uploaded AS image_uploaded, image.calibrated AS image_calibrated, image.detections AS image_detections, image.camera_id AS image_camera_id #012FROM image JOIN camera ON camera.id = image.camera_id #012WHERE camera.id = ? AND image."createDate" > ? ORDER BY image."createDate" DESC#012 LIMIT ? OFFSET ?]#012[parameters: (1, '2022-05-05 10:35:59.881456', 1000, 0)]#012(Background on this error at: https://sqlalche.me/e/14/e3q8)

May 5 10:53:09 astroberry [ERROR] MainProcess app.log_exception() #1456: Exception on /indi-allsky/imageviewer [GET]#012Traceback (most recent call last):#012 File "/home/astroberry/indi-allsky/virtualenv/indi-allsky/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1820, in _execute_context#012 cursor, statement, parameters, context#012 File "/home/astroberry/indi-allsky/virtualenv/indi-allsky/lib/python3.7/site-packages/sqlalchemy/engine/default.py", line 732, in do_execute#012 cursor.execute(statement, parameters)#012sqlite3.OperationalError: no such column: image.exp_elapsed#012#012The above exception was the direct cause of the following exception:#012#012Traceback (most recent call last):#012 File "/home/astroberry/indi-allsky/virtualenv/indi-allsky/lib/python3.7/site-packages/flask/app.py", line 2077, in wsgi_app#012 response = self.full_dispatch_request()#012 File "/home/astroberry/indi-allsky/virtualenv/indi-allsky/lib/python3.7/site-packages/flask/app.py", line 1525, in full_dispatch_request#012 rv = self.handle_user_exception(e)#012 File "/home/astroberry/indi-allsky/virtualenv/indi-allsky/lib/python3.7/site-packages/flask/app.py", line 1523, in full_dispatch_request#012 rv = self.dispatch_request()#012 File "/home/astroberry/indi-allsky/virtualenv/indi-allsky/lib/python3.7/site-packages/flask/app.py", line 1509, in dispatch_request#012 return self.ensure_sync(self.view_functions[rule.endpoint])(req.view_args)#012 File "/home/astroberry/indi-allsky/virtualenv/indi-allsky/lib/python3.7/site-packages/flask/views.py", line 84, in view#012 return current_app.ensure_sync(self.dispatch_request)(*args, *kwargs)#012 File "/home/astroberry/indi-allsky/indi_allsky/flask/views.py", line 259, in dispatch_request#012 context = self.get_context()#012 File "/home/astroberry/indi-allsky/indi_allsky/flask/views.py", line 1097, in get_context#012 context['form_viewer'] = IndiAllskyImageViewerPreload(data=form_data)#012 File "/home/astroberry/indi-allsky/virtualenv/indi-allsky/lib/python3.7/site-packages/wtforms/form.py", line 208, in call#012 return type.call(cls, args, kwargs)#012 File "/home/astroberry/indi-allsky/indi_allsky/flask/forms.py", line 940, in init#012 .order_by(IndiAllSkyDbImageTable.createDate.desc())#012 File "/home/astroberry/indi-allsky/virtualenv/indi-allsky/lib/python3.7/site-packages/sqlalchemy/orm/query.py", line 2819, in first#012 return self.limit(1)._iter().first()#012 File "/home/astroberry/indi-allsky/virtualenv/indi-allsky/lib/python3.7/site-packages/sqlalchemy/orm/query.py", line 2906, in _iter#012 execution_options={"_sa_orm_load_options": self.load_options},#012 File "/home/astroberry/indi-allsky/virtualenv/indi-allsky/lib/python3.7/site-packages/sqlalchemy/orm/session.py", line 1696, in execute#012 result = conn._execute_20(statement, params or {}, execution_options)#012 File "/home/astroberry/indi-allsky/virtualenv/indi-allsky/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20#012 return meth(self, args_10style, kwargs_10style, execution_options)#012 File "/home/astroberry/indi-allsky/virtualenv/indi-allsky/lib/python3.7/site-packages/sqlalchemy/sql/elements.py", line 326, in _execute_on_connection#012 self, multiparams, params, execution_options#012 File "/home/astroberry/indi-allsky/virtualenv/indi-allsky/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1508, in _execute_clauseelement#012 cache_hit=cache_hit,#012 File "/home/astroberry/indi-allsky/virtualenv/indi-allsky/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1863, in _execute_context#012 e, statement, parameters, cursor, context#012 File "/home/astroberry/indi-allsky/virtualenv/indi-allsky/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 2044, in _handle_dbapi_exception#012 sqlalchemy_exception, with_traceback=excinfo[2], from=e#012 File "/home/astroberry/indi-allsky/virtualenv/indi-allsky/lib/python3.7/site-packages/sqlalchemy/util/compat.py", line 207, in raise_#012 raise exception#012 File "/home/astroberry/indi-allsky/virtualenv/indi-allsky/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1820, in _execute_context#012 cursor, statement, parameters, context#012 File "/home/astroberry/indi-allsky/virtualenv/indi-allsky/lib/python3.7/site-packages/sqlalchemy/engine/default.py", line 732, in do_execute#012 cursor.execute(statement, parameters)#012sqlalchemy.exc.OperationalError: (sqlite3.OperationalError) no such column: image.exp_elapsed#012[SQL: SELECT image.id AS image_id, image.filename AS image_filename, image."createDate" AS "image_createDate", image."dayDate" AS "image_dayDate", image.exposure AS image_exposure, image.exp_elapsed AS image_exp_elapsed, image.gain AS image_gain, image.binmode AS image_binmode, image."temp" AS image_temp, image.night AS image_night, image.adu AS image_adu, image.stable AS image_stable, image.moonmode AS image_moonmode, image.moonphase AS image_moonphase, image.adu_roi AS image_adu_roi, image.sqm AS image_sqm, image.stars AS image_stars, image.uploaded AS image_uploaded, image.calibrated AS image_calibrated, image.detections AS image_detections, image.camera_id AS image_camera_id #012FROM image ORDER BY image."createDate" DESC#012 LIMIT ? OFFSET ?]#012[parameters: (1, 0)]#012(Background on this error at: https://sqlalche.me/e/14/e3q8)

[aaronwmorris]

Okay, this was from my last schema update for v2 where I added new column to the DB.

You just need to re-run the setup script to kick off the DB migration. You can also manually run the migration if you are interested in doing that.

https://github.com/aaronwmorris/indi-allsky/wiki/Database-migrations

[rkilgard]

Attempting either method gives a flask error. Should I nuke the existing database?

Setup DB Backup DB prior to migration /home/astroberry/indi-allsky/virtualenv/indi-allsky/lib/python3.7/site-packages/paramiko/transport.py:236: CryptographyDeprecationWarning: Blowfish has been deprecated "class": algorithms.Blowfish, ERROR [flask_migrate] Error: Can't locate revision identified by '46e5b9d20412'

[aaronwmorris]

Yeah, nuke it and delete the existing migrations and rerun the setup script.

rm -i /var/lib/indi-allsky/indi-allsky.sqlite

# from the indi-allsky folder
rm -i migrations/versions/*.py

[rkilgard]

Alright, we're in business. We'll see how it does tonight. Thanks again!

[aaronwmorris]

Did everything work out?

[rkilgard]

For the most part, yes. It is successfully taking images and generating charts in real time. It isn't saving any charts as far as I can tell, and the pulldown menus for charts don't seem to do anything when it is not actively collecting data. The Timelapses year and month pulldowns also only say "None" with no selectable options.

I haven't set up sftp yet–does it save charts if those aren't enabled?

[aaronwmorris]

The behavior you are seeing is expected for now. I do not have the ability to display historical data currently. I hope to get to this eventually.

[fockez]

It seems that I still got 0.0001s exposure for a long time. The Gain -1, Temp -273.1C Stars 0 and Moon Mode. But the setting is Gain 75 for Moon mode. The configuration is default, should I change them?

I even set very high default exposure and min exposure, but still get 0.0001 exposure. Using other client to connect the indiserver and exposure as high as 60s and get actual image.

[aaronwmorris]

Are you restarting (or reloading) the indi-allsky service when making the changes?

You may want to try setting a default exposure of something like 0.01 or 0.1 to see if you can get a good starting exposure. Some cameras give the auto-exposure algorithm problems with extremely low exposures. Sometimes dark frames can help with this.

[fockez]

I set 0.1 and restart, now it keeps at 0.1.

[aaronwmorris]

Can you post your config file here?

[fockez]

I change to 30s and it successfully adjust to 60s. So, I believe 0.1 is not a good start. But what would be the best practice for this camera?

[aaronwmorris]

Unfortunately, I do not have a SX camera so I have no real basis to give a recommendation. My understanding is that the SX cameras have a fixed gain, so that is probably why the 0.1 default exposure is not working (other cameras can adjust to a higher gain level at night).

The default exposure just needs to be high enough to allow the exposure algorithm to start scaling properly. You may want to try 1.0 or 5.0.

[fockez]

If I change the default exposure, should I change the minimal exposure? During the daytime, the exposure will still get lower to the minimal exposure, after that could it possible to get back to the long exposure when getting dark? Maybe I can provide a remote connection with SX camera, is that enough for you to test this camera?

[fockez]

In fact, I find that indi driver of sx oculus will cause smear during short exposure, but indigo driver will not.

[aaronwmorris]

The exposure algorithm will scale the exposure up and down automatically to keep the average brightness of the image at a certain level.

You do not need to change the minimum exposure when you change the default exposure. The default exposure is just the initial starting exposure, the exposure algorithm will start adjusting the exposure up or down from that level to get an average brightness.

[aaronwmorris]

If you want to grant remote SSH access or something, I would love to login and take a look. Let me know.

[aaronwmorris]

Closing this issue now now. Reopen if there is anything else needed.