Browser Access to USB Drive and Hotspot Setup

[HxPI]

Please provide support to remedy issues with 403 errors related to USB drive access. Thanks.

[HxPI]

Another suggestion is have gain ramp up/down during the transition from day/night and night/day and also when changing to/from moon gain. That way there isn’t an abrupt exposure change.

When gain changes from day to night, the algorithm should gradually change exposure time to maximum first then exposure gain to maximum (night gain). When gain changes from night to day, the algorithm should gradually change exposure gain to minimum (day gain) first then exposure time to minimum. This is similar to holy grail exposure ramping used in photography and makes it easier to use deflicker software to create smooth sunrise/sunset timelapses.

[HxPI]

Another question is with TIFF compression setting in config but TIFF is not an option for file format. Can this be made an available file format? Also, is there a way to set 16bit mode for ASI color camera to capture full color debayered image files, not MONO16 raw? Some apps only accept TIFF or raw for format import for timelapse processing and export. Specifically I’m referring to LRTimelapse.

[HxPI]

Trying to do dark frames. Getting error when activating python virtual environment: bash

virtualenv/indi-allsky/bin/activate: No such file or directory

Typed exactly as stated in Manual operation.

Update: Figured it out, must be in the indi-allsky directory to run command. Please update this in instruction. Darks are working as expected.

It would be nice if Dark Frame Capture Mode could be added as an option on System page, similar to Reset and Shutdown system? When dark frames are recorded, are the previous ones removed or overwritten? Also how many frames are stacked for each master frame? I still see a lot of hot/cold/dead/bad pixels on my non-cooled camera. In another software, I used bad pixel map and that was very effective at removing all the undesirable pixels. Perhaps a bad pixel map would also be a useful option?

[HxPI]

PNG files are working as expected. USB is also working as expected. All pages are showing correct information. The only thing left is getting Wi-Fi to work. I’ll wait for script review before proceeding.

[aaronwmorris]

Network manager is what is used in the script to setup the hotspot. I have tested multiple times with the hotspot setup and it always works. I see you are using an iPad, I will have to test with an iPad since I was testing with an Android phone. I will run more tests.

I have not enabled the ability to configure TIFF files in the config due to TIFF files not being supported by all browsers. indi-allsky can produce TIFFs, but they will never be displayed [in the browser]. It would probably be best to create PNG files and perform a mass conversion to TIFF.

As for the sky brightness.... this is considerably more difficult than I originally envisioned. In order to use a camera to detect sky brightness, you have to account for the QE and spectrum detected by the camera. If your camera has a UV/IR cut filter, then most of the light detected is visual. If you do not, then some of the brightness detected is outside the visual spectrum. It may take someone with a lot more experience in this area to come up with a way to do the conversion.

For the darks, you may have to check the log to make sure the darks are matching. There will be a message about which dark was matched to the frame. I have not had an issue with persistent noise (hot pixels, etc) in longer exposures (+5s) but I have seen issues with shorter exposures. Some cameras return inconsistent noise at exposures shorter than 1-2 seconds.

[HxPI]

I tried an Android device also and wasn’t able to connect. I could have introduced errors when trying to setup network manager separately.

Since the reinstall and keeping everything default, there have been no major issues. I will try the hotspot setup and see if it works with the default password. If it does, then all I really need to do is change the password. Can that be done by editing the script?

[aaronwmorris]

Yes, you can just edit the script with the new password. It is just a variable at the top.

[HxPI]

Ok. Since network manager is being used, is there a way to access and/or install a gui for it?

[aaronwmorris]

I can pretty easily add a desktop environment that will only be started if you connect via remote desktop (RDP). Meaning it would not run in the console via HDMI. This could have a network manager widget to edit the settings.

[HxPI]

Well I ran the hotspot script and now I don't see the IndiAllsky SSID on any device. I'll try installing again.

[aaronwmorris]

Have you tried rebooting to see if the SSID shows up?

[aaronwmorris]

Can you run nmcli device and nmcli connection and show the output here?

[HxPI]

I have reboot no change. Will run cli

[HxPI]

CLI Shows wlan0 connected as HotSpot

[HxPI]

I’m not connect to it though with any device

[HxPI]

pi@raspberrypi:~ $ nmcli device DEVICE TYPE STATE CONNECTION
eth0 ethernet connected Wired connection 1 wlan0 wifi connected HotSpot
lo loopback unmanaged --
pi@raspberrypi:~ $ nmcli connection NAME UUID TYPE DEVICE HotSpot 395c6a2d-9bd0-4b38-923e-e9ab7932f875 wifi wlan0
Wired connection 1 0044027f-9925-3cf6-ac37-9f5c038da318 ethernet eth0

[HxPI]

Reran script no change. All I did is change the password in Text Editor and save. Did I need to save the script in a special way? Does it matter if indi-allsky is running?

[HxPI]

OK well now I see the SSID and the connection works! All I did was disconnect the keyboard/mouse and reattach the camera, basically setup in normal remote operation. Why would that matter?

[HxPI]

OK another issue. I deleted all media files from the USB drive. The Timelapses page is showing empty placeholder links where the files used to be. Links give Not Found error. Is there a database flush that needs to happen? Should I not manually delete files from the USB?

[aaronwmorris]

The status of indi-allsky does not matter. The camera and keyboard should not matter either, but they are variables.

[aaronwmorris]

You can flush the images table with the following command...

sqlite3 /var/lib/indi-allsky/indi-allsky.sqlite "DELETE FROM image;" (edited)

[HxPI]

Do I need to stop indi-allsky first? How do I cleanup the USB drive in the future? This would also be a good option to put in System page.

[aaronwmorris]

It should not matter if indi-allsky is running, but you can stop just to be on the safe side.

I will write another script to verify the files on the filesystem and remove entries that no longer exist.

[HxPI]

Awesome. Thanks.

[HxPI]

Can't reach the RPi on Wifi with VNC using 10.42.0.1. So I noticed my iPad has a 169.254.#.# IP address but the WLAN is 10.42.0.1. Shouldn't the iPad have a 10.42.#.# IP address? Also it is complaining that WPA is not secure. Can the WLAN be set to WPA2 (WPA2-PSK: AES)?

[aaronwmorris]

The raspi does not have a DHCP server running, so you have to set a static IP in the 10.42.0.0/24 subnet.

I need to take a closer look at the WPA settings in network manager.

[HxPI]

You can flush the images table with the following command...

sqlite3 /var/lib/indi-allsky/indi-allsky.sqlite DELETE FROM image;

Getting Error: incomplete input

[HxPI]

So when the camera is disconnected, the wifi is inaccessible. When the camera is connected, the wifi works. There must be some process depending on the camera that keeps the wifi from working. Strange!

[aaronwmorris]

That does not make any sense... I will do some research.

[HxPI]

Well now SSID is not coming up again. Everything is configure as normal. I'll play around with it and see what comes up.

[HxPI]

Also Reboot and Shutdown on System page are not working. Something happened to them after hotspot was installed. Everything else appears to be working using wired connection to access.

[aaronwmorris]

I left out the quotes from the SQL command

sqlite3 /var/lib/indi-allsky/indi-allsky.sqlite "DELETE FROM image;"

[aaronwmorris]

Also Reboot and Shutdown on System page are not working. Something happened to them after hotspot was installed. Everything else appears to be working using wired connection to access.

When trying to reboot, watch for exceptions in /var/log/indi-allsky/webapp-indi-allsky.log

[HxPI]

Mar 26 14:22:00 raspberrypi [ERROR] MainProcess app.log_exception() #1458: Exception on /indi-allsky/ajax/system [POST]#012Traceback (most recent call last):#012 File "/home/pi/indi-allsky/virtualenv/indi-allsky/lib/python3.7/site-packages/flask/app.py", line 2073, in wsgi_app#012 response = self.full_dispatch_request()#012 File "/home/pi/indi-allsky/virtualenv/indi-allsky/lib/python3.7/site-packages/flask/app.py", line 1518, in full_dispatch_request#012 rv = self.handle_user_exception(e)#012 File "/home/pi/indi-allsky/virtualenv/indi-allsky/lib/python3.7/site-packages/flask/app.py", line 1516, in full_dispatch_request#012 rv = self.dispatch_request()#012 File "/home/pi/indi-allsky/virtualenv/indi-allsky/lib/python3.7/site-packages/flask/app.py", line 1502, in dispatch_request#012 return self.ensure_sync(self.view_functions[rule.endpoint])(req.view_args)#012 File "/home/pi/indi-allsky/virtualenv/indi-allsky/lib/python3.7/site-packages/flask/views.py", line 84, in view#012 return current_app.ensure_sync(self.dispatch_request)(*args, *kwargs)#012 File "/home/pi/indi-allsky/indi_allsky/flask/views.py", line 1402, in dispatch_request#012 r = self.rebootSystemd()#012 File "/home/pi/indi-allsky/indi_allsky/flask/views.py", line 1459, in rebootSystemd#012 r = manager.Reboot(False)#012 File "/home/pi/indi-allsky/virtualenv/indi-allsky/lib/python3.7/site-packages/dbus/proxies.py", line 72, in call#012 return self._proxy_method(args, keywords)#012 File "/home/pi/indi-allsky/virtualenv/indi-allsky/lib/python3.7/site-packages/dbus/proxies.py", line 147, in call#012 **keywords)#012 File "/home/pi/indi-allsky/virtualenv/indi-allsky/lib/python3.7/site-packages/dbus/connection.py", line 653, in call_blocking#012 message, timeout)#012dbus.exceptions.DBusException: org.freedesktop.DBus.Error.InteractiveAuthorizationRequired: Interactive authentication required.

[aaronwmorris]

Something is wrong with the policy kit authorization....

Can you show the contents of this file?

sudo cat /etc/polkit-1/localauthority/50-local.d/90-org.aaronwmorris.indi-allsky.pkla

[HxPI]

I left out the quotes from the SQL command

sqlite3 /var/lib/indi-allsky/indi-allsky.sqlite "DELETE FROM image;"

Did the command but the dead links are still on the Timelapses page. The other pages appear to be showing the correct information.

[HxPI]

sudo cat /etc/polkit-1/localauthority/50-local.d/90-org.aaronwmorris.indi-allsky.pkla

wmorris.indi-allsky.pkla [Allow indi-allsky user to shutdown] Identity=unix-user:pi Action=org.freedesktop.login1.power-off;org.freedesktop.login1.power-off-multiple-sessions;org.freedesktop.login1.reboot;org.freedesktop.login1.reboot-multiple-sessions ResultActive=yes ResultAny=yes

[Allow indi-allsky user to manage network] Identity=unix-user:pi Action=org.freedesktop.NetworkManager.enable-disable-wifi;org.freedesktop.NetworkManager.network-control;org.freedesktop.NetworkManager.wifi.share.protected;org.freedesktop.NetworkManager.settings.modify.system ResultActive=yes ResultAny=yes

[Allow indi-allsky user to mount USB disks] Identity=unix-user:pi Action=org.freedesktop.udisks2.filesystem-mount-system;org.freedesktop.udisks2.filesystem-mount;org.freedesktop.udisks2.filesystem-mount-other-seat ResultActive=yes ResultAny=yes

BTW the Reboot and Shutdown are working again. I did a browser history delete and several RPi reboots. Who knows what it was!

[HxPI]

Well now SSID is not coming up again. Everything is configure as normal. I'll play around with it and see what comes up.

SSID appears to be coming and going. Could be weak signal or something else. Will keep investigating.

[aaronwmorris]

SSID appears to be coming and going. Could be weak signal or something else. Will keep investigating.

I am working on a commit to disable the power saving modes for WiFi. I had tried to do this using a different method, but I am not sure it worked.

[HxPI]

sudo cat /etc/polkit-1/localauthority/50-local.d/90-org.aaronwmorris.indi-allsky.pkla

wmorris.indi-allsky.pkla [Allow indi-allsky user to shutdown] Identity=unix-user:pi Action=org.freedesktop.login1.power-off;org.freedesktop.login1.power-off-multiple-sessions;org.freedesktop.login1.reboot;org.freedesktop.login1.reboot-multiple-sessions ResultActive=yes ResultAny=yes

[Allow indi-allsky user to manage network] Identity=unix-user:pi Action=org.freedesktop.NetworkManager.enable-disable-wifi;org.freedesktop.NetworkManager.network-control;org.freedesktop.NetworkManager.wifi.share.protected;org.freedesktop.NetworkManager.settings.modify.system ResultActive=yes ResultAny=yes

[Allow indi-allsky user to mount USB disks] Identity=unix-user:pi Action=org.freedesktop.udisks2.filesystem-mount-system;org.freedesktop.udisks2.filesystem-mount;org.freedesktop.udisks2.filesystem-mount-other-seat ResultActive=yes ResultAny=yes

BTW the Reboot and Shutdown are working again. I did a browser history delete and several RPi reboots. Who knows what it was!

Spoke too soon. Shutdown and Reboot not working again. All I'm doing is just sitting watching the status and after a while selecting Shutdown. Using Firefox because it is the only browser that will allow connection.

Getting this error in Chrome

NET::ERR_CERT_INVALID

You cannot visit raspberrypi.local right now because the website sent scrambled credentials that Chrome cannot process. Network errors and attacks are usually temporary, so this page will probably work later.

Safari is hopeless!

Tried on another computer with MS Edge and same problem. Something must be wrong at the web server end.

[aaronwmorris]

Spoke too soon. Shutdown and Reboot not working again. All I'm doing is just sitting watching the status and after a while selecting Shutdown. Using Firefox because it is the only browser that will allow connection.

Getting this error in Chrome

NET::ERR_CERT_INVALID

You cannot visit raspberrypi.local right now because the website sent scrambled credentials that Chrome cannot process. Network errors and attacks are usually temporary, so this page will probably work later.

Safari is hopeless!

Yeah, I am not really happy with the state of browsers. With self-signed certificates on Mac, Safari and Firefox are your only real options. Chrome will not accept a self signed certificate at all.

"Modern" web development strongly encourages HTTPS, but I may try to allow HTTP access. I just have to make sure things like cookies do not have the secure flag set. I do not explicitly use cookies, but flask does for session management.

[HxPI]

I probably should have broken these issues up into separate threads.

[aaronwmorris]

Merged #133 which disables wifi powersave in the connection config. It also fixes the "Weak Security" message on iOS by forcing WPA2 and AES.

[aaronwmorris]

Merged #134 to allow saving files in TIFF format.

[aaronwmorris]

Merged #136 with a script to validate and cleanup orphaned DB entries.

Script is at misc/validate_db_entries.py

[HxPI]

I'll download and try these out.

[HxPI]

I'll download and try these out.

Got an error, won't allow me to git pull origin master because hotspot script was locally modified. Is there a way to force download?

Update: I just moved the hotspot script and the git pull origin master completed. Hopefully.

remote: Enumerating objects: 41, done. remote: Counting objects: 100% (41/41), done. remote: Compressing objects: 100% (12/12), done. remote: Total 29 (delta 22), reused 20 (delta 17), pack-reused 0 Unpacking objects: 100% (29/29), done. From https://github.com/aaronwmorris/indi-allsky

• branch master -> FETCH_HEAD 0ef4e35..8d01f8c master -> origin/master Updating 0ef4e35..8d01f8c error: Your local changes to the following files would be overwritten by merge: misc/setup_hotspot.sh Please commit your changes or stash them before you merge.

[HxPI]

Merged #136 with a script to validate and cleanup orphaned DB entries.

Script is at misc/validate_db_entries.py

How do I run this?

[HxPI]

Merged #133 which disables wifi powersave in the connection config. It also fixes the "Weak Security" message on iOS by forcing WPA2 and AES.

The hotspot script is causing my RPi4 to crash and reboot. It sucessfully removes Hotspot, but when it adds Hotspot the RPI reboots. The script failed to complete and there is no hotspot!

[HxPI]

Firefox developer tools shows this when trying to run Reboot Uncaught SyntaxError: JSON.parse: unexpected character at line 1 column 1 of the JSON data error https://raspberrypi.local/indi-allsky/system:465 jQuery 6 button_click https://raspberrypi.local/indi-allsky/system:451

https://raspberrypi.local/indi-allsky/system:513 jQuery 8 https://raspberrypi.local/indi-allsky/system:50