[BUG] All sky not starting at boot

[martinka]

Distribution (run cat /etc/os-release): PRETTY_NAME="Raspbian GNU/Linux 11 (bullseye)" NAME="Raspbian GNU/Linux" VERSION_ID="11" VERSION="11 (bullseye)" VERSION_CODENAME=bullseye ID=raspbian ID_LIKE=debian HOME_URL="http://www.raspbian.org/" SUPPORT_URL="http://www.raspbian.org/RaspbianForums" BUG_REPORT_URL="http://www.raspbian.org/RaspbianBugs"

Related Application and/or Package Version (run apt policy $PACKAGE NAME): allsky version from the version file is v2022.03.01. Not sure what package I would be looking for here.

Issue/Bug Description: After install the allsky service is not running sudo systemctl status allsky ● allsky.service - All Sky Camera Loaded: loaded (/etc/systemd/system/allsky.service; enabled; vendor preset: enable> Active: inactive (dead) Running the allsky.sh script seems to work: it collects images and is writing to stdout. the service creates the /var/log/allsky.log file but it is empty, even after setting debug to level 4.. the file was not created until I increased the log level. The only error message I have been able to find is this is /var/log/syslog Jan 1 13:49:53 allskypi systemd-udevd[198]: 2-1: Spawned process '/bin/systemctl start allsky' [261] is taking longer than 59s to complete Jan 1 13:49:53 allskypi systemd-udevd[169]: 2-1: Worker [198] processing SEQNUM=1285 is taking a long time Jan 1 13:51:53 allskypi systemd-udevd[198]: 2-1: Spawned process '/bin/systemctl start allsky' [261] timed out after 2min 59s, killing Jan 1 13:51:53 allskypi systemd-udevd[169]: 2-1: Worker [198] processing SEQNUM=1285 killed

Steps to reproduce (if you know): the only thing I can thing to add here is the hardware information as I just followed the installation instructions on your github page. I did install the web ui as well. I am running this on a Raspberry Pi 4 Model B Rev 1.5 The system has 58.23 GB left and the memory is not being taxed when running the script. I have a zwo ASI385MC connected via USB 3.0.

Expected behavior: the service should start and stay running.

Other Notes: Sorry to bug you guys with this.. but really excited to get this camera working.

[EricClaeys]

@martinka Michael, sorry you are having problems. Lots of other people are running Allsky on Bullseye so I am confident we can get yours running. Did you reboot after installing Allsky? Is Allsky in /home/pi/allsky or somewhere else? Run systemctl stop allsky; cd ~/allsky; ./allsky.sh then wait a few minutes and attach /var/log/allsky.log.

Eric

[martinka]

@EricClaeys Thank you for your response. Here is my best effort to address your questions. Allsky is in /home/pi/allsky. I have rebooted many times.. oddly when I run ./allsky.sh it seems to write to standard out not the /var/log/allsky.log. I am including the output I got from standard out.

`     ***** Starting AllSky *****
CAMERA: ZWO
CAMERA_SETTINGS: /etc/raspap/settings_ZWO.json
*** WARNING: Could not set locale to en_US.UTF-8 ***

**********************************************
*** Allsky Camera Software v0.8.3.3 |  2022 ***
**********************************************

Capture images of the sky with a Raspberry Pi and an ASI Camera

Add -h or --help for available options

Author: Thomas Jacquin - <jacquin.thomas@gmail.com>

Contributors:
 -Knut Olav Klo
 -Daniel Johnsen
 -Yang and Sam from ZWO
 -Robert Wagner
 -Michael J. Kidd - <linuxkidd@gmail.com>
 -Chris Kuethe
 -Eric Claeys

ZWO ASI385MC Information:
  - Native Resolution: 1936x1096
  - Pixel Size: 3.8microns
  - Supported Bins: 1 2
  - Color Camera: bayer pattern: RG

  - Camera Serial Number: 1d0d45031d010900
- Sensor temperature: 24.30
  - Current camera mode: NORMAL (0)
Control Caps:
- Gain:
   - Description = Gain
   - MinValue = 0
   - MaxValue = 600
   - DefaultValue = 200
   - IsAutoSupported = 1
   - IsWritable = 1
   - ControlType = 0
- Exposure:
   - Description = Exposure Time(us)
   - MinValue = 32
   - MaxValue = 2000000000
   - DefaultValue = 10000
   - IsAutoSupported = 1
   - IsWritable = 1
   - ControlType = 1
- WB_R:
   - Description = White balance: Red component
   - MinValue = 1
   - MaxValue = 99
   - DefaultValue = 52
   - IsAutoSupported = 1
   - IsWritable = 1
   - ControlType = 3
- WB_B:
   - Description = White balance: Blue component
   - MinValue = 1
   - MaxValue = 99
   - DefaultValue = 95
   - IsAutoSupported = 1
   - IsWritable = 1
   - ControlType = 4
- Offset:
   - Description = offset
   - MinValue = 0
   - MaxValue = 240
   - DefaultValue = 1
   - IsAutoSupported = 0
   - IsWritable = 1
   - ControlType = 5
- BandWidth:
   - Description = The total data transfer rate percentage
   - MinValue = 40
   - MaxValue = 100
   - DefaultValue = 50
   - IsAutoSupported = 1
   - IsWritable = 1
   - ControlType = 6
- Flip:
   - Description = Flip: 0->None 1->Horiz 2->Vert 3->Both
   - MinValue = 0
   - MaxValue = 3
   - DefaultValue = 0
   - IsAutoSupported = 0
   - IsWritable = 1
   - ControlType = 9
- AutoExpMaxGain:
   - Description = Auto exposure maximum gain value
   - MinValue = 0
   - MaxValue = 600
   - DefaultValue = 300
   - IsAutoSupported = 0
   - IsWritable = 1
   - ControlType = 10
- AutoExpMaxExpMS:
   - Description = Auto exposure maximum exposure value(unit ms)
   - MinValue = 1
   - MaxValue = 60000
   - DefaultValue = 100
   - IsAutoSupported = 0
   - IsWritable = 1
   - ControlType = 11
- AutoExpTargetBrightness:
   - Description = Auto exposure target brightness value
   - MinValue = 50
   - MaxValue = 160
   - DefaultValue = 100
   - IsAutoSupported = 0
   - IsWritable = 1
   - ControlType = 12
- HighSpeedMode:
   - Description = Is high speed mode:0->No 1->Yes
   - MinValue = 0
   - MaxValue = 1
   - DefaultValue = 0
   - IsAutoSupported = 0
   - IsWritable = 1
   - ControlType = 14
- MonoBin:
   - Description = bin R G G B to one pixel for color camera, color will loss
   - MinValue = 0
   - MaxValue = 1
   - DefaultValue = 0
   - IsAutoSupported = 0
   - IsWritable = 1
   - ControlType = 18
- Temperature:
   - Description = Sensor temperature(degrees Celsius)
   - MinValue = -500
   - MaxValue = 1000
   - DefaultValue = 20
   - IsAutoSupported = 0
   - IsWritable = 0
   - ControlType = 8
Supported video formats:
   - ASI_IMG_RAW8
   - ASI_IMG_RGB24
   - ASI_IMG_Y8
   - ASI_IMG_RAW16
Supported modes:
   - NORMAL (0)
^[[32m
Capture Settings:
 Image Type: RGB24
 Resolution (before any binning): 1936x1096
 Quality: 95
 Daytime capture: Yes
 Exposure (day): 0.500ms, Auto: Yes
 Exposure (night): 10000ms, Auto: Yes
 Max Auto-Exposure (day): 60000ms (60.0s)
 Max Auto-Exposure (night): 20000ms (20.0s)
 Delay (day): 5000ms
 Delay (night): 10ms
 Gain (night only): 50, Auto: No, max: 200
 Gain Transition Time: 15.0 minutes
 Brightness (day): 50
 Brightness (night): 50
 Skip Frames (day): 5
 Skip Frames (night): 1
 Aggression: 75%
 Gamma: 50
 WB Red: 53, Blue: 90, Auto: No
 Binning (day): 1
 Binning (night): 1
 USB Speed: 80, auto: Yes
 Text Overlay: [none]
 Text Extra File: [none], Age: 600 seconds
 Text Line Height 60px
 Text Position: 15px from left, 30px from top
 Font Name: SIMPLEX (0)
 Font Color: 255, 255, 255
 Small Font Color: 0, 0, 255
 Font Line Type: 16
 Font Size: 7.0
 Font Line Width: 1
 Outline Font : No
 Flip Image: 0
 Filename: image.jpg
 Filename Save Directory: /home/pi/allsky/tmp
 Latitude: 60.7N, Longitude: 135.05W
 Sun Elevation: -6
 Locale: en_US.UTF-8
 Notification Images: Yes
 Histogram Box: 500 500 50 50, center: 968x548, upper left: 718x298, lower right: 1218x798
 Show Histogram Box: No
 Show Histogram Mean: No
 Show Time: Yes (format: %Y%m%d %H:%M:%S)
 Show Temperature: Yes, type: C
 Show Exposure: Yes
 Show Gain: Yes
 Show Brightness: No
 Preview: No
 Taking Dark Frames: No
 Debug Level: 4
 On TTY: Yes
 Video OFF Between Images: Yes
 ZWO SDK version 1, 20
^[[0m
NOTICE: Camera does not support ControlCap # 2; not setting to 50.
Will adjust gain at transitions
*** Press Ctrl+C to stop ***

xxx resetGainTransitionVariables(0, 50) called at NIGHT
==========
=== Starting nighttime capture ===
==========
Using night exposure (10.0 sec)
Buffer size: 6365568
STARTING EXPOSURE at: 2023-01-07 08:01:00   @ 10.0 sec
  > Exposure set to 10.0 sec, timeout: 25000 ms
xxxxxxx exposure_time_us=10000000, estimated timeToTakeImage_us=10220979
  > Got image @ mean 3.  Reported exposure: 14.8 sec, auto=yes
  >>>> Skipping this frame
STARTING EXPOSURE at: 2023-01-07 08:01:11   @ 14.8 sec
  > Camera set auto-exposure to 14.8 sec, timeout: 34600 ms
xxxxxxx exposure_time_us=14800000, estimated timeToTakeImage_us=14976652
  > Got image @ mean 3.  Reported exposure: 20.0 sec, auto=yes
  > Changing next exposure by 3.9 sec from 14.8 sec to 18.7 sec
  > Sleeping: 5.2 sec
  > Saving NIGHT image 'image-20230107080111.jpg'
  > Image took 105.2 ms to save (average 105.2 ms).
DEBUG: saveImage.sh NIGHT /home/pi/allsky/tmp/image-20230107080111.jpg EXPOSURE_US=14800000 BRIGHTNESS=50 MEAN=3 AUTOEXPOSURE=1 AUTOGAIN=0 AUTOWB=0 WBR=53 WBB=90 TEMPERATURE=25 GAIN=50 GAINDB=1.78 BIN=1 FLIP=0 BIT_DEPTH=8
STARTING EXPOSURE at: 2023-01-07 08:01:32   @ 18.7 sec
  > Camera set auto-exposure to 18.7 sec, timeout: 42400 ms
     ***** Stopping AllSky *****
allsky.sh: GOT_SIGTERM=false, GOT_SIGUSR1=false, GOT_SIGINT=true`

Please note. I ran this with the lense cap on so the exposure information is probably meaningless, but the camera does work. Also I did a sig term which is why it stopped.
I see that the service file seems to be set to restart on success after 5 seconds, but when I run the script it seems to run forever and just keep taking pictures. not sure its relevant to any of this but thougth I would mention it.

THank you again for the help.

[martinka]

a couple other things I have tried / noticed.. sudo systemctl restart allsky hangs. I tried moving allsky from /home/pi to /opt since some folks seem to indicate running from /home is not ideal. That didn't seem to change anything. I also tried opening permissions on /var/log that also did not matter.

I then tried changing the service file a bit to set Type=simple and a working directory that also did not change results. I also tried specifically telling it to redirect output to /var/log/allsky.log this also didn't work. .this makes me think it might be some kind of permission issue with the log file.

I went ahead and reimaged the SD card.. I did not do the apt-get update and apt-get upgrade this time in case its related to recent patches.. and I did not load the web ui.. but the results are unchanged.

[EricClaeys]

@martinka I should have remembered that allsky.sh always writes to stdout but when it's called via the service, stdout is redirected to the log file. allsky.sh will run forever until it is killed, either via systemctl or when someone changes a setting in the WebUI. In that case, allsky.sh is sent a signal that tells it to exit with success, so the service controller will restart it in 5 seconds.

Even after reimaging your SD card and installing a clean Allsky, it still doesn't start via the service (systemctl start allsky times out) but running allsky.sh manually works fine. Is that correct? If so, try the following:

cd ~/allsky
mv allsky.sh allsky.sh.ORIGINAL
touch allsky.sh
chmod 775 allsky.sh
sudo truncate -s 0 /var/log/allsky.log
chgrp pi /var/log/allsky.log
chmod 664 /var/log/allsky.log

#  in a text editor add these lines to the new allsky.sh:
/bin/bash
echo from test allsky.sh
exit 99

Now run systemctl start allsky and see if it starts and you have a single line in /var/log/allsky.log ("from test allsky.sh"). If THAT works there must be something in allsky.sh that is hanging.

[martinka]

Eric, thanks for looking at this.. You are correct that the script runs fine when run manually. I did the steps above and I can verify that the pi user could not write the allsky.log prior so I touched it and changed permissions. and can verify the pi user can now write to the file. Unfortunately systemctl still hangs with either the test script or the original, and no logs are written.

There are two things I wanted to mention from a reproduction point of view. 1) I used the Lite version of the pi OS. 2) rather than setting a password I am using an ssh key. (I don't think either of these should be causing an issue, but for completeness wanted to mention it.

I also just for testing tried running allsky.sh > /var/log/allsky.log 2>&1 after the permission changes and it can write to the log file. So we do seem to have fixed that.

I am curious how the service knows to log to /var/log/allsky.log.. I don't see anything in the service file pointing to that.

[martinka]

Not pushing but realized I didn't include an @EricClaeys in the previous reply. Please let me know if there is anything else I might try.. I think my next step is going to try building a different service pointing at someplace else run as root and if that works.. work my way step at a time toward the one that failing.

[EricClaeys]

@martinka Michael, the allsky service file has SyslogFacility=local5 in it, and the rsyslog file in allsky/config_repo defines local5 as /var/log/allsky.log.

Having the Lite version of the OS may be causing the problem. It doesn't install as many packages as the full version, and it's possible the service program needs one of those packages.

Here's what I would do to debug: In /var/log, truncate some log files like syslog, daemon, kern (not sure these are the exact names): sudo truncate -s 0 FILE_NAME. Then systemctl start allsky. Then look in the log files for any error messages. Also, run systemctl status allsky to see if it gives any useful info.

[martinka]

Well I managed to find a flag in /etc/systemd/system.conf to set the systemd logging to debug.. I got the following for the allsky service. `● allsky.service - All Sky Camera Loaded: loaded (/etc/systemd/system/allsky.service; enabled; vendor preset: enable> Active: inactive (dead)

Jan 14 14:38:19 allskypi systemd[1]: allsky.service: Trying to enqueue job allsky.servi> Jan 14 14:38:19 allskypi systemd[1]: allsky.service: Merged allsky.service/start into i> Jan 14 14:38:19 allskypi systemd[1]: allsky.service: Enqueued job allsky.service/start > Jan 14 14:38:20 allskypi systemd[1]: allsky.service: starting held back, waiting for: s> Jan 14 14:38:20 allskypi systemd[1]: allsky.service: starting held back, waiting for: m> ` I am continuting to look at it .. but I think held back might be waiting for multi.user.start .. but I am not sure. At least some information on what is going on.

[martinka]

@EricClaeys I got it to start.. I remove the After=multi-user.target from the [Unit] description. now going to install the web ui.. If you want I can pull the code and submit a PR for that. I am not sure if it will break other things.. but it seems to be working for me.

[EricClaeys]

@martinka Michael, don't change anything yet. We need to understand why this happened to you and not others.

My guess is that the service only starts in multi-user mode but the Lite version of Pi OS only has single user. Do all your Pi's have the Lite version?

[martinka]

I have one that is setup as a full desktop. I could take a look at that. I also know that patriotastro.com which is where I heard about the software is recommending the pi lite version, so I suspect it might be something "new" in the OS possibly. I know it appeared to work for him. It will likely take me a day or two to get to look at the other PI but I will give it a shot sometime this week.

[EricClaeys]

@martinka Michael, any update?

[martinka]

Unfortunately I haven’t had a chance to look at it. I should have time in the next day or two.On Jan 30, 2023, at 1:44 PM, EricClaeys @.***> wrote: @martinka Michael, any update?

—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you were mentioned.Message ID: @.***>

[martinka]

I was able to test with a Pi that had a full desktop loaded. The service works as is based on your current code with the full version. I then tried the fix I suggested.. which is removing the after clause... this worked most of the time. but I did have one reboot where it failed to find the camera.. best I could tell it tried to start before the USB system was up.. I think if we set the After target to network.target we should be fine. I have rebooted it multiple time and been unable to get it to fail. I am going to go back to the lite version and see if it works with the network.target. I should be done with that later today. If it works I will let you know.

[martinka]

@EricClaeys I was able to test this on the lite version and it worked fine.. I put a PR up https://github.com/thomasjacquin/allsky/pull/2183 let me know if you have any questions.

[augjohnson]

Maybe this is applicable. I was setting up a replacement camera on my system, decided to burn a new image to the card and start over. Downloaded new Lite version and had problems that look similar to the ones martinka was describing. When I reverted to the Lite OS I'd downloaded about 6 months ago, everything worked fine again. Something evidently changed in the OS. Tomorrow I can look and see the exact version the old one is.

[EricClaeys]

@martinka, @augjohnson, it works fine for other people with Bullseye Lite, so clearly there's a difference somewhere between your systems and others. Their systems DO enter multi-user mode. Would you please run the following?

ls -al /lib/systemd/system
sudo systemctl status multi-user.target
systemctl --user status multi-user.target

I suspect one of the last two commands will fail.

Thanks - Eric

[augjohnson]

I'll have to build another system tomorrow with the non-working OS to see what happens, I have my camera working now and want to keep it that way. Here's the OS that works that I am running now. I downloaded this OS back in October. The non-working one I downloaded a couple days ago.

allsky@allsky:~ $ sudo systemctl status multi-user.target ● multi-user.target - Multi-User System Loaded: loaded (/lib/systemd/system/multi-user.target; indirect; vendor pr> Active: active since Sun 2023-02-05 13:23:53 PST; 6h ago Docs: man:systemd.special(7)

Feb 05 13:23:53 allsky systemd[1]: Reached target Multi-User System.

allsky@allsky:~ $ systemctl --user status multi-user.target Unit multi-user.target could not be found. allsky@allsky:~ $ allsky@allsky:~ $ cat /etc/os-release PRETTY_NAME="Raspbian GNU/Linux 11 (bullseye)" NAME="Raspbian GNU/Linux" VERSION_ID="11" VERSION="11 (bullseye)" VERSION_CODENAME=bullseye ID=raspbian ID_LIKE=debian HOME_URL="http://www.raspbian.org/" SUPPORT_URL="http://www.raspbian.org/RaspbianForums" BUG_REPORT_URL="http://www.raspbian.org/RaspbianBugs" allsky@allsky:~ $

After I load the OS, I do nothing else but install Allsky, the WebUI and the Website.

Let me know if you also want the ls output.

August

[EricClaeys]

@augjohnson Thanks for that info. When you get a chance the ls output would be useful. I'd like to compare all the output to the "bad" system. I'm thinking the sudo systemctl status multi-user.target might fail and the systemctl --user status multi-user.target might succeed (the opposite of the "good" system).

[martinka]

I should be able to try the commands out on the lite pi I have sometime today. I will let you know what I find.

[martinka]

@EricClaeys not sure what you are looking for the with the ls command.. lots of files there here is the output from the sudo systemctl status multi-user.target `multi-user.target - Multi-User System Loaded: loaded (/lib/systemd/system/multi-user.target; static) Active: inactive (dead) Docs: man:systemd.special(7)

Feb 04 14:36:19 allskypi systemd[1]: multi-user.target: starting held back, waiting for: raspi-config.service Feb 04 14:36:19 allskypi systemd[1]: multi-user.target: starting held back, waiting for: raspi-config.service Feb 04 14:36:20 allskypi systemd[1]: multi-user.target: starting held back, waiting for: raspi-config.service Feb 04 14:36:20 allskypi systemd[1]: multi-user.target: starting held back, waiting for: raspi-config.service Feb 04 14:36:21 allskypi systemd[1]: multi-user.target: starting held back, waiting for: raspi-config.service Feb 04 14:36:21 allskypi systemd[1]: multi-user.target: starting held back, waiting for: raspi-config.service Feb 04 14:36:22 allskypi systemd[1]: multi-user.target: starting held back, waiting for: raspi-config.service Feb 04 14:36:23 allskypi systemd[1]: multi-user.target: starting held back, waiting for: dhcpcd.service Feb 04 14:36:23 allskypi systemd[1]: multi-user.target: starting held back, waiting for: dhcpcd.service Feb 04 14:36:35 allskypi systemd[1]: multi-user.target: starting held back, waiting for: userconfig.service`

[martinka]

and here is the other one systemctl --user status multi-user.target Unit multi-user.target could not be found.

[martinka]

not sure if this will be of use but figured the status of the userconfig.service might be of interest `sudo systemctl status userconfig.service ● userconfig.service - User configuration dialog Loaded: loaded (/lib/systemd/system/userconfig.service; enabled; vendor preset: enabled) Active: activating (start) since Sat 2023-02-04 14:36:23 EST; 2 days ago Main PID: 593 (userconf-servic) Tasks: 2 (limit: 4915) CPU: 3.281s CGroup: /system.slice/userconfig.service ├─ 593 /bin/sh -e /usr/lib/userconf-pi/userconf-service └─1009 whiptail --inputbox Please enter new username: 20 60

Feb 04 14:36:23 allskypi systemd[1]: userconfig.service: Passing 0 fds to service Feb 04 14:36:23 allskypi systemd[1]: userconfig.service: About to execute /usr/lib/userconf-pi/userconf-service Feb 04 14:36:23 allskypi systemd[1]: userconfig.service: Forked /usr/lib/userconf-pi/userconf-service as 593 Feb 04 14:36:23 allskypi systemd[1]: userconfig.service: Changed dead -> start Feb 04 14:36:23 allskypi systemd[1]: Starting User configuration dialog... Feb 04 14:36:23 allskypi systemd[593]: userconfig.service: Executing: /usr/lib/userconf-pi/userconf-service `

[EricClaeys]

Interesting. The user.config service is waiting for a new username to be entered. It's probably holding up going int multi-user mode which is when Allsky starts. The key will be to determine why user.config is prompting, and where. whiptail is what the Allsky install.sh uses to prompt -the blue screens.

[EricClaeys]

Try to Google linux userconfig service. The answer is likely there somewhere.

[augjohnson]

Well, this is good/bad news. Today I was going to build another system, just like the one I was having trouble with a couple days ago so I could run the commands on it. In fact, I had re-built before it several times, wasn't sure if there was something wrong with my SD card or what.

Anyway, using the SAME downloaded OS image, everything works perfectly! Absolutely the same installation procedure, burn image to SD card, install GIT, install allsky, install WebUI, install website. Didn't download new OS image, used the one I'd used before that failed. It's the 9/22 file that's the same one that you get if you download the OS today.

August

[Alex-developer]

@martinka Quick question.

How are you creating the image? Are you using the pi imager and if so do you setup all of the options i.e. user/password, hostname, wifi etc?

The reason I ask is looking at the messages above the Pi seems to be stuck trying to create a user which it should only do on first boot if you have not specified a user/password in the imager. Its also running in interactive mode which is odd

When I create the images I always specify a user/password but will try later not specifying one and see what happens.

Also if you currently have a pi stuck in this state could you log into it and run

runlevel

to see what level its at

I have had a look around and there are a few reports of the userconfig.service getting stuck but nothing really conclusive at the moment,

Whilst changing the allsky service will get around this issue my main concern is that if the Pi is not fully booted there will be other services that may not have started which could lead to problems in the future.

Many Thanks

Alex

[martinka]

@Alex-developer I did use pi imager.. I set it up everything except the password. I have the pi configured for ssh access by key. I had a couple minutes this morming so was able to check the runlevel and its does seem something is off runlevel unknown I tried with sudo as well with the same results.

[martinka]

I think the following link might be the root cause of my issue

https://www.raspberrypi.com/news/raspberry-pi-bullseye-update-april-2022/

I will see if I can set a password for the user. Seems like the opposite of security to require one.

[Alex-developer]

@Alex-developer I did use pi imager.. I set it up everything except the password. I have the pi configured for ssh access by key. I had a couple minutes this morming so was able to check the runlevel and its does seem something is off runlevel unknown I tried with sudo as well with the same results.

This really looks like a bug in the Pi OS to me. There is a very similar report of it here https://github.com/RPi-Distro/userconf-pi/issues/5

It would seem the user service is expecting a password which is a bit silly if you are using keys.

I would log it with the RPI-Distro team, see what they say and for now set a password for the user.

My concern with changing the point at which AllSky starts to get around this is that you don't have a Pi thats running in multiuser mode so there could be other issues.

P.S. I have just tried this at home with a screen plugged into the pi using an image with ssh keys. After booting the pi is asking for a user to be created, which is exactly what you are seeing.

And trying it with a headless boot it is indeed waiting for a user to be created - Please log this with the Pi team.

root 522 1 0 18:45 ? 00:00:00 /usr/sbin/rngd -r /dev/hwrng root 545 1 0 18:45 ? 00:00:00 /usr/sbin/ModemManager root 564 1 0 18:45 tty8 00:00:00 /bin/sh -e /usr/lib/userconf-pi/userconf-service root 584 2 0 18:45 ? 00:00:00 [kworker/u8:4-events_unbound] root 589 1 0 18:45 ? 00:00:00 wpa_supplicant -B -c/etc/wpa_supplicant/wpa_supplicant.conf -iwlan0 root 604 564 0 18:45 tty8 00:00:00 /usr/bin/perl -w /usr/sbin/dpkg-reconfigure -p critical keyboard-configuration

There are a few ways I can think of to get around this but they really should fix it properly

[martinka]

@Alex-developer Thanks .. I added a comment to the existing Issue for this. Hopefully it will get some traction. From my perspective we can close this issue. I have a work around at this point and to your point, it seems like its an issue with the current distro not Allsky. Thank you and @EricClaeys again for all the help.

[EricClaeys]

Closing per comment from submitter.