Out of Disc Space

[JohnAdstra]

Have my AllSky cam running of an RPi 2GB version with a 16GB SD card. When using the WEBUI System page, my typical Disk Usage is at 99%. I am not saving thumbnails and am autodeleting after 4 days trying to save space. While I can view a do a LiveView no problem, I can no longer get TimeLapse Videos, Keograms or Startrails. I think there is no storage left to do these. If I delete all images, I get down to the high 90%s and get things working for a short while.

I added a 64GB thumbdrive but can't seem to figure out how to save and retrieve images from there so the AllSky ware can access. Is there a work around for this?

Or should I clone the 16GB SD card to a larger size, like 64 or 128GB to solve the issue that way?

Any thoughts or comments appreciated. I am not code fluent bt any means, I just followed the directions :). But the cam has been working well for over 3 months now, just can't use all the features anymore.

Thanks,

John

[EricClaeys]

@JohnAdstra A 16 GB card is pretty small, but if your disk usage was much lower until recently, then something's taking the space. The df command will tell you have much space you have (hopefully it'll agree with the WebUI's "System" page). To make sure Allsky isn't taking all the space, you can:

du -s /home/pi/allsky
du -s /home/pi/allsky/images
du -s /var/www/html

Those are the high-level directories Allsky uses. If you subtract the 2nd from the first it'll say how much space the non-images are taking, and should be a pretty small number. /var/www/html is where the WebUI and Website (if you have it installed) reside. The WebUI doesn't take much space - it's only web pages. If you have the Website it's in /var/www/html/allsky and will contain prior timelapse videos, keograms, and startrails. The newest version of Allsky has a "WEB_DAYS_TO_KEEP" variable which specifies how many days of Website files to keep. If the Website is causing the problems you should set that variable.

To make sure timelapse, keograms, and startrails are failing due to lack of disk space, look in /var/log/allsky.log. There should be a message like "Starting timelapse" followed eventually by "Timelapse completed". Ditto for the others.

[JohnAdstra]

Thanks for your reply Eric!

I got the following:

pi@allskyshedcam:~ $ du -s /home/pi/allsky 6240964 /home/pi/allsky pi@allskyshedcam:~ $ du -s /home/pi/allsky/images 135988 /home/pi/allsky/images pi@allskyshedcam:~ $ du -s /var/www/html 15260 /var/www/html pi@allskyshedcam:~ $ /var/log/allsky.log bash: /var/log/allsky.log: Permission denied

These numbers all look small and don't amount to 16GB. I am not using the Website. So I tried just: pi@allskyshedcam:~ $ du

There are hundreds of listings under ./.config/chromium which I think add up to much more space. If I purge these somehow perhaps this will solve the issue?

Thanks again,

John

[JohnAdstra]

Also got the following: pi@allskyshedcam:~ $ df -h Filesystem Size Used Avail Use% Mounted on /dev/root 17G 16G 0 100% / devtmpfs 782M 0 782M 0% /dev tmpfs 943M 0 943M 0% /dev/shm tmpfs 377M 1.3M 376M 1% /run tmpfs 5.0M 4.0K 5.0M 1% /run/lock /dev/mmcblk0p6 253M 49M 204M 20% /boot tmpfs 189M 28K 189M 1% /run/user/1000 /dev/sda1 59G 1.5G 58G 3% /media/pi/0675-40DB /dev/mmcblk0p9 8.6G 20M 8.1G 1% /media/pi/Storage /dev/mmcblk0p8 512M 142M 371M 28% /media/pi/System

which looks like all storage is being used.

[EricClaeys]

@JohnAdstra /home/pi/allsky shouldn't be so large given the images directory size. Try du /home/pi/allsky. If your allsky is a few months old the allsky/tmp directory may be large with a lot of files. If so, remove them and upgrade Allsky.

Is ./.config/chromium used as a browser cache? If so, you can delete the files which will free up space, but it will eventually get used up again.

[JohnAdstra]

Well, it turns out I had a 32GB SD card in the RPi, but the System page said I had 16GB storage. Today I used Balena etcher to copy everything to a 64GB SD card and all started up again as normal. My System page still says I have a 16.41GB SD card. Why is AllSky Administration not letting use all the free storage space? I did not partition the card in any way.

By the way, clearing my Tmp files gives me about 3.75GB of free space, but this will probably fill up again eventually.

Any tips or comments appreciated. Would like to use more of the storage on the larger SD card if possible.

Thanks.

John

[EricClaeys]

@JohnAdstra I hope you're really asking why the WebUI only reports 16 GB and not asking "Why is AllSky Administration not letting use all the free storage space". The WebUI only reports what Linux says.

Please look at this Wiki page that addresses the problem you're reporting. I don't know why Linux does this - I consider it a bug, but there may be a valid reason.

If your allsky/tmp directory had a lot of images, including recent ones, please upgrade your allsky. That was a bug we fixed a couple months ago. Normally allsky/tmp has less than 10 MB.

[JohnAdstra]

Thanks Eric. I had no idea of the cause; the AllSky ware works great in most cases, so no knock on the well written app.

Tried the solutions listed in the link but no luck, as below:

Will need to explore the RPi route further. I'll need to check of what version of the AllSky ware I am using and will try to update accordingly.

Much appreciated.

John

[EricClaeys]

@JohnAdstra I just fixed the documentation to say to execute sudo resize2fs /dev/sda. Sorry about that.

I would only run sudo fsck is asked to do so by the resize2fs command. When I ran it on my Pi I wasn't asked to run fsck.

If you look in /var/log/allsky.log for the word Software the version will be on that line.

[JohnAdstra]

Thanks Eric,

I did try the sudo before the commands and I'm getting a lot of permission denied replies. Looks like the last time I installed the ware was January 16, 2022 from looking at the files involved, so I'm not sure if that's the version when some issues were addressed.

I was looking around the web for various other solutions to increasing the Pi storage but I still get permission denied on virtually everything.

I just started with the cam in late 2021 and used a new NOOBS SD card at that time. Perhaps something else is going on that I am not aware of as a non-expert in RPi tech. My solution might be to redo the SD card from scratch and install everything fresh.

Thanks for your efforts.

John

[EricClaeys]

@JohnAdstra in the first line of your screenshot you are trying to execute /var/log/allsky.log which is a text file, not an executable file. That's why you gor the Permission Denied message - the file isn't executable. I assume you want to view or edit the file? To view youcan use more /var/log/allsky.log which will display a page at a time. If you are logging in via VNC you can use the file explorer and right click on it and select one of the editors.

For the resize2fs errors, it's possible/dev/sda isn't your hard drive (items in /dev are devices in Linux, kinda similar to Device Manager in Windows). Run mount and post the results. It may say what device your disk is. You can also Google the command to see what device the disk is.

[JohnAdstra]

OK! /var/log/allsky.log gives 05:44:32: This is Geany 1.37.1. Is this the value you wanted to see? Please hundreds of line of exposure values.

Mount yields that below:

Hope the info you are looking for is in there.

[EricClaeys]

@JohnAdstra If you use Geany to edit /var/log/allsky.log you should be able to search for Software to get the Allsky version. 1.37.1 is probably Geany's version.

Unfortunately I'm not enough of a Linux guru to be able to tell based on the mount output what device your disk is. What do you get by typing resize2fs? Many commands take a -h or --help option to display their options. You can try that, e.g., resize2fs --help.

It looks like you have a second disk mounted?

Eric

[JohnAdstra]

Eric,

I searched the file location and got the following:

20:51:33: This is Geany 1.37.1.

So that is correct.

And then:

pi@allskyshedcam:~ $ resize2fs resize2fs 1.46.2 (28-Feb-2021) Usage: resize2fs [-d debug_flags] [-f] [-F] [-M] [-P] [-p] device [-b|-s|new_size] [-S RAID-stride] [-z undo_file]

Not sure what selection to make here.

Yes, I do have a 64GB thumbdrive inserted. I was hoping to use this for extra storage somehow, but the AllSky files write to the SD card per standard practice.

Thanks,

John

[EricClaeys]

@JohnAdstra Other than using resize2fs once on my Pi, I'm not familiar with it. Google would probably know, or the Pi forum or GitHub page.

What file location did you search to get 10:51:33: This is Geany 1.37.1. If that line is in /var/log/allsky.log, then something overwrote the log file.

Some people have made the ~/allsky/images folder a symbolic link to a 2nd disk. You'd create a new directory on your second disk, then move the contents of ~/allsky/images/* to that directory then use ls -s path_to_the_new_directory ~/allsky/images. Basically a symbolic link acts like a file or directory, but actually points to a different location.

The only gotcha is that when you upgrade Allsky you'd need to make sure the link is still in place.

Since the initially reported Issue isn't due to Allsky, I'd like to close this Issue. You ok with that?

[JohnAdstra]

Yes. Thanks Eric. Need to troubleshoot the RPi myself. The issues were not with the AllSky ware.

Thanks for your assistance.

John

[EricClaeys]

@JohnAdstra Please let me know what you find out regarding how to use all the disk space. I'd like to update the Wiki.

Closing Issue per user. Problem is not with Allsky software, although the Allsky Wiki needs to be updated since its suggestion didn't work.

[JohnAdstra]

Eric,

After some digging, I found out that my storage problems resulted from initially using NOOBS as the OS on the RPi4b. Regardless of the SD card size, NOOBS apparently will only give you a 16GB storage drive system. Once burned into place you can’t enlarge it to another size regardless. I see there is a warning in the initial AllSky directions about NOOBS which I didn’t catch 4 or 5 months ago when I started, but that is the unfortunate result.

So I started over with Raspberry Pi Imager v1.7.2 using the 32-bit RPi OS, and it gave me nearly all of the 128GB storage on the new SD card. Now the WebUI System page is telling me I am at 9% storage, not 98%. Took a few hours to reload everything but now all is operating well.

Thanks for all your tips and clues which led to the solution.

John Kmetz

From: EricClaeys @.> Sent: Tuesday, April 26, 2022 1:44 AM To: thomasjacquin/allsky @.> Cc: JohnAdstra @.>; Mention @.> Subject: Re: [thomasjacquin/allsky] Out of Disc Space (Issue #1173)

@JohnAdstra https://github.com/JohnAdstra Please let me know what you find out regarding how to use all the disk space. I'd like to update the Wiki.

Closing Issue per user. Problem is not with Allsky software, although the Allsky Wiki needs to be updated since its suggestion didn't work.

— Reply to this email directly, view it on GitHub https://github.com/thomasjacquin/allsky/issues/1173#issuecomment-1109368031 , or unsubscribe https://github.com/notifications/unsubscribe-auth/AXFGV3KC6WLEUT6FXVP6SITVG57BHANCNFSM5UA6QBDQ . You are receiving this because you were mentioned. https://github.com/notifications/beacon/AXFGV3OC663YWJDAAGEPXRDVG57BHA5CNFSM5UA6QBD2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOIIPZZXY.gif Message ID: @. @.> >