Upload issues

[benhartwich]

Environment

• Camera: ZWO
• OS: bookworm
• Allsky version: v2023.05.01_04
• Pi model and amount of memory (8GB)

Bug Description

I have defined the upload paths and created the specific folders on the remote server. The upload of the command json file works, but the image upload doesn´t:

2024-07-28T22:02:40.378973+02:00 allsky allsky[15106]: INFO: Image /home/allsky/allsky/images/20240728-clean/image-20240728220221.jpg Saved
2024-07-28T22:02:41.958778+02:00 allsky allsky[15124]: *** upload.sh: ERROR: 'htdocs/allsky-website/img' directory not found!

But a test upload the allsky user works great:

 ~/allsky/scripts/upload.sh --debug /tmp/test.txt htdocs/allsky-website/img test.txt
Upload output from '/tmp/test.txt:
   quote: PWD: Operation not supported
drwxr-xr-x    3 root     root         4096 Jul  5  2020 .
drwxr-xr-x    3 root     root         4096 Jul  5  2020 ..
drwxr-xr-x    6 1001     1002         4096 Jul 28 19:21 htdocs
In DIRECTORY=htdocs/allsky-website/img/:
drwxr-xr-x    2 1001     1002         4096 Jul 28 19:22 .
drwxr-xr-x   11 1001     1002         4096 Jul 28 19:56 ..
-rw-r--r--    1 1001     1002           40 Jul 28 18:29 Allsky_upload_test.txt
-rw-r--r--    1 1001     1002         5649 Jul 28 18:53 configuration.json
-rw-r--r--    1 1001     1002            3 Jul 28 19:21 test.txt

I don´t know what could be the reason. Also the config file is uploaded to the path set in IMAGE_DIR, what is very odd. What can be the reason that the upload doesn´t work via sftp while manual upload works?

Log / configuration files

#!/bin/bash
    # How will files be uploaded?
PROTOCOL="sftp"

    # The directory where the current image should be copied to.
IMAGE_DIR="htdocs/allsky-website"
WEB_IMAGE_DIR="htdocs/allsky-website/img"

    # The directory where the timelapse video should be uploaded to.
VIDEOS_DIR="htdocs/allsky-website/videos"
VIDEOS_DESTINATION_NAME=""
WEB_VIDEOS_DIR="htdocs/allsky-website/videos/web"

    # The directory where the keogram image should be copied to.
KEOGRAM_DIR="htdocs/allsky-website/keogram"
KEOGRAM_DESTINATION_NAME=""
WEB_KEOGRAM_DIR="htdocs/allsky-website/keogram/web"

    # The directory where the startrails image should be copied to.
STARTRAILS_DIR="htdocs/allsky-website/startrails"
STARTRAILS_DESTINATION_NAME=""
WEB_STARTRAILS_DIR="htdocs/allsky-website/startrails/web"

############### ftp, ftps, sftp, and scp PROTOCOLS only:
    # Name of the remote server.
REMOTE_HOST="xxx.eu"

    # Optionally enter the port required by your server.
REMOTE_PORT="22"

############### ftp, ftps, and sftp PROTOCOLS only.  REMOTE_USER is also used by the scp PROTOCOL:
    # The username of the login on the remote server.
REMOTE_USER="astro"

    # The password of the login on your FTP server.  Does not apply to PROTOCOL=scp.
REMOTE_PASSWORD=""

[EricClaeys]

A couple thoughts. Using non-standard locations/names is asking for trouble. Not saying that's the issue here, but it's not helping. I suggest renaming htdocs/allsky-website to htdocs/allsky. The most recent image will go there, and the startrails, keograms, and videos directory go there.

The way you've configured ftp-settings.sh suggests you have both a REMOTE Website and a LOCAL Website. The WEB_*_DIR variable are only used for local Websites, and there is no "htdocs" directory on the Pi. Please see the Settings -> Allsky Website documentation page, specifically near the end of the "ftp-settings.sh" section, for examples of what to use for a local Website, a remote Website, and both Websites.

[EricClaeys]

FYI, this is significantly easier and less error-prone in the next release of Allsky. A LOT of people struggle with these settings - so did I when I started.

[benhartwich]

OK thanks, now it works. Is there any command to bulk upload all files again, which were not uploaded successfully?

[EricClaeys]

Glad you got it working. Only one image file is on the Website, so I assume you mean the startrails, keograms, and timelapse. How many days' files are missing? try this:

cd ~/allsky/images
for i in DATE1 DATE2   # Add all the dates, e.g, 20240727
do
   generateForDay.sh --upload ${i}
done

that will upload all 3 image types for all the dates you specify. To upload ALL dates, use for i in *

Eric

[benhartwich]

Thanks for the help - looking forward to the new release :)

[EricClaeys]

@benhartwich, please let me know how the loop I showed worked for you. Other people have asked the same question, and I should probably document the answer.

[benhartwich]

Yes it works.

[franksemi]

I had the exact same problem and yes the loop works 👍