[BUG]Failure on remote website install

WVAstro commented 10 months ago

Environment

Camera: RPi HQ
OS: Buster or Bullseye
Allsky version: v2023.05.01_02
Raspberry Pi 4 Model B Rev 1.5

Bug Description

*manually running "update.sh" works as expected, however normal operation of the Allsky flow fails to update the hosted website.

Can it be reproduced? YES, how: Restart Allsky and this is the default behavior
Did anything change? Brand new rebuild with all of past settings.
Screenshot on remote install attached.
See attached log excerpt:
If this is a configuration-related issue (or you are not sure), attach these files, appending ".txt" to their names first:
- Allsky Website problems:
- ~/allsky/config/remote_configuration.json (remote Website) **NOT INCLUDED SINCE INSTALL FAILS

Log / configuration files

Follow the instructions for Reporting Issues in the Wiki, then ATTACH a copy of the file(s) above. Do NOT copy/paste them into this Issue.

LOG EXCERPT.txt settings.json.txt configuration.json.txt ftp-settings.sh.txt

EricClaeys commented 10 months ago

You ran the update.sh command that's in ~/allsky? It currently doesn't work and we only distribute it as a placeholder. None of the documentation mentions it. What were you trying to do by running it?

For the remote Website did you follow the instructions in "Installation -> Allsky Website", including running website/install.sh --remote? If so, there should be a log file in ~allsky/config/installation_logs called something like "Website install". Would you please attach it?

Your "LOG EXCERPT.txt" had an error that it was unable to upload a file. Check the "Troubleshooting -> Uploads" page for how to fix.

FYI, your ftp-settings.sh attachment has your password. I suggest deleting that link and attaching a copy of the file with the username and password blanked out.

WVAstro commented 10 months ago

Thanks Eric. The Login user/pw has been changed since that file.
I did follow the remote Website guide and ran website/install.sh --remote. There is no log in the installation_logs.bkp directory for Website install. Trying again on that log excerpt txt file. LOG EXCERPT.txt

WVAstro commented 10 months ago

Screenshot of error on remote install.

EricClaeys commented 10 months ago

Bill see the troubleshooting page on uploads. The installation is working fine, it just fails when performing a test upload. Eric

WVAstro commented 10 months ago

Eric, I have followed the document several times. The "are you still having problems" section is failing with permission denied and appears to be using FTP. My settings in the FTP config file have been verified as working with Filezilla. Directories exist and are correct. I am available for troubleshooting if possible.

WVAstro commented 10 months ago

line in the x.txt that fails is attached.

X_Failure.txt

EricClaeys commented 10 months ago

Were you trying to execute the open ... command at the terminal prompt? That command is not the same as the open command that lftp uses.

If you trust me on your server, email me the ~/allsky/tmp/lftp_cmds/x.txt file and your password (please create a temporary one for me) and I will try. I suspect it's a certificate problem since it doesn't look like upload.sh can log in. Eric. AstroEric at_sign eccssw.com

WVAstro commented 10 months ago

Email sent

EricClaeys commented 10 months ago

I got it to work by following step 1 in the "Certificate-related errors" section of the Troubleshooting - Uploads page.

WVAstro commented 10 months ago

Thanks Eric. That was it totally. I was able to install the external site now and it populates I just have a few things to finalize. Appreciate your help!

EricClaeys commented 10 months ago

Glad it's working, and thanks for trusting me. Did you miss the error in the output the first time? The Troubleshooting page lists the error exactly, but I'm always looking for ways to improve the documentation or the software, so input is appreciated. If it was "user error" on your part I guarantee it'll happen to other people so I like to make the software more bullet-proof.

One idea I have is to have a testUpload.sh script that calls upload.sh and parses the output, displaying easier-to-read error messages.

WVAstro commented 10 months ago

Thanks so much - I have it all back up to where the sd crashed previously!

Sent from my iPhone Bill Brown On Aug 24, 2023, at 5:43 PM, Eric Claeys @.***> wrote:

Glad it's working, and thanks for trusting me. Did you miss the error in the output the first time? The Troubleshooting page lists the error exactly, but I'm always looking for ways to improve the documentation or the software, so input is appreciated. If it was "user error" on your part I guarantee it'll happen to other people so I like to make the software more bullet-proof.

One idea I have is to have a testUpload.sh script that calls upload.sh and parses the output, displaying easier-to-read error messages.

— Reply to this email directly, view it on GitHubhttps://github.com/thomasjacquin/allsky/issues/2931#issuecomment-1692452085, or unsubscribehttps://github.com/notifications/unsubscribe-auth/BBZYCMWZ65FFHBBOT4ZHRK3XW7DHRANCNFSM6AAAAAA332JC6U. You are receiving this because you authored the thread.Message ID: @.***>

WVAstro commented 10 months ago

I like that idea and yes it was user error.

Sent from my iPhone

On Aug 24, 2023, at 5:43 PM, Eric Claeys @.***> wrote:

Glad it's working, and thanks for trusting me. Did you miss the error in the output the first time? The Troubleshooting page lists the error exactly, but I'm always looking for ways to improve the documentation or the software, so input is appreciated. If it was "user error" on your part I guarantee it'll happen to other people so I like to make the software more bullet-proof.

One idea I have is to have a testUpload.sh script that calls upload.sh and parses the output, displaying easier-to-read error messages.

— Reply to this email directly, view it on GitHubhttps://github.com/thomasjacquin/allsky/issues/2931#issuecomment-1692452085, or unsubscribehttps://github.com/notifications/unsubscribe-auth/BBZYCMWZ65FFHBBOT4ZHRK3XW7DHRANCNFSM6AAAAAA332JC6U. You are receiving this because you authored the thread.Message ID: @.***>

EricClaeys commented 10 months ago

I just wrote a testUpload.sh script for the next major release, hopefully due around the end of the year.

WVAstro commented 10 months ago

Good deal.

Sent from my iPhone

On Aug 24, 2023, at 10:29 PM, Eric Claeys @.***> wrote:

I just wrote a testUpload.sh script for the next major release, hopefully due around the end of the year.

— Reply to this email directly, view it on GitHubhttps://github.com/thomasjacquin/allsky/issues/2931#issuecomment-1692667763, or unsubscribehttps://github.com/notifications/unsubscribe-auth/BBZYCMUSILR5H2BHSJNXR6DXXAEX3ANCNFSM6AAAAAA332JC6U. You are receiving this because you authored the thread.Message ID: @.***>

EricClaeys commented 7 months ago

This comment is no longer on GitHub. Did you delete it? If you fixed the problem, what was it?

Eric

From: Dhovin @.> Sent: Wednesday, December 6, 2023 10:51 AM To: thomasjacquin/allsky @.> Cc: Eric Claeys @.>; Comment @.> Subject: Re: [thomasjacquin/allsky] [BUG]Failure on remote website install (Issue #2931)

I'm having a little bit of a similar issue with the exception that the test file uploads fine but custom.css and data.json are failing. I tried looking at PostData.txt but there is nothing useful here that indicates why it failed.

Screenshot.2023-12-06.104919.jpg (view on web) https://github.com/thomasjacquin/allsky/assets/4074997/37e8a1fb-a99a-4851-8454-631c448dafdd

EricClaeys commented 7 months ago

Initial problem fixed.

Dhovin commented 7 months ago

Hey Eric,

I basically gave up on it for now. I reformatted my remote Pi and the remote system. I have a new camera, on order, that I am waiting to come in before I try again. It was just so infuriating because I couldn't find any reason why it wasn't working. I used the Bullseye 64-bit Lite image on my pi and Ubuntu 22.04 LTS on the remote system. I do have some more suggestions on the documentation that I think could help improve some clarity.

Below is a link to the camera. I have been really impressed by STARVIS cameras and wanted to try it out. I hope it works. https://www.arducam.com/product/2mp-imx462-color-ultra-low-light-starvis-camera-module-with-141h-wide-angle-m12-lens-for-raspberry-pi/

On Sat, Dec 9, 2023 at 2:54 AM Eric Claeys @.***> wrote:

This comment is no longer on GitHub. Did you delete it? If you fixed the problem, what was it?

Eric

From: Dhovin @.> Sent: Wednesday, December 6, 2023 10:51 AM To: thomasjacquin/allsky @.> Cc: Eric Claeys @.>; Comment @.> Subject: Re: [thomasjacquin/allsky] [BUG]Failure on remote website install (Issue #2931)

I'm having a little bit of a similar issue with the exception that the test file uploads fine but custom.css and data.json are failing. I tried looking at PostData.txt but there is nothing useful here that indicates why it failed.

Screenshot.2023-12-06.104919.jpg (view on web) < https://github.com/thomasjacquin/allsky/assets/4074997/37e8a1fb-a99a-4851-8454-631c448dafdd>

— Reply to this email directly, view it on GitHub < https://github.com/thomasjacquin/allsky/issues/2931#issuecomment-1843281428> , or unsubscribe < https://github.com/notifications/unsubscribe-auth/AT2PYK33JEHNNN5JWZSVI7DYICPAJAVCNFSM6AAAAAA332JC6WVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQNBTGI4DCNBSHA> . You are receiving this because you commented. < https://github.com/notifications/beacon/AT2PYK43WM7VTHD65RQQ7JLYICPAJA5CNFSM6AAAAAA332JC6WWGG33NNVSW45C7OR4XAZNMJFZXG5LFINXW23LFNZ2KUY3PNVWWK3TUL5UWJTTN3Y7BI.gif> Message ID: @. @.> >

— Reply to this email directly, view it on GitHub https://github.com/thomasjacquin/allsky/issues/2931#issuecomment-1848331380, or unsubscribe https://github.com/notifications/unsubscribe-auth/AA7C35NRFDVZ3QSGP5MMU5DYIQRMZAVCNFSM6AAAAAA332JC6WVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQNBYGMZTCMZYGA . You are receiving this because you commented.Message ID: @.***>

EricClaeys commented 7 months ago

@Dhovin, suggestions for improvement are ALWAYS welcomed. Please try to be as specific as possible and add them in a new Discussion item. I will close this one since the initial problem was resolved.

If I remember your Issue it was that during installation of a remote Website (website/install.sh --remote) the test file uploaded fine but not the other files. Correct? That's a strange one. I'd appreciate it if you would work with me to get it resolved because other people will likely have the same problem. If it's user error I would like to improve Allsky so it can detect that error and provide better feedback. Ok? If so, would you please create a new Discussion with as much info as possible?

Allsky doesn't currently support the IMX462. Once you receive it, please run

libcamera --list-cameras -v > IMX462.txt

Then attach the IMX462.txt file to a new Issue (not Discussion) called "Support IMX462". This will give me the info needed to support the camera.

Thanks

AllskyTeam / allsky