Hosting website on Pi3 but video and images not saved to var/www/html/allsky folders

[ShianMull]

Hi All,

I'm puzzling the last issue before I package the allsky camera and put it outside, I have installed Thomas's website on the Pi3 I'm using and made appropriate changes to ftp script, but the compiled images and timelapse etc are not copied to the html folders. The live view is also not updated and I'm scratching my head here and the scripts associated with this function seem to be working fine.

Has this issue been seen before and resolved - I couldn't see anything on the closed issues.

Many thanks for your help.

Mark

[EricClaeys]

Mark, Are you using allsky version 0.8 and allsky-portal version 0.7? If so, please post your /home/pi/allsky/config, /home/pi/allsky/scripts/ftp_settings.sh, and /etc/raspap/*ZWO.json files. The problem is probably in your settings.

When you say "Thomas's website", what is the URL?

Eric

[ShianMull]

Hi Eric,

/home/pi/allsky/config:

!/bin/bash

Choose between ZWO or RPiHQ or auto (prefers ZWO if present)

CAMERA="auto"

Set to true to upload current image to your website

UPLOAD_IMG=true

Set to true to upload timelapse video to your website at the end of each night

UPLOAD_VIDEO=true

Set to true to send data to your server at the end of each night

POST_END_OF_NIGHT_DATA=true

Set to true to generate a timelapse at the end of the night

TIMELAPSE=true

Set the width and height parameter for timelapse generation (sizes must be divisible by 2)

TIMELAPSEWIDTH=0 TIMELAPSEHEIGHT=0

Timelapse frame rate (number of frames per second)

FPS=25

Set to true to generate a keogram at the end of the night (image summary of the night)

KEOGRAM=true

Set to true to upload the keogram after it has been generated

UPLOAD_KEOGRAM=true

Set to true to generate a startrails image of the night. Will skip brighter images to avoid over-exposure

STARTRAILS=true

Images with a brightness higher than the threshold will be skipped for startrails image generation

BRIGHTNESS_THRESHOLD=0.1

Set to true to upload the startrails after it has been generated

UPLOAD_STARTRAILS=true

Set to true to enable automatic deletion of archived data (images + videos)

AUTO_DELETE=true

Set this value to the number of archived nights you want to keep. Needs AUTO_DELETE=true to work

NIGHTS_TO_KEEP=7

Set to true if you want to enable dark frame subtraction on your night time images

DARK_FRAME_SUBTRACTION=false

Set to 0 to disable Daytime Capture

DAYTIME="1"

Set 24Hr capture to true to save both night and day images to disk. By default, only night images are saved.

CAPTURE_24HR=false

Resize images before cropping and saving. Adjust width and height according to your own sensor ratio

IMG_RESIZE=false IMG_HEIGHT=1520 IMG_WIDTH=2028

Crop the captured image, used to improve the images when using a fisheye lens

CROP_IMAGE=false CROP_WIDTH=640 CROP_HEIGHT=480 CROP_OFFSET_X=0 CROP_OFFSET_Y=0

Auto stretch images saved at night

AUTO_STRETCH=false AUTO_STRETCH_AMOUNT=10 AUTO_STRETCH_MID_POINT=10%

Path to the camera settings (exposure, gain, delay, overlay, etc)

CAMERA_SETTINGS_DIR="/etc/raspap"

if [[ $CAMERA -eq "auto" ]]; then

restore currently saved autodiscovered camera mode if any

source "$ALLSKY_HOME/autocam.sh" fi CAMERA_SETTINGS="$CAMERA_SETTINGSDIR/settings$CAMERA.json"

ftp_settings.sh

!/bin/bash

FTP/SFTP/S3 settings

PROTOCOL='local' USER='username' PASSWORD='password' HOST='example.com' IMGDIR = '/var/www/html/allsky/' MP4DIR = '/var/www/html/allsky/videos' KEOGRAM_DIR = '/var/www/html/allsky/keograms' STARTRAILS_DIR = '/var/www/html/allsky/startrails'

S3 settings (Set PROTOCOL='S3'above and configure DIR settings above also as required.)

You will need to install the AWS CLI:

sudo apt-get install python3-pip

pip3 install awscli --upgrade --user

export PATH=/home/pi/.local/bin:$PATH

aws configure

Enter a valid access key ID, Secret Access Key and Default region name (e.g. us-west-2) when prompted.

Set the Default output format to json when prompted

#

AWS CLI directory where the AWS CLI tools are installed.

AWS_CLI_DIR='/home/pi/.local/bin'

Name of S3 Bucket where the files are to be uploaded (must be in Default region specified above).

You may want to turn off or limit bucket versioning to avoid consuming lots of space with multiple versions of the live 'image-resize.*' file.

S3_BUCKET='allskybucket'

S3_ACL is set to private by default. If you want to serve your uploaded files vis http(s), change S3_ACL to 'public-read'.

You will need to ensure the S3 bucket policy is configured to allow public access to objects with a public-read ACL.

You may need to set a CORS policy in S3 if the files are to be accessed by Javascript from a different domain.

S3_ACL='private'

…..ZWO.json

[ { "name":"width", "default":0, "description":"Image width. 0 = max width", "label":"Width", "type":"number" }, { "name":"height", "default":0, "description":"Image height. 0 = max height", "label":"Height", "type":"number" }, { "name":"exposure", "default":10000, "description":"Time in ms (1000ms = 1 sec)", "label":"Exposure", "type":"number" }, { "name":"maxexposure", "default":20000, "description":"Max exposure time in ms (1000ms = 1 sec)", "label":"Max Exposure", "type":"number" }, { "name":"autoexposure", "default":"1", "description":"Activate to enable auto-exposure", "label":"Auto-Exposure", "type":"checkbox" }, { "name":"gain", "default":"50", "description":"Light sensitivity. Starts at 0. A high value returns a brighter image but more noise too", "label":"Gain", "type":"number" }, { "name":"maxgain", "default":200, "description":"Upper gain limit when using auto-gain", "label":"Max Gain", "type":"number" }, { "name":"autogain", "default":"0", "description":"Activate to enable auto-gain", "label":"Auto-Gain", "type":"checkbox" }, { "name":"coolerEnabled", "default":"0", "description":"Activate to use cooling (works only on cooled cameras)", "label":"Cooling", "type":"checkbox" }, { "name":"targetTemp", "default":"0", "description":"Sensor's target temperature when cooler is enabled (degrees Celsius)", "label":"Target Temp.", "type":"number" }, { "name":"gamma", "default":"50", "description":"0 to 100. Changes the difference between light and dark areas. A hight value increases contrast", "label":"Gamma", "type":"number" }, { "name":"brightness", "default":"50", "description":"0 to 100. Changes the amount of light in the image. Low value = dark. high value = bright", "label":"Brightness", "type":"number" }, { "name":"wbr", "default":"53", "description":"Red component for the White Balance", "label":"Red balance", "type":"number" }, { "name":"wbb", "default":"90", "description":"Blue component for the White Balance", "label":"Blue balance", "type":"number" }, { "name":"bin", "default":"1", "description":"1 = binning OFF (1x1), 2 = 2x2 binning, 4 = 4x4 binning. Check available binning mode for your camera", "label":"Binning", "type":"number" }, { "name":"delay", "default":"10", "description":"Delay between images in milliseconds. 1000ms = 1 sec.", "label":"Delay", "type":"number" }, { "name":"daytimeDelay", "default":"5000", "description":"Delay between images during daytime in milliseconds. 1000ms = 1 sec.", "label":"Daytime Delay", "type":"number" }, { "name":"type", "default":"1", "description":"Choose between Color (RGB24) or Grayscale (RAW8, RAW16). 16 bits only works with PNG extension", "label":"Image Type", "type":"select", "options": [ {"value": "0", "label": "RAW8"}, {"value": "1", "label": "RGB24"}, {"value": "2", "label": "RAW16"} ] }, { "name":"quality", "default":"95", "description":"Range for a PNG image: 0-9. Range for a JPG image: 0-100", "label":"Quality", "type":"number" }, { "name":"usb", "default":"40", "description":"Values between 40-100. This is BandwidthOverload", "label":"USB bandwidth", "type":"number" }, { "name":"filename", "default":"image.jpg", "description":"No whitespaces in filename. Extensions allowed: .jpg or .png", "label":"Filename", "type":"text" }, { "name":"flip", "default":"0", "description":"Select a flipping option if you want to reverse the image along an axis", "label":"Flip", "type":"select", "options": [ {"value": "0", "label": "Original"}, {"value": "1", "label": "Horizontal"}, {"value": "2", "label": "Vertical"}, {"value": "3", "label": "Both"} ] }, { "name":"text", "default":"", "description":"Text Overlay", "label":"Text Overlay", "type":"text" }, { "name":"extratext", "default":"", "description":"Extra Text File (Requires the full path to the file) If you are not using an extra text file ensure this field is empty", "label":"Extra Text File", "type":"text" }, { "name":"extratextage", "default":"600", "description":"The maximum age of the extra text file in seconds. After this time the file will no longer be dispalyed. Set to 0 to always display", "label":"Max Age Of Extra", "type":"number" }, { "name":"textlineheight", "default":"30", "description":"The line height of the text, if increasing the font size causes the text overlap then increase this value", "label":"Line Height", "type":"number" }, { "name":"textx", "default":"15", "description":"Text Placement Horizontal from LEFT in Pixels", "label":"Text X", "type":"number" }, { "name":"texty", "default":"35", "description":"Text Placement Vertical from TOP in Pixels", "label":"Text Y", "type":"number" }, { "name":"fontname", "default":"0", "description":"Font Types for the text overlay", "label":"Font Name", "type":"select", "options": [ {"value": "0", "label": "Simplex"}, {"value": "1", "label": "Plain"}, {"value": "2", "label": "Duplex"}, {"value": "3", "label": "Complex"}, {"value": "4", "label": "Triplex"}, {"value": "5", "label": "Complex Small"}, {"value": "6", "label": "Script Simplex"}, {"value": "7", "label": "Script Complex"} ] }, { "name":"fontcolor", "default":"255 255 255", "description":"BGR value of text color. 0 to 255 for each channel. NOTE: When using RAW 16 only the B and G values are used i.e. 255 128 0", "label":"Font Color", "type":"text" }, { "name":"smallfontcolor", "default":"0 0 255", "description":"BGR value of text color. 0 to 255 for each channel. NOTE: When using RAW 16 only the B and G values are used i.e. 255 128 0", "label":"Small Font Color", "type":"text" }, { "name":"fonttype", "default":"0", "description":"Choose between smooth font lines (antialiased), 8-pixel or 4 pixel connectivity (harder edges)", "label":"Font Smoothness", "type":"select", "options": [ {"value": "0", "label": "Antialiased"}, {"value": "1", "label": "8 connected"}, {"value": "2", "label": "4 connected"} ] }, { "name":"fontsize", "default":"7", "description":"Text Font Size", "label":"Font Size", "type":"number" }, { "name":"fontline", "default":"1", "description":"Text Font Line Thickness", "label":"Font Weight", "type":"number" }, { "name":"outlinefont", "default":"0", "description":"Adds an outline to the text overlay to improve contrast", "label":"Use Outline Font", "type":"checkbox" }, { "name":"latitude", "default":"60.7N", "description":"Latitude of the camera. Needs a number (0 to 90) and direction (N or S). Ex: 60.7N", "label":"Latitude", "type":"text" }, { "name":"longitude", "default":"135.05W", "description":"Longitude of the camera. Needs a number (0 to 180) and direction (E or W). Ex: 135.05W", "label":"Longitude", "type":"text" }, { "name":"angle", "default":"-6", "description":"Altitude of the sun in degrees relative to the horizon at which to start/stop capture", "label":"Angle", "type":"text" }, { "name":"time", "default":"1", "description":"Adds the time to the image. Combine with Text X and Text Y for placement", "label":"Time", "type":"checkbox" }, { "name":"darkframe", "default":"0", "description":"Activate to capture a dark frame. This mode hides text and time overlays", "label":"Dark Frame", "type":"checkbox" }, { "name":"showDetails", "default":"1", "description":"Activate to display the image metadata on the overlay", "label":"Show Details", "type":"checkbox" }

]

I meant Thomas’s default website that is copied onto the local RasPi as part of the install.sh - I think it just ports the website over.

All the best

Mark

[ShianMull]

Hi Eric,

I realised I had made an error in the FTP_settings.sh and included “spaces” in between the “=“ and the directory names - rectified and now Startrails and Keograms upload fine.

The live view is also uploading to var/www/html/allsky OK, but the file name is “image-resize.jpg”, which isn’t being displayed on the default webpage instant view.

There are also no time lapse videos being saved to the default MP4 directly for the website, so I suspect there is a file name/directory mismatch for the video and the live image.

Would you be able to point me in the right direction to check what name the website uses for the live image, and any thoughts on why the time lapse isn’t uploading?

Many thanks for your help.

Mark

[EricClaeys]

Mark, There are two websites:

1. The GUI site where you can display rhe "Liveview", change camera settings, and do other things. This is the "allsky-portal"Git package. Dou you see the Live image? If not, right click on the little icon and open in a new window and look at the URL to see what it is looking for.
2. The website where you can view a constellation overview. This is Git "allsky-website". Do you see this image? If not, do the same as above. Also modify the filename in /var/www/html/allsky/config.js.

[ShianMull]

Thanks Eric,

GUI website is working fine, no problem with live view.

I’ve already tried editing config.js to change “image.jpeg” to “image-resize.jpeg” (the posted file that seems to update as expected as a file), but this hasn’t resulted in the live image being displayed on the Pi hosted website (with the constellation view)…

The other issue is that the Timelapse video (.mp4) is not being copied to the var/www/html/allsky/videos folder. Keogram and Startrial pictures are being copied fine, so I suspect it is also the video file name causing issues on this.

Many thanks for your help.

Mark

[ShianMull]

Update before the bedtime! I have edited both website script and timelapse.sh to use a consistent name. Not sure why I have scripts that use differing file names to each other, but it must have been down to installing website separately from GUI and original install script. Will update if successful, but I think your pointers, Eric, have helped me identify the salient issues - many thanks.

Mark

[ShianMull]

Everything except the copy of time lapse works now, but my recommendation is that the website is not hosted from the Pi (Pi4 might be more capable), as it probably puts more workload on the flash drive than necessary.

The issue I have found is tracking what files are saved where under which config file when using GUI vs command prompt. I guess one needs to keep up to date on the various updates, so my suggestion to anyone making this excellent project is that it’s worth making your own note of what is installed where and how it’s edited to keep on top of things 😊