Serve images as regular files

[frafra]

This is a suggestion for a potential improvement. Images are stored in a bytes column and then served in a JSON encoded as base64 data. Serving images as a regular file would make it easier to load them from different clients and reduce overhead.

[frafra]

Simple workaround to fetch an image, using jq, httpie and bash:

#!/bin/bash
url="$1"
credentials="$2"
id=$3

res=$(http "$url/data/images/$id" --auth "$credentials")
filename=$(jq -r '(.id|tostring) + "_" + .name' <<< "$res")
jq -r '.data' <<< "$res" | base64 -d > "$filename"

Usage (fetches image ID 123): bash fetch-gss-image.sh https://myserver username:password 123

[frafra]

There is no way to fetch all the images, except opening all the notes or looking at the database. Assuming that IDs of the images are serial numbers incremented by one and without any gap, I wrote this quick and dirt solution to save all the images:

#!/bin/bash

url="$1"
credentials="$2"

while true
do
    : $(( id++ ))
    echo "Fetching image n. $id"
    res=$(http "$url/data/images/$id" --auth "$credentials")
    jq -e '.id' <<< "$res" &> /dev/null || exit
    filename=$(jq -r '(.id|tostring) + "_" + .name' <<< "$res")
    jq -r '.data' <<< "$res" | base64 -d > "$filename"
done

Usage: bash fetch-all-gss-images.sh https://myserver username:password

Note: GSS returns HTTP 200 with an empty JSON response even if the image is missing.

This command can be useful to add leading zeros: rename '' 0 ?_*

[frafra]

I just discovered I was just downloading the thumbnails. Images are served over /imagedata: https://github.com/geopaparazzi/GSS/blob/9f39c39208cc655d16b0706b2e13310fc164c1a9/flutter_server_backbone/src/main/java/com/hydrologis/kukuratus/gss/GssServerApiJavalin.java#L102

It takes a size parameter as second argument, and it is not possible to retrieve the original image: https://github.com/geopaparazzi/GSS/blob/9f39c39208cc655d16b0706b2e13310fc164c1a9/flutter_server_backbone/src/main/java/com/hydrologis/kukuratus/gss/GssServerApiJavalin.java#L818

[frafra]

Here is a Python script to dump all the images from the database (it requires psycopg2):

#!/usr/bin/env python3

import psycopg2
import sys

query = '''
select images.id, text, data
  from images
  join imagedata on images.id=imagedata.id
'''

with psycopg2.connect(dsn=sys.argv[1]) as conn:
    with conn.cursor() as curs:
        curs.execute(query)
        for uid, filename, data in curs:
            with open('%d-%s' % (uid, filename), 'wb') as image:
                image.write(data)

Usage: python3 fetch-all-gss-images.py postgresql://postgres:PASSWORD@SERVER/DATABASE

[moovida]

Nice! :-)

Just a note. It is now (from the repo) possible to get the original image, if size is not given as parameter.

[frafra]

As there is now an endpoint to get the original images without decoding or JavaScript parsing, I would close this issue. Thank you :)

[moovida]

:-) Still your scripts can be of help.