Document how to get the data even without this library (to demonstrate long-term support of stored data)

[giovannipizzi]

The following bash script does the job of extracting one object, for instance.

This should be documented in the documentation (see #3).

The requirements are:

• bash executable
• sqlite3 executable
• zlib-flate executable (from package qpdf, installable both on Ubuntu with apt install qpdf and on Mac brew install qpdf). This is actually needed only if data is zlib-compressed (unfortunately, one cannot use gzip that expects also the gzip headers, that however are redundant and not used by the object store implementation).

#!/bin/bash
CONTAINER_PATH="$1"
HASHKEY="$2"

METADATA=`sqlite3 "$CONTAINER_PATH"/packs.idx 'SELECT offset, length, pack_id, compressed FROM db_object WHERE hashkey = "'"$HASHKEY"'"'`

if [ -z "$METADATA" ]
then
    echo "No object '" $HASHKEY "' found in container."
    exit 1
fi

IFS='|' read -ra METADATA_ARR <<< "$METADATA"
OFFSET=${METADATA_ARR[0]}
LENGTH=${METADATA_ARR[1]}
PACK_ID=${METADATA_ARR[2]}
COMPRESSED=${METADATA_ARR[3]}

let OFFSET_PLUS_ONE=OFFSET+1

if [ "$COMPRESSED" == "0" ]
then
    tail -c+$OFFSET_PLUS_ONE "${CONTAINER_PATH}/packs/${PACK_ID}" | head -c"${LENGTH}"
elif [ "$COMPRESSED" == "1" ]
then
    tail -c+${OFFSET_PLUS_ONE} "${CONTAINER_PATH}/packs/${PACK_ID}" | head -c"${LENGTH}" | zlib-flate -uncompress
else
    echo "Unknown compression mode "$COMPRESSED" for object '" $HASHKEY "'"
    exit 2
fi

As a note: if one doesn't want to install zlib-flate, it can be easily replaced with a short python script using zlib.decompressobj.