Import Documents with JSON

[JoelProminic]

I remembered that we had a feature in Moonshine to quickly import some Domino data. I'll update this project with some example JSON to use for that.

[JoelProminic]

@Aszusz, @piotrzarzycki21, I added some example JSON and updated README.md in this branch: https://github.com/Moonshine-IDE/PieChartExample/tree/features/issue_2_import_json

I got the data from the DevExpress example, but we can change it if desired.

Unfortunately, I found that the capability was missing on the Vagrant side. I have an issue to fix this.

If you want to test this manually, then vagrant ssh to the VM and create /opt/domino/scripts/import_json_documents.sh:

#!/bin/bash
# Import a JSON file containing documents into database(s) using Genesis.

USAGE="./import_json_documents.sh <json-file>"

set -e

if [ "$#" -lt 1 ]; then
    echo "At least one JSON file required.  USAGE:"
    echo "    $USAGE"
    exit 1
fi

# Parameters
SOURCE_FILE=$1

# Genesis directories
GENESIS_DIR=/local/notesdata/JavaAddin/Genesis
JSON_TRIGGER_DIR=$GENESIS_DIR/json
JSON_RESPONSE_DIR=$GENESIS_DIR/jsonresponse

importJsonFile() {
    # parameters
    JSON_SOURCE_FILE=$1

    # validate the file
    if jq -reM '""' "$JSON_SOURCE_FILE" 2>&1; then
        echo "No errors found in file '$JSON_SOURCE_FILE'";
    else
        echo "Invalid JSON:  '$JSON_SOURCE_FILE'"
        exit 1
    fi

    # Ensure directories exist
    sudo mkdir -p "$JSON_TRIGGER_DIR"
    sudo mkdir -p "$JSON_RESPONSE_DIR"
    sudo chown domino:domino "$JSON_TRIGGER_DIR"
    sudo chown domino:domino "$JSON_RESPONSE_DIR"

    # extract the file name
    JSON_BASE_NAME=`basename -s .json "$JSON_SOURCE_FILE"`
    # build a unique target name and path
    TIMESTAMP=`date +%Y%m%d%H%M%S`
    JSON_TARGET_NAME="${TIMESTAMP}_${JSON_BASE_NAME}.json"
    JSON_TARGET_FILE=$JSON_TRIGGER_DIR/$JSON_TARGET_NAME

    # Start operation
    sudo chown domino:domino "$JSON_SOURCE_FILE"
    sudo cp "$JSON_SOURCE_FILE" "$JSON_TARGET_FILE"
    echo "Starting import:  $JSON_TARGET_FILE"

    # Wait for operation to complete
    ALLOWED_RETRIES=3
    RETRIES=0
    while [ -e $JSON_TARGET_FILE ]; # && "$RETRIES" -lt "$ALLOWED_RETRIES" ];
    do
        sleep 1;
        RETRIES=$((RETRIES+1));
    done
    if [ "$RETRIES" -ge "$ALLOWED_RETRIES" ]; then
        echo "Timeout on import attempt.";
        exit 1;
    fi

    # check response
    JSON_RESPONSE_FILE="$JSON_RESPONSE_DIR/$JSON_TARGET_NAME"
    RESPONSE=$(cat "$JSON_RESPONSE_FILE" | tr -d '\n')
    if [ "$RESPONSE" = "OK" ]; then
        echo "Import successful."
    else
        echo "Deployment failed:  '$RESPONSE'"
        exit 1
    fi
}

importJsonFile "$SOURCE_FILE"

Then update /opt/rest-interface/config/application.yml to add the capability. You will need to restart the VM to see the capability:

grails:
    controllers:
        upload:
            maxFileSize: 104857600
            maxRequestSize: 104857600
restinterface:
    capabilities:
        - upload-database
        - upload-html
        - java-domino-gradle
        - nsfodp
        - import-domino-json
    serverName: demo/DEMO
    baseURL: https://127.0.0.1:8080

[piotrzarzycki21]

@JoelProminic I have changed example to the one which are part of Apache ECharts examples - you can take a look for the data here in series. Previously example was taken from one of the DevExpress demo project.

[JoelProminic]

The import-domino-json capability is not available on the Super.Human.Installer development branch, if you want to include it in the example.

I updated the data to match the new example. The graph label has not been updated yet on this branch:

[piotrzarzycki21]

@JoelProminic I have merged your changes to main. I have tested with new development build of SHI 0.8.23 - everything worked perfectly. We will add this to article once we release next production version of SHI. Meanwhile you can close issue if you want.