Support STOREBASE from named Docker volume

[jennydaman]

Motivation

There is no clean answer to the STOREBASE problem. The current solution to export STOREBASE as the path on the host is configured outside of docker-compose.yml, requiring a wrapper script around docker-compose up and later ./unmake.sh <<< y; rm -fr FS; sudo rm -fs FS; to remove ./FS/remote.

https://github.com/FNNDSC/ChRIS_ultron_backEnd/blob/78670f6abf0b6ebac7aeef75989893b4502d4823/docker-compose_dev.yml#L208-L222

Usage

A discoverable host path can be designated by the use of a named docker volume. It's completely managed by docker-compose and can be cleaned up seamlessly by docker-compose down -v.

services:
  pfioh:
    image: fnndsc/pfioh:latest
    command: ["--forever", "--httpResponse", "--createDirsAsNeeded", "--storeBase", "/hostFS/storeBase"]
    volumes:
      - remote:/hostFS/storeBase
 pman:
    image: fnndsc/pman:latest
    environment:
      - PMAN_DOCKER_VOLUME=chris-remote
    command: ["--rawmode", "1", "--http", "--port", "5010", "--listeners", "12", "--verbosity", "1"]
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
      - remote:/hostFS/storeBase
volumes:
  remote:
    name: chris-remote

Demo

This demo breaks down what is automatically configured and executed by docker-compose up.

1. Create Volume

docker volume create pfioh-remote

2. Start pman

docker pull fnndsc/swarm
docker swarm init --advertise-addr=127.0.0.1
docker build -t local/pman:storebase-volume
docker run --rm --name pman -v /var/run/docker.sock:/var/run/docker.sock -v pfioh-remote:/wow/hostFS:ro -e PMAN_DOCKER_VOLUME=pfioh-remote -p 5010:5010 local/pman:storebase-volume

3. Mock pfioh incoming data

docker run --rm -v pfioh-remote:/hostFS/storeBase:rw --entrypoint /bin/bash fnndsc/ubuntu-python3 -c "cd /hostFS/storeBase && mkdir a_folder && touch a_folder/why_hello_there"

4. Schedule a swarm job to operate on date inside volume

pfurl --verb POST --raw --http localhost:5010/api/v1/cmd --jsonwrapper 'payload' --msg \
'{  "action": "run",
    "meta":  {
        "cmd":      "ls /share",
        "auid":     "rudolphpienaar",
        "jid":      "not-a-uuid-2",
        "threaded": true,
        "container":   {
                "target": {
                    "image":        "fnndsc/pl-simpledsapp"
                },
                "manager": {
                    "image":        "fnndsc/swarm",
                    "app":          "swarm.py",
                    "env":  {
                        "shareDir":     "/wow/hostFS/a_folder",
                        "serviceType":  "docker",
                        "serviceName":  "testService"
                    }
                }
        }
    }
}'

5. Assert data was visible to job

pfurl --verb POST --raw --http localhost:5010/api/v1/cmd --jsonwrapper 'payload' --msg \
'{  "action": "status",
        "meta": {
                "key":          "jid",
                "value":        "not-a-uuid-2"
        }
}'

In the response we expect l_logs to contain why_hello_there

{                                                                                                     
    "RESTheader": "POST /api/v1/cmd HTTP/1.1\r",                                                      
    "RESTverb": "POST",                                                                               
    "action": "status",                                                                               
    "client_json_len": 82,                                                                            
    "client_json_payload": "{\"payload\": {\"action\": \"status\", \"meta\": {\"key\": \"jid\", \"value\": \"not-a-uuid-2\"}}}",                                                                            
    "d_ret": {                                                                                        
        "0.container": {                                                                              
            "jobRoot": "20210116023319.246058_f91db8e9-1518-4b80-8e82-c1d8db064763",                  
            "tree": {                                                                                 
                "20210116023319.246058_f91db8e9-1518-4b80-8e82-c1d8db064763": {                       
                    "container": {                                                                    
                        "manager": {                                                                  
                            "app": "swarm.py",                                                        
                            "env": {                                                                  
                                "serviceName": "testService",                                         
                                "serviceType": "docker",                                              
                                "shareDir": "/wow/hostFS/a_folder"                                    
                            },                                                                        
                            "image": "fnndsc/swarm"                                                   
                        },                                                                            
                        "target": {                                                                   
                            "image": "fnndsc/pl-simpledsapp"                                          
                        }                                                                             
                    }                                                                                 
                }                                                                                     
            }                                                                                         
        },                                                                                            
        "0.end": {                                                                                    
            "jobRoot": "20210116023319.246058_f91db8e9-1518-4b80-8e82-c1d8db064763",                  
            "returncode": [                                                                           
                0                                                                                     
            ]                                                                                         
        },                                                                                            
        "0.start": {                                                                                  
            "jobRoot": "20210116023319.246058_f91db8e9-1518-4b80-8e82-c1d8db064763",                  
            "startTrigger": [                                                                         
                true                                                                                  
            ]                                                                                         
        },                                                                                            
        "l_logs": [                                                                                   
            "why_hello_there\r\n"                                                                     
        ],
        "l_status": [
            "finishedSuccessfully"
        ]
    },
    "meta": {
        "key": "jid",
        "value": "not-a-uuid-2"
    },
    "path": "/api/v1/cmd",
    "payloadsize": 82,
    "receivedByServer": [
        "POST /api/v1/cmd HTTP/1.1\r",
        "Host: localhost:5010\r",
        "User-Agent: PycURL/7.43.0.6 libcurl/7.74.0 OpenSSL/1.1.1i zlib/1.2.11 zstd/1.4.5 libidn2/2.3.0 libpsl/0.21.1 (+libidn2/2.3.0) libssh2/1.9.0 nghttp2/1.41.0\r",
        "Accept: */*\r",
        "Mode: control\r",
        "Content-Length: 82\r",
        "Content-Type: application/x-www-form-urlencoded\r",
        "\r",
        "{\"payload\": {\"action\": \"status\", \"meta\": {\"key\": \"jid\", \"value\": \"not-a-uuid-2\"}}}"
    ],
    "status": true
}

Notes

It is not strictly necessary to mount pfioh-remote:/wow/hostFS:ro inside pman however pman does an assertion for the path's existence: if it does not exist, then pman sleeps for 20 seconds before firing the job anyways.