Build REST interface for QCFractal

[shantenujha]

.

[dgasmith]

Queue REST API PR: https://github.com/MolSSI/QCFractal/pull/72

[dgasmith]

With a fresh install of QCFractal

Run qcfractal-server mydb in the background, this will assume that the MongoDB URI of "MongoDB://localhost:27017". The following should be seen:

[W 181024 13:51:03 server:125] No SSL files passed in, generating self-signed SSL certificate.
[W 181024 13:51:03 server:126] Clients must use `verify=False` when connecting.

[I 181024 13:51:03 server:165] Connected to 'mongodb://localhost'' with database name 'mydb'
    .
[I 181024 13:51:03 server:222] FractalServer successfully initialized at https://localhost:7777/
[I 181024 13:51:03 server:230] FractalServer successfully started. Starting IOLoop.

Many tasks can be added with:

import qcfractal

client = qcfractal.interface.FractalClient("https://localhost:7777/", verify=False)

molecules = []
for x in range(1, 20):
    molecules.append({"symbols": ["He", "He"], "geometry": [0, 0, -x, 0, 0, x]})

ret = client.add_compute("psi4", "b3lyp", "cc-pVDZ", "energy", "default", molecules)
print(ret) # Returns ids of submitted jobs
{'submitted': ['5bd0b19c7b878703d376e6f8', '5bd0b19c7b878703d376e6f1', '5bd0b19c7b878703d376e6ee', ...}

---

Starting a requests instance as:

import json
import requests

# Turn off SSL 3rd party verification warnings
from urllib3.exceptions import InsecureRequestWarning
requests.packages.urllib3.disable_warnings(category=InsecureRequestWarning)

address = "https://localhost:7777/"

New tasks can be pulled via:

# Pull data
payload = {
    "meta": {
        "name": "testing_manager",    # The name of the manager, usually (userlabel + hostname + uuid)
        "tag": None,    # The "tag" or queue category to pull from
        "limit": 1    # The number of tasks to pull
    },
    "data": {}
}
r = requests.get(address + "queue_manager", json=payload, verify=False)

tmp = r.json()
tmp["data"][0]["spec"]["args"] = None    # Collapse the input
print(json.dumps(tmp, indent=2))
{
  "meta": {
    "n_found": 1,
    "success": true
  },
  "data": [
    {
      "hash_index": "4cd3f69f5f1259e274010c1c04e7f142fce876a5",
      "spec": {
        "function": "qcengine.compute",
        "args": null,
        "kwargs": {}
      },
      "hooks": [],
      "parser": "single",
      "id": "5bd0b19c7b878703d376e6fc"
    }
  ]
}

---

Complete tasks can be pushed:

# Push data
# Pull data
payload = {
    "meta": {
        "name": "testing_manager",    # The name of the manager, usually (userlabel + hostname + uuid)
    },
    "data": {x["id"]: ("return_data", x["parser"], x["hooks"])
             for x in tmp["data"]}
    # Current we trace a bit more of parsers/hooks back and forther which will eventally be removed.
    # This will likely be simplified down to a list of result JSONs directly from QCEngine
}
r = requests.post(address + "queue_manager", json=payload, verify=False)
print(r.json())
{'meta': {'n_inserted': 1}, 'data': True}

---

Finally, tasks can be returned to the queue:

payload = {
    "meta": {
        "name": "testing_manager",    # The name of the manager, usually (userlabel + hostname + uuid)
        "operation": "shutdown"    # Returns any remaining jobs associated with this name to the queue
    },
    "data": []
}
r = requests.put(address + "queue_manager", json=payload, verify=False)
print(r.json())
{'meta': {}, 'data': True}

This is our current API which will likely undergo quite a few changes as we make enhancements and improvements to the current capabilities.

[dgasmith]

I should note that at the moment we have managers that communicate between Dask/Fireworks and QCFractal. If a QCFractal server is running on can access them via the following CLI:

qcfractal-manager fireworks

The manager code can be found here with adapters for dask and fireworks.

[mturilli]

Schema will be stable from now on. Waiting for RADICAL feed back.

[mturilli]

Done.