Import function

[bobvanluijt]

The import function is non-trivial because it will need to create cross-refs that might not exist yet.

Configuration:

--data-import=./file_to_import.json # file to import, must be set, can be on disk or url
--skip-validation=bool # validation validates if the import files schema == the schema in Weaviate. Fails if not. Defaults to false.
--skip-crossref-check=bool # checks if a cross-ref already is available within the Weaviate. This is needed when an import ADDs data. Will slow the process down. Default = true
--fail-on-error=bool # should the import skip or fail on error? For example if a string value is set for an int. Defaults to false.

How the import function works;

1. Validate schema in file.
2. Add all things and actions without crossrefs. Replaces all uuids with new ones in a lookup table.
3. Creates all cross refs by checking if the provided crossref first, if it not exsists it will use the lookup table. (note: if --skip-crossref-check=true it will skip the checking part and directly looks into the lookup table).

Lookup table; The lookup table takes any uuid of a concept and replaces it with new uuid.

Definition;

Example:

[{
    "type": "thing or action",
    "@class": "Airport",
    "id": "UUID",
    "schema": {
        "code": "30000",
        "inCity": {
            "$cref": "weaviate+http://localhost/cfd0b3ea-9128-4fe5-a024-b7e68fe22206"
        }
    }
}]

[bobvanluijt]

Has a dependency on: https://github.com/creativesoftwarefdn/weaviate/issues/677