typedb-osi / typedb-loader

TypeDB Loader - Data Migration Tool for TypeDB
https://github.com/typedb-osi/typedb-loader
Apache License 2.0
58 stars 17 forks source link

Current Implementation of RelationInsertGenerator.playersMatch does not cast player tokens to corresponding conceptGenerator idValueType #10

Closed colt-jay closed 3 years ago

colt-jay commented 3 years ago

Context

I am trying to load in a relation table that looks like the following:

# tag.csv

label_name,text_id
blue,28974
flower,83682
might,263684
...

Quick and dirty sample schema:

# schema.gql

uid sub attribute,
    value long;

name sub attribute,
    value string;

text sub entity,
    key uid,
    plays tagged;

label sub entity,
    key name,
    plays tagger;

tag sub relation,
    relates tagger,
    relates tagged;
# processorConfig.json

"tagged": {
            "playerType": "text",
            "uniquePlayerId": "uid",
            "idValueType": "long",
            "roleType": "tagged",
            "required": true
          }

The label and text entity loaders have worked fine however upon attempting to migrate this tag.csv file I am seeing the following error:

java.util.concurrent.ExecutionException: grakn.client.exception.GraknClientException: UNKNOWN: grakn.core.kb.graql.exception.GraqlSemanticException: 
Value 203088375 is not compatible with attribute value type: java.lang.Long. 
Please check server logs for the stack trace.

Suspected Error

I suspected that the match query was not correctly casting the text_id to long and found that currently the token is passed through the cleanToken method here https://github.com/bayer-science-for-a-better-life/grami/blob/f98b2b4e33094eb33275bd7fb6d1cc3676edc51a/src/main/java/generator/RelationInsertGenerator.java#L148 but that there is no further cast statement that I can find.

I was curious why this didn't break on the entity insertion but it seems that you do cast the value in the EntityInsertGenerator here https://github.com/bayer-science-for-a-better-life/grami/blob/f98b2b4e33094eb33275bd7fb6d1cc3676edc51a/src/main/java/generator/GeneratorUtil.java#L64

hkuich commented 3 years ago

@colt-jay : awesome catch - I just realized that I only have strings for my keys/identifying attributes. And thanks for the nice tracing of the bug! I just rewrote that whole section for the next version, but I will provide a hotfix asap.

hkuich commented 3 years ago

reprodced:

09:34:54.237 [Test worker] INFO  com.bayer.dt.grami - nothing previously migrated for [tag] - starting with row 0
java.util.concurrent.ExecutionException: grakn.client.exception.GraknClientException: UNKNOWN: grakn.core.kb.graql.exception.GraqlSemanticException: Value 28974 is not compatible with attribute value type: java.lang.Long. Please check server logs for the stack trace.
    at java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:357)
    at java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1908)
hkuich commented 3 years ago

image

hkuich commented 3 years ago

@colt-jay : please use GraMi 0.0.2-hotfix-1 and let me know if this resolves your issue.

colt-jay commented 3 years ago

@hkuich Looks to have solved the issue and thank you for the quick turn around!