gleanerio / nabu

Nabu: Synchronize data graph objects with a triplestore
1 stars 3 forks source link

release failing #35

Open valentinedwv opened 1 year ago

valentinedwv commented 1 year ago

Failed to create a release. And I think if one release fails, any additional releases do not get generated.

@geocodes:~/indexing$ ./glcon nabu release --cfgName aws_geocodes
Using nabu config file: /home/ubuntu/indexing/configs/aws_geocodes/nabu
nabu release called

2023/05/13 13:07:20 Error decoding triples: syntax error: bad IRI: disallowed character ' '

files in opencore on oss.geocodes-dev.earthcube.org org is empty.

GOROOT=/usr/local/opt/go/libexec #gosetup
GOPATH=/Users/valentin/go #gosetup
/usr/local/opt/go/libexec/bin/go build -ldflags -X main.VERSION=testline -o /private/var/folders/t2/t39bprkn16dg9nr4v6c18v0w0000gn/T/GoLand/___nabu_release_opencore /Users/valentin/development/dev_earthcube/gleanerio/gleaner/cmd/glcon/main.go #gosetup
/private/var/folders/t2/t39bprkn16dg9nr4v6c18v0w0000gn/T/GoLand/___nabu_release_opencore nabu release --cfg configs/opencore_nabu
Using nabu config file: configs/opencore_nabu
nabu release called
2023/05/03 07:06:22 Error decoding triples: 1:195 unexpected 3 as dot (.)

Process finished with the exit code 0
minio:
    address: oss.geocodes-dev.earthcube.org
    port: 443
    ssl: true
    accesskey:
    secretkey:
    bucket: opencore
objects:
    bucket: opencore
    domain: us-west-2
    prefix:
        - summoned/opencoredata
        - org
    prefixoff: []
sparql:
    endpoint: https://graph.geocodes-dev.earthcube.org/blazegraph/namespace/opencore/sparql
    authenticate: false
    username: ""
    password: ""
txtaipkg:
    endpoint: http://0.0.0.0:800
valentinedwv commented 1 year ago

Release file, and first two jsonld's... it does not get far. summonedopencoredata_2023-05-03-07-06-01_release.nq.txt 00015d030b0288e604a5465ecb48e2dc8c243a75..txt url: https://opencoredata.org/id/csdco/do/77949cee149acfd6bee3035e0f33d5e1cd315f192dd503fa87d56134735dcb64.jsonld 0001409ce5ce0d4e305e3c5d2d582987cea88cb3.txt url: https://opencoredata.org/id/csdco/res/RAMBLE-PWBH13-1B.jsonld

valentinedwv commented 1 year ago

based on the gleaner-wf bucket and latest, it looks like it may be happening to other releases: geocodes_demo, magic, ieda

but they all worked on the geocodes bucket on oss.geocodes.ncsa

valentinedwv commented 1 year ago

Error from one run 2023/05/03 11:53:35 Error decoding triples: syntax error: bad IRI: disallowed character ' '

valentinedwv commented 1 year ago

also happening with ssiodp (gleaner-wf/... ) lots of files in milled, but no release

valentinedwv commented 1 year ago

Unavco failed for a release, also. jsonld becomes 1 statement when milled. source: https://www.unavco.org/data/doi/10.7283/T5QF8QW6

ffff922c286db97f587dd387e14704dddf9d39ba.jsonld.txt ffff922c286db97f587dd387e14704dddf9d39ba.rdf.txt

from the runlogs.

{"file":"/home/runner/work/nabu/nabu/internal/services/releases/bulkLoader.go:20","func":"github.com/gleanerio/nabu/internal/services/releases.BulkRelease","level":"info","msg":"Release:BulkAssembly","time":"2023-08-25T18:48:31Z"}
{"file":"/home/runner/work/nabu/nabu/internal/objects/pipecopy.go:28","func":"github.com/gleanerio/nabu/internal/objects.PipeCopy","level":"info","msg":"PipeCopy with name: unavco_release.nq   bucket: gleaner-wf  prefix: summoned/unavco","time":"2023-08-25T18:48:31Z"}
{"file":"/home/runner/work/nabu/nabu/internal/objects/pipecopy.go:64","func":"github.com/gleanerio/nabu/internal/objects.PipeCopy.func1","level":"info","msg":"\nChannel/object length: 15505\n","time":"2023-08-25T18:48:32Z"}
{"file":"/home/runner/work/nabu/nabu/internal/objects/pipecopy.go:65","func":"github.com/gleanerio/nabu/internal/objects.PipeCopy.func1","level":"info","msg":"Single file mode set: false","time":"2023-08-25T18:48:32Z"}
2023/08/25 18:48:32 Error decoding triples: 1:86 unexpected 4 as object
{"file":"/home/runner/work/nabu/nabu/internal/objects/copyobject.go:33","func":"github.com/gleanerio/nabu/internal/objects.Copy","level":"info","msg":"Successfully copied object: {gleaner-wf graphs/archive/unavco/summoned__unavco_2023-08-25-18-48-33_release.nq d41d8cd98f00b204e9800998ecf8427e 0 2023-08-25 18:48:33.86 +0000 UTC   0001-01-01 00:00:00 +0000 UTC }","time":"2023-08-25T18:48:34Z"}