hbz / lobid-organisations

Transformation, web frontend, and API for lobid-organisations
http://lobid.org/organisations
Eclipse Public License 2.0
13 stars 3 forks source link

Translate morph into fix and switch to fix #450

Closed TobiasNx closed 1 year ago

TobiasNx commented 1 year ago

Related to #452 and replaces #454

This PR replaces the morph transformation with a fix transformation.

@acka47 and @fsteeg could you have a look at GUI of test.lobid.org/organisations and the json-data there. I finished the transformation i my opinion.

I also included fixes for:

361 - added gndId and links to lobid-gnd

411 - added sigils

458 - added an note[]-element

457 and #76 added replacedBy, replaces and dissolutionDate

99 added libraries.org links

269 removed test entries

309 added parentOrganisation

383 added wikidata

428 use https for isil registry links

128 add provenance Info but without links to metafacture since the early version breaks elasticSearch

TobiasNx commented 1 year ago

Thanks to @Phu2 I had the files to test with all records. It seems that : https://github.com/hbz/lobid-organisations/blob/9a9ceefa37325af673a4c1e8b8c29d4ec45ffb52/app/transformation/TransformAll.java#L86C1-L87C3

Seem not to work yet:

org.metafacture.metafix.FixProcessException: Error while executing Fix expression (at file:/home/tobias/git/lobid-organisations/conf/fix-enriched.fix, line 528): lookup("@geoLookupDbs","addLatMap", delete:"true")
        at org.metafacture.metafix.RecordTransformer.tryRun(RecordTransformer.java:249) ~[metafix-0.6.0-SNAPSHOT.jar:na]
        at org.metafacture.metafix.RecordTransformer.lambda$transform$2(RecordTransformer.java:125) ~[metafix-0.6.0-SNAPSHOT.jar:na]
        at java.lang.Iterable.forEach(Iterable.java:75) ~[na:1.8.0_362]
        at org.metafacture.metafix.RecordTransformer.transform(RecordTransformer.java:124) ~[metafix-0.6.0-SNAPSHOT.jar:na]
        at org.metafacture.metafix.RecordTransformer.lambda$null$14(RecordTransformer.java:176) ~[metafix-0.6.0-SNAPSHOT.jar:na]
        at org.metafacture.metafix.RecordTransformer.lambda$null$22(RecordTransformer.java:226) ~[metafix-0.6.0-SNAPSHOT.jar:na]
        at org.metafacture.metafix.RecordTransformer.lambda$null$1(RecordTransformer.java:125) ~[metafix-0.6.0-SNAPSHOT.jar:na]
        at org.metafacture.metafix.RecordTransformer.tryRun(RecordTransformer.java:237) ~[metafix-0.6.0-SNAPSHOT.jar:na]
        at org.metafacture.metafix.RecordTransformer.lambda$transform$2(RecordTransformer.java:125) ~[metafix-0.6.0-SNAPSHOT.jar:na]
        at java.lang.Iterable.forEach(Iterable.java:75) ~[na:1.8.0_362]
Caused by: java.lang.ArrayIndexOutOfBoundsException: 1
        at transformation.GeoLookupMap.get(GeoLookupMap.java:54) ~[classes/:na]
        at transformation.GeoLookupMap.get(GeoLookupMap.java:25) ~[classes/:na]
        at org.metafacture.metafix.FixMethod$38.lambda$null$0(FixMethod.java:523) ~[metafix-0.6.0-SNAPSHOT.jar:na]
        at org.metafacture.metafix.Record.lambda$transform$2(Record.java:207) ~[metafix-0.6.0-SNAPSHOT.jar:na]
        at org.metafacture.metafix.Value.lambda$asList$3(Value.java:200) ~[metafix-0.6.0-SNAPSHOT.jar:na]
        at org.metafacture.metafix.Value.newArray(Value.java:123) ~[metafix-0.6.0-SNAPSHOT.jar:na]
        at org.metafacture.metafix.Value.asList(Value.java:196) ~[metafix-0.6.0-SNAPSHOT.jar:na]
        at org.metafacture.metafix.Value.asList(Value.java:184) ~[metafix-0.6.0-SNAPSHOT.jar:na]
        at org.metafacture.metafix.Record.transform(Record.java:202) ~[metafix-0.6.0-SNAPSHOT.jar:na]
        at org.metafacture.metafix.FixMethod$38.lambda$apply$1(FixMethod.java:522) ~[metafix-0.6.0-SNAPSHOT.jar:na]
TobiasNx commented 1 year ago

Also adding a strictness and logging as in lobid-resources would be nice.

TobiasNx commented 1 year ago

Bugs spotted:

TobiasNx commented 1 year ago
dr0i commented 1 year ago

Offline agreed to more checking functionality vs code review (which is a bit unhandy re amount of commits and changed files).

dr0i commented 1 year ago

Updates seems to work. After bash -x cron.sh the resource http://test.lobid.org/organisations/DE-4229#! with "dateCreated":"15-08-23" exists in test system (and not yet in production). So I think this PR is fine :+1: