ALMA-Fix: Identify errors from log

TobiasNx commented 2 years ago

[x] describedBy.sourceOrganization.id



WARN [Thread-4] (Metafix.java:421) - Error while executing Fix expression (at file:/home/sol/git/lobid-resources-alma-fix/src/main/resources/alma/fix/describedBy.fix, line 166): replace_all("describedBy.sourceOrganization.id", " ", "")
java.lang.IllegalStateException: Expected String, got Array

Beispiel: 990015048200206447. Grund: invalides Wiederholtes Feld 040.a
________________

- [x] `supplement[].*.id`

WARN [Thread-8] (Metafix.java:421) - Error while executing Fix expression (at file:/home/sol/git/lobid-resources-alma-fix/src/main/resources/alma/fix/relatedRessourcesAndLinks.fix, line 68): replace_all("supplement[]..id","^$DE-605$(.)$","http://lobid.org/resources/$1#!") java.lang.IllegalArgumentException: Can't find: 3 in: null

Beispiel: 991005935279706485, Reason: Seems to be an array. Multiple $w subfields. Need to introduce filter that only id's for ZDB or HBZ are allowed.

____________
- [ ] `publication[]`

WARN [Thread-4] (Metafix.java:421) - Error while executing Fix expression (at file:/home/sol/git/lobid-resources-alma-fix/src/main/resources/alma/fix/titleRelatedFields.fix, line 112): do list(path:"260[ 23] |264[ 23][ 1]", "var":"$i") add_field("publication[].$append.test","") paste("publication[].$last.publicationHistory", "$i.c") do list(path: "$i.c", "var":"$j") replace_all("$j", "\[|\]|ca. |c ", "") unless exists("publication[].$last.startDate") if any_match("$j",".?(\d{4}).") paste("publication[].$last.startDate", "$j") end end unless exists("publication[].$last.endDate") if any_match("$j",".*-(\d{4})$") paste("publication[].$last.endDate", "$j") end end end set_array("publication[].$last.type[]","PublicationEvent") set_array("publication[].$last.location[]") copy_field("$i.a", "publication[].$last.location[].$append") set_array("publication[].$last.publishedBy[]") copy_field("$i.b", "publication[].$last.publishedBy[].$append") end java.lang.IllegalStateException: No match available


Beispiel: 99371087584606441, 

____________

- [x] `related[]`

WARN [Thread-8] (Metafix.java:421) - Error while executing Fix expression (at file:/home/sol/git/lobid-resources-alma-fix/src/main/resources/alma/fix/relatedRessourcesAndLinks.fix, line 315): do list(path:"775??|776??", "var":"$i") unless exists("$i.M") add_field("related[].$append.test","") if all_match("$i.w", "$DE-605$(.)") copy_field("$i.w", "related[].$last.id") replace_all("related[].$last.id", "^$DE-605$(.)$", "http://lobid.org/resources/$1#!") end paste("related[].$last.note","$i.i", "$i.n") copy_field("$i.t", "related[].$last.label") copy_field("$i.x", "related[].$last.issn") copy_field("$i.z", "$i.ISBN13") isbn("$i.isbn13", to: "clean") if any_match("$i.isbn13", ".{13}") copy_field("$i.isbn13", "$i.isbn10") isbn("$i.isbn10", to:"isbn10") elsif any_match("$i.isbn13", ".{10}") copy_field("$i.isbn13", "$i.isbn10") isbn("$i.isbn13", to:"isbn13") end set_array("related[].$last.isbn[]") copy_field("$i.isbn13|isbn10", "related[].$last.isbn[].$append") end end java.lang.NullPointerException



Beispiel: 99371087571106441,

TobiasNx commented 2 years ago

[x] abstract[]

 WARN [Thread-7] (Metafix.java:421) - Error while executing Fix expression (at file:/home/sol/git/lobid-resources-alma-fix/src/main/resources/alma/fix/otherFields.fix, line 221): copy_field("$i.[ab]", "abstract[].$append")
java.util.ConcurrentModificationException

Beispiel: 99371088096206441

[x] describedBy.provider.id


 WARN [Thread-7] (Metafix.java:421) - Error while executing Fix expression (at file:/home/sol/git/lobid-resources-alma-fix/src/main/resources/alma/fix/describedBy.fix, line 184): replace_all("describedBy.provider.id", " ", "")
java.lang.IllegalStateException: Expected String, got Array

Beispiel: 99371447897606441

[x] describedBy.modifiedBy.id


 WARN [Thread-7] (Metafix.java:421) - Error while executing Fix expression (at file:/home/sol/git/lobid-resources-alma-fix/src/main/resources/alma/fix/describedBy.fix, line 213): replace_all("describedBy.modifiedBy.id", "(DE-)?(.*)", "$2")
java.lang.IllegalStateException: Expected String, got Array

Beispiel: 99371449208306441

TobiasNx commented 2 years ago

The errors in describedBy with createdy, modifiedBy, sourceOrganization and provider seems to be a problem that appears often. Since the incoming data is invalid (repeated non-repeatable subfield)

TobiasNx commented 2 years ago

[ ] urn[].$append

 INFO [Thread-4] (Metafix.java:414) - Current record: {leader=01494ngm a2200373za 4500, 006=m     o  c        , 007=[vz uzazuu, cr una---unuuu], 008=150908p20152007cau100        o   vlund d, 001=991025930244606442, 02852={a=1120119, b=Kanopy}, 035  =[{a=(OCoLC)921955872, 0=(uri) http://www.worldcat.org/oclc/921955872}, {a=(CaSfKAN)kan1120119}], 040  ={a=VDU, b=eng, c=VDU}, 24500={a=Love's Harvest}, 264 1={a=[San Francisco, California, USA] :, b=Kanopy Streaming,, c=2015.}, 300  ={a=1 online resource (streaming video file)}, 306  ={a=Duration: 100 minutes}, 336  ={a=tdi, b=tdi, 2=rdacontent}, 337  ={b=c, 2=rdamedia}, 338  ={b=[cr, cr], 2=rdacarrier}, 500  =[{a=Title from title frames.}, {a=In Process Record.}], 518  ={a=Originally produced by Rebel Films in 2007.}, 520  ={a=A series of four beautiful and intimate stories about organic farmers toiling hard for their belief in organic goodness and the life it provides. Director/writer/camera - Brian McKenzie Producer - Jen McMahon Editor - Bergan O'Brien  Award Australian Directors Guild Award - Best Direction in A documentary - 2007}, 538  ={a=Mode of access: World Wide Web.}, 546  ={a=In English}, 653  =[{a=Australian and Indigenous Studies}, {a=Environmental Sciences}], 7001 ={a=McKenzie, Brian,, e=filmmaker}, 7102 ={a=Kanopy (Firm)}, 85640={u=https://bielefeld.kanopy.com/node/120120, z=A Kanopy streaming video}, 85642={z=Cover Image, u=https://www.kanopy.com/node/120120/external-image}, 9801 ={a=ZDB-177-KAN}, MBD  ={M=49HBZ_BIE, i=991025930244606442, n=Universität Bielefeld}, MNG  ={c=System, f=OTHER, i=marc21, k=01, e=false, d=2022-06-07 18:14:53 Europe/Berlin, g=kan1120119, j=00, a=System, b=2022-06-07 18:14:55 Europe/Berlin}, POR  ={x=System, a=53344020730006442, M=49HBZ_BIE, p=61334816600006442, q=Kanopy, c=static, B=17.131 Titel, l=A Kanopy streaming video, y=2022-06-07 18:15:10 Europe/Berlin, w=2022-06-07 18:15:04 Europe/Berlin, D=https://eu04.alma.exlibrisgroup.com/view/uresolver/49HBZ_NETWORK/openurl?u.ignore_date_coverage=true&portfolio_pid=53344020730006442&Force_direct=true, v=System, z=2022-06-07 16:15:04, i=false, g=UB_BI, t=BIE-8465, S=62343002520006442, d=https://hbz-network.userservices.exlibrisgroup.com/view/uresolver/49HBZ_NETWORK/openurl?u.ignore_date_coverage=true&rft.mms_id=991025930244606442, b=Available, f=VIDEO, e=https://bielefeld.kanopy.com/node/120120, 8=53344020730006442}, @context=http://lobid.org/resources/context.jsonld, almaMmsId=991025930244606442, id=http://lobid.org/resources/991025930244606442#!, urn[]=[]}
 INFO [Thread-5] (Metafix.java:414) - Current record: {leader=01879ngm a2200385ua 4500, 006=m     o  c        , 007=[cr una---unuuu, vz uzazuu], 008=160826p20162006cau023        o   vlund d, 001=991025930286006442, 02852={a=1089373, b=Kanopy}, 035  =[{a=(OCoLC)956893319, 0=(uri) http://www.worldcat.org/oclc/956893319}, {a=(CaSfKAN)kan1089373}], 040  ={a=VDU, b=eng, c=VDU}, 24500={a=Lockie Leonard Series 1. Episode 13: It's not You, it's Me}, 264 1=[{b=Australian Children's Television Foundation,, c=2006.}, {a=[San Francisco, California, USA] :, b=Kanopy Streaming,, c=2016.}], 300  ={a=1 online resource (streaming video file) (24 minutes):, b=digital, .flv file, sound}, 306  ={a=002354}, 336  ={a=tdi, b=tdi, 2=rdacontent}, 337  ={b=[c, c], 2=rdamedia}, 338  ={b=[cr, cr], 2=rdacarrier}, 500  =[{a=Filmclip}, {a=Title from title frames.}, {a=In Process Record.}], 518  ={a=Originally produced by Australian Children's Television Foundation in 2006.}, 520  ={a=Lockie has some problems.  First, he's decided to break up with Dot.  It's just figuring out how that's tricky.  Meanwhile, Mum is acting weird.  She's been staring at the TV for days, when it's not even tuned to a channel.  Phillip's not impressed - he's missing his favourite superhero cartoon, Captain Chicken.   And to top things off, Egg and Lockie's quest to save the river is going nowhere...    Lockie has got some serious work to do.   And he's going to have to find his inner Captain Chicken to sort it all out.}, 538  ={a=Mode of access: World Wide Web.}, 546  ={a=In English}, 653  ={a=Australian Studies}, 7102 =[{a=Australian Children's Television Foundation (Firm),, 4=dst}, {a=Kanopy (Firm),, 4=dst}], 85640={u=https://bielefeld.kanopy.com/node/89374, z=A Kanopy streaming video}, 85642={z=Cover Image, u=https://www.kanopy.com/node/89374/external-image}, 9801 ={a=ZDB-177-KAN}, MBD  ={M=49HBZ_BIE, i=991025930286006442, n=Universität Bielefeld}, MNG  ={c=System, f=OTHER, i=marc21, k=01, e=false, d=2022-06-07 18:14:09 Europe/Berlin, g=kan1089373, j=00, a=System, b=2022-06-07 18:14:11 Europe/Berlin}, POR  ={x=System, a=53344020580006442, M=49HBZ_BIE, p=61334816600006442, q=Kanopy, c=static, B=17.131 Titel, l=A Kanopy streaming video, y=2022-06-07 18:14:20 Europe/Berlin, w=2022-06-07 18:14:16 Europe/Berlin, D=https://eu04.alma.exlibrisgroup.com/view/uresolver/49HBZ_NETWORK/openurl?u.ignore_date_coverage=true&portfolio_pid=53344020580006442&Force_direct=true, v=System, z=2022-06-07 16:14:16, i=false, g=UB_BI, t=BIE-8465, S=62343002520006442, d=https://hbz-network.userservices.exlibrisgroup.com/view/uresolver/49HBZ_NETWORK/openurl?u.ignore_date_coverage=true&rft.mms_id=991025930286006442, b=Available, f=VIDEO, e=https://bielefeld.kanopy.com/node/89374, 8=53344020580006442}, @context=http://lobid.org/resources/context.jsonld, almaMmsId=991025930286006442, id=http://lobid.org/resources/991025930286006442#!, urn[]=[]}
 WARN [Thread-4] (Metafix.java:421) - Error while executing Fix expression (at file:/home/sol/git/lobid-resources-alma-fix/src/main/resources/alma/fix/identifiers.fix, line 23): do list(path: "0247?", "var": "$i") if any_equal("$i.2","urn") copy_field("$i.a","urn[].$append") end end
java.lang.IllegalStateException: Key '0' already used
...
 WARN [Thread-5] (Metafix.java:421) - Error while executing Fix expression (at file:/home/sol/git/lobid-resources-alma-fix/src/main/resources/alma/fix/identifiers.fix, line 23): do list(path: "0247?", "var": "$i") if any_equal("$i.2","urn") copy_field("$i.a","urn[].$append") end end
java.lang.NullPointerException

~This is weird. Thread 4 and 5 state an error for the same record!~ Looked wrong They are different records. But it is the same error.

[ ] @title

INFO [Thread-4] (Metafix.java:414) - Current record: {leader=01494ngm a2200373za 4500, 006=m     o  c        , 007=[vz uzazuu, cr una---unuuu], 008=150908p20152007cau100        o   vlund d, 001=991025930244606442, 02852={a=1120119, b=Kanopy}, 035  =[{a=(OCoLC)921955872, 0=(uri) http://www.worldcat.org/oclc/921955872}, {a=(CaSfKAN)kan1120119}], 040  ={a=VDU, b=eng, c=VDU}, 24500={a=Love's Harvest}, 264 1={a=[San Francisco, California, USA] :, b=Kanopy Streaming,, c=2015.}, 300  ={a=1 online resource (streaming video file)}, 306  ={a=Duration: 100 minutes}, 336  ={a=tdi, b=tdi, 2=rdacontent}, 337  ={b=c, 2=rdamedia}, 338  ={b=[cr, cr], 2=rdacarrier}, 500  =[{a=Title from title frames.}, {a=In Process Record.}], 518  ={a=Originally produced by Rebel Films in 2007.}, 520  ={a=A series of four beautiful and intimate stories about organic farmers toiling hard for their belief in organic goodness and the life it provides. Director/writer/camera - Brian McKenzie Producer - Jen McMahon Editor - Bergan O'Brien  Award Australian Directors Guild Award - Best Direction in A documentary - 2007}, 538  ={a=Mode of access: World Wide Web.}, 546  ={a=In English}, 653  =[{a=Australian and Indigenous Studies}, {a=Environmental Sciences}], 7001 ={a=McKenzie, Brian,, e=filmmaker}, 7102 ={a=Kanopy (Firm)}, 85640={u=https://bielefeld.kanopy.com/node/120120, z=A Kanopy streaming video}, 85642={z=Cover Image, u=https://www.kanopy.com/node/120120/external-image}, 9801 ={a=ZDB-177-KAN}, MBD  ={M=49HBZ_BIE, i=991025930244606442, n=Universität Bielefeld}, MNG  ={c=System, f=OTHER, i=marc21, k=01, e=false, d=2022-06-07 18:14:53 Europe/Berlin, g=kan1120119, j=00, a=System, b=2022-06-07 18:14:55 Europe/Berlin}, POR  ={x=System, a=53344020730006442, M=49HBZ_BIE, p=61334816600006442, q=Kanopy, c=static, B=17.131 Titel, l=A Kanopy streaming video, y=2022-06-07 18:15:10 Europe/Berlin, w=2022-06-07 18:15:04 Europe/Berlin, D=https://eu04.alma.exlibrisgroup.com/view/uresolver/49HBZ_NETWORK/openurl?u.ignore_date_coverage=true&portfolio_pid=53344020730006442&Force_direct=true, v=System, z=2022-06-07 16:15:04, i=false, g=UB_BI, t=BIE-8465, S=62343002520006442, d=https://hbz-network.userservices.exlibrisgroup.com/view/uresolver/49HBZ_NETWORK/openurl?u.ignore_date_coverage=true&rft.mms_id=991025930244606442, b=Available, f=VIDEO, e=https://bielefeld.kanopy.com/node/120120, 8=53344020730006442}, @context=http://lobid.org/resources/context.jsonld, almaMmsId=991025930244606442, id=http://lobid.org/resources/991025930244606442#!, urn[]=[], @isbn[]=[], isbn[]=[], issn[]=[], ismn[]=[], doi[]=[], oclcNumber[]=[921955872]}
WARN [Thread-4] (Metafix.java:421) - Error while executing Fix expression (at file:/home/sol/git/lobid-resources-alma-fix/src/main/resources/alma/fix/titleRelatedFields.fix, line 9): copy_field("245??.a","@title")
java.lang.NullPointerException

[ ] alternativeTitle[].$append

 INFO [Thread-4] (Metafix.java:414) - Current record: {leader=01494ngm a2200373za 4500, 006=m     o  c        , 007=[vz uzazuu, cr una---unuuu], 008=150908p20152007cau100        o   vlund d, 001=991025930244606442, 02852={a=1120119, b=Kanopy}, 035  =[{a=(OCoLC)921955872, 0=(uri) http://www.worldcat.org/oclc/921955872}, {a=(CaSfKAN)kan1120119}], 040  ={a=VDU, b=eng, c=VDU}, 24500={a=Love's Harvest}, 264 1={a=[San Francisco, California, USA] :, b=Kanopy Streaming,, c=2015.}, 300  ={a=1 online resource (streaming video file)}, 306  ={a=Duration: 100 minutes}, 336  ={a=tdi, b=tdi, 2=rdacontent}, 337  ={b=c, 2=rdamedia}, 338  ={b=[cr, cr], 2=rdacarrier}, 500  =[{a=Title from title frames.}, {a=In Process Record.}], 518  ={a=Originally produced by Rebel Films in 2007.}, 520  ={a=A series of four beautiful and intimate stories about organic farmers toiling hard for their belief in organic goodness and the life it provides. Director/writer/camera - Brian McKenzie Producer - Jen McMahon Editor - Bergan O'Brien  Award Australian Directors Guild Award - Best Direction in A documentary - 2007}, 538  ={a=Mode of access: World Wide Web.}, 546  ={a=In English}, 653  =[{a=Australian and Indigenous Studies}, {a=Environmental Sciences}], 7001 ={a=McKenzie, Brian,, e=filmmaker}, 7102 ={a=Kanopy (Firm)}, 85640={u=https://bielefeld.kanopy.com/node/120120, z=A Kanopy streaming video}, 85642={z=Cover Image, u=https://www.kanopy.com/node/120120/external-image}, 9801 ={a=ZDB-177-KAN}, MBD  ={M=49HBZ_BIE, i=991025930244606442, n=Universität Bielefeld}, MNG  ={c=System, f=OTHER, i=marc21, k=01, e=false, d=2022-06-07 18:14:53 Europe/Berlin, g=kan1120119, j=00, a=System, b=2022-06-07 18:14:55 Europe/Berlin}, POR  ={x=System, a=53344020730006442, M=49HBZ_BIE, p=61334816600006442, q=Kanopy, c=static, B=17.131 Titel, l=A Kanopy streaming video, y=2022-06-07 18:15:10 Europe/Berlin, w=2022-06-07 18:15:04 Europe/Berlin, D=https://eu04.alma.exlibrisgroup.com/view/uresolver/49HBZ_NETWORK/openurl?u.ignore_date_coverage=true&portfolio_pid=53344020730006442&Force_direct=true, v=System, z=2022-06-07 16:15:04, i=false, g=UB_BI, t=BIE-8465, S=62343002520006442, d=https://hbz-network.userservices.exlibrisgroup.com/view/uresolver/49HBZ_NETWORK/openurl?u.ignore_date_coverage=true&rft.mms_id=991025930244606442, b=Available, f=VIDEO, e=https://bielefeld.kanopy.com/node/120120, 8=53344020730006442}, @context=http://lobid.org/resources/context.jsonld, almaMmsId=991025930244606442, id=http://lobid.org/resources/991025930244606442#!, oclcNumber[]=[921955872], title=, alternativeTitle[]=[]}
 WARN [Thread-4] (Metafix.java:421) - Error while executing Fix expression (at file:/home/sol/git/lobid-resources-alma-fix/src/main/resources/alma/fix/titleRelatedFields.fix, line 81): copy_field("246?[ 345678].a","alternativeTitle[].$append")
java.lang.IllegalStateException: No match available
    at java.util.regex.Matcher.start(Matcher.java:343)

[ ] @titleOfSubSeries_p.$append

 INFO [Thread-6] (Metafix.java:414) - Current record: {leader=01655ngm a2200409ua 4500, 006=m     o  c        , 007=[cr una---unuuu, vz uzazuu], 008=160826p20162011cau026        o   vlund d, 001=991025930295506442, 02852={a=1089337, b=Kanopy}, 035  =[{a=(OCoLC)1035144056, 0=(uri) http://www.worldcat.org/oclc/1035144056}, {a=(CaSfKAN)kan1089337}], 040  ={a=VDU, b=eng, c=VDU}, 24504={a=The March of Time Corporate Films and Public Service Announcements. My Son's Dad}, 264 1=[{b=Australian Children's Television Foundation,, c=2011.}, {a=[San Francisco, California, USA] :, b=Kanopy Streaming,, c=2016.}], 300  ={a=1 online resource (streaming video file) (26 minutes):, b=digital, .flv file, sound}, 306  ={a=002600}, 336  ={a=tdi, b=tdi, 2=rdacontent}, 337  ={b=[c, c], 2=rdamedia}, 338  ={b=[cr, cr], 2=rdacarrier}, 500  =[{a=Filmclip}, {a=Title from title frames.}, {a=In Process Record.}], 518  ={a=Originally produced by Australian Children's Television Foundation in 2011.}, 520  ={a=Busy Christian white-collar father takes time to spend with son at YMCA. Tour of Y brings back memories of childhood and importance of YMCA in his life.  Son arranges ostracizing lessons at camp for misbehaving bully}, 538  ={a=Mode of access: World Wide Web.}, 546  ={a=In English}, 653  =[{a=Film Studies}, {a=History - Modern}, {a=Jewish Studies}], 7102 =[{a=Australian Children's Television Foundation (Firm),, 4=dst}, {a=Kanopy (Firm),, 4=dst}], 85640={u=https://bielefeld.kanopy.com/node/89338, z=A Kanopy streaming video}, 85642={z=Cover Image, u=https://www.kanopy.com/node/89338/external-image}, 9801 ={a=ZDB-177-KAN}, MBD  ={M=49HBZ_BIE, i=991025930295506442, n=Universität Bielefeld}, MNG  ={c=System, f=OTHER, i=marc21, k=01, e=false, d=2022-06-07 18:14:09 Europe/Berlin, g=kan1089337, j=00, a=System, b=2022-06-07 18:14:11 Europe/Berlin}, POR  ={x=System, a=53344020430006442, M=49HBZ_BIE, p=61334816600006442, q=Kanopy, c=static, B=17.131 Titel, l=A Kanopy streaming video, y=2022-06-07 18:14:20 Europe/Berlin, w=2022-06-07 18:14:16 Europe/Berlin, D=https://eu04.alma.exlibrisgroup.com/view/uresolver/49HBZ_NETWORK/openurl?u.ignore_date_coverage=true&portfolio_pid=53344020430006442&Force_direct=true, v=System, z=2022-06-07 16:14:16, i=false, g=UB_BI, t=BIE-8465, S=62343002520006442, d=https://hbz-network.userservices.exlibrisgroup.com/view/uresolver/49HBZ_NETWORK/openurl?u.ignore_date_coverage=true&rft.mms_id=991025930295506442, b=Available, f=VIDEO, e=https://bielefeld.kanopy.com/node/89338, 8=53344020430006442}, @context=http://lobid.org/resources/context.jsonld, almaMmsId=991025930295506442, id=http://lobid.org/resources/991025930295506442#!, urn[]=[], @isbn[]=[], isbn[]=[], issn[]=[], ismn[]=[], doi[]=[], oclcNumber[]=[1035144056], @titleOfSubSeries_n=, @titleOfSubSeries_p=[]}
 WARN [Thread-6] (Metafix.java:421) - Error while executing Fix expression (at file:/home/sol/git/lobid-resources-alma-fix/src/main/resources/alma/fix/titleRelatedFields.fix, line 39): do list(path:"245??.p", "var":"$i") copy_field("$i","@titleOfSubSeries_p.$append") end
java.lang.NullPointerException

____-

[ ] contribution[].$last.role.id

 INFO [Thread-4] (Metafix.java:414) - Current record: {leader=00862nam#a2200313#cb4500, 005=20210409162606.0, 007=tu, 008=180115|1975####xx############|||#|#####c, 003=DE-605, 001=990222755220206441, 0167 ={a=1075143759, 2=OCoLC}, 035  =[{a=(DE-605)TT003819122}, {a=(OCoLC)1075143759, 0=(uri) http://www.worldcat.org/oclc/1075143759}], 040  ={a=B 23, b=ger, c=B 23, e=rakwb}, 080  =[{a=551.5}, {a=06.05}, {a=(100)}, {a=061.3}, {a=1975}, {a=Genf}], 1102 ={a=WMO, 0=[(DE-588)2034291-3, (uri) https://portal.dnb.de/opac.htm?method=simpleSearch&cqlMode=true&query=idn=004901681, (uri) http://viaf.org/viaf/sourceID/DNB|004901681], 4=aut}, 24510={a=SEVENTH WORLD METEOROLOGICAL CONGRESS. GENEVA, 28 APRIL - 23 MAY 1975. ABRIDGED REPORT WITH RESOLUTIONS.}, 264 1={a=GENEVA, b=(SELBSTVERL.), c=1975}, 300  ={a=XXVII, 255 S.}, 4901 ={a=WMO, v=416}, 830 0={a=WMO, w=(DE-605)HT001231887, v=416, 9=O:1, 0=[(uri) https://portal.dnb.de/opac.htm?method=simpleSearch&cqlMode=true&query=idn=004901681, (uri) http://viaf.org/viaf/sourceID/DNB|004901681]}, 9640s=[{F=030, A=a|1uc||||||27}, {F=050, A=a|||||||||||||}, {F=051, A=s|||||||}], MBD  ={M=49HBZ_NETWORK, i=990222755220206441, n=HBZ Network}, MNG  ={c=System, f=ILS, i=marc21, k=01, e=false, d=2022-01-20 05:43:32 Europe/Berlin, g=022275522-HBZ01, j=60, a=import, b=2021-04-05 10:07:13 Europe/Berlin}, @context=http://lobid.org/resources/context.jsonld, almaMmsId=990222755220206441, id=http://lobid.org/resources/990222755220206441#!, hbzId=TT003819122, oclcNumber[]=[1075143759], @title=SEVENTH WORLD METEOROLOGICAL CONGRESS. GENEVA, 28 APRIL - 23 MAY 1975. ABRIDGED REPORT WITH RESOLUTIONS, title=SEVENTH WORLD METEOROLOGICAL CONGRESS. GENEVA, 28 APRIL - 23 MAY 1975. ABRIDGED REPORT WITH RESOLUTIONS, alternativeTitle[]=[], otherTitleInformation[]=[], edition[]=[], publication[]=[{test=, publicationHistory=1975, startDate=1975, type[]=[PublicationEvent], location[]=[GENEVA], publishedBy[]=[(SELBSTVERL.)]}], describedBy={id=http://lobid.org/resources/990222755220206441, label=Webseite der hbz-Ressource 990222755220206441, type[]=[BibliographicDescription], dateCreated=20210405, dateModified=20220120, inDataset={id=http://lobid.org/resources/dataset#!, label=lobid-resources – Der hbz-Verbundkatalog als Linked Open Data}, resultOf={type[]=[CreateAction], endTime=0000-00-00T00:00:00, instrument={id=https://github.com/hbz/lobid-resources, type[]=[SoftwareApplication], label=Software lobid-resources}, object={id=https://lobid.org/hbz01/990222755220206441, type[]=[DataFeedItem], label=hbz-Ressource 990222755220206441 im Exportformat MARC21 XML, inDataset={id=https://datahub.io/dataset/hbz_unioncatalog, label=hbz_unioncatalog}}}, license[]=[{id=http://creativecommons.org/publicdomain/zero/1.0, label=Creative Commons-Lizenz CC0 1.0 Universal}], sourceOrganization={id=http://lobid.org/organisations/DE-B23#!, label=lobid Organisation}, provider={id=http://lobid.org/organisations/DE-B23#!, label=lobid Organisation}}, sameAs[]=[{id=http://hub.culturegraph.org/resource/HBZ-TT003819122, label=Culturegraph Ressource}, {id=http://worldcat.org/oclc/1075143759, label=OCLC Ressource}], supplement[]=[], isPartOf[]=[{type[]=[IsPartOfRelation], hasSuperordinate[]=[{test=, id=http://lobid.org/resources/HT001231887#!, label=WMO}], numbering=416}], containedIn[]=[], primaryForm[]=[], tableOfContents[]=[], description[]=[], seeAlso[]=[], fulltextOnline[]=[], related[]=[], inCollection[]=[], similar[]=[], predecessor[]=[], succesor[]=[], @language=[], @008-lang=####c, language[]=[], note[]=[], abstract[]=[], thesisInformation[]=[], bibliographicCitation=, natureOfContent[]=[], subject[]=[], subjectAltLabel[]=[], spatial[]=[], hasItem[]=[{id=https://lobid.org/item/990222755220206441, type[]=[Item, MBD], heldBy={id=http://lobid.org/organisations/DE-605#!}}], medium[]=[{label=Print, id=http://rdaregistry.info/termList/RDAproductionMethod/1010}], @leaderPos06-07=am, @leaderTyp=Book, @leaderTyp+008=Book180115|1975####xx############|||#|#####c, type[]=[BibliographicResource, Book], responsibilityStatement[]=[], contribution[]=[]}
 WARN [Thread-4] (Metafix.java:421) - Error while executing Fix expression (at file:/home/sol/git/lobid-resources-alma-fix/src/main/resources/alma/fix/contribution.fix, line 184): do list(path:"110[012] ", "var":"$i") unless exists("$i.M") unless exists("$i.a.1") unless exists("$i.4") if any_match("type[]","ArchivedWebPage|Miscellaneous|Bibliography|Statistics|Legislation|PublishedScore|Game|Image|Map|Standard") add_field("$i.4","cre") elsif any_match("type[]","Periodical|Collection|Series|Newspaper|Journal|PublicationIssue|EditedVolume|Proceedings|Festschrift|ReferenceSource") add_field("$i.4","edt") elsif any_match("type[]","Book|MultiVolumeBook|Article|Thesis|OfficialPublication|Schoolbook|Biography|Report") add_field("$i.4","aut") else add_field("$i.4","cre") end end do list(path: "$i.4", "var":"$j") set_hash("contribution[].$append.agent") do list(path:"$i.0","var":"$k") if all_match("$k", "^\\(DE-588\\).*$") paste("contribution[].$last.agent.gndIdentifier","$k") paste("contribution[].$last.agent.id","$k") end end paste("contribution[].$last.agent.label", "$i.a", "$i.b", join_char: ". ") set_array("contribution[].$last.agent.type[]","CorporateBody") copy_field("$j","contribution[].$last.role.id") end end end end
java.util.ConcurrentModificationException

__-

[ ]ismn[].$append

 INFO [Thread-3] (Metafix.java:414) - Current record: {leader=00851nam#a2200253#c#4500, 005=20220602082800.0, 007=tu, 008=960226|1966####lh############|||#|#ger#c, 003=DE-605, 001=990026772260206441, 0167 ={a=601785568, 2=OCoLC}, 035  =[{a=(DE-605)HT007077628}, {a=(OCoLC)601785568, 0=(uri) http://www.worldcat.org/oclc/601785568}], 040  ={c=DE-605, a=61, b=ger, d=[385, DE-605], e=rakwb}, 041  ={a=ger}, 044  ={c=XA-LI}, 24500={a=Fürstentum Liechtenstein, b=eine Dokumentation, c=bearb. von Walter Kranz}, 264 1={a=Vaduz, b=Presse- u. Informationsstelle d. Fürstentums Liechtenstein, c=1966}, 300  ={a=142 S. : Ill.}, 85642=[{m=V:DE-605;X:migimw, q=application/pdf, u=http://digitale-objekte.hbz-nrw.de/storage/2013/06/26/file_6/5166565.pdf, 3=Inhaltsverzeichnis, 9=O:PKN}, {m=V:DE-605;X:migimw, q=application/pdf, u=http://digitale-objekte.hbz-nrw.de/storage/2013/06/26/file_6/5166565.pdf, 3=Inhaltsverzeichnis, 9=O:PKN, M=49HBZ_DUE}], 9640s=[{F=051, A=m|||||||}, {F=030, A=a|1uc||||||37}, {F=050, A=a|||||||||||||}], 9640i={I=marc-FMT via MAB2MARC, F=FMT, A=BK}, 7001 ={a=Kranz, Walter, 4=oth}, MBD  =[{M=49HBZ_NETWORK, i=990026772260206441, n=HBZ Network}, {M=49HBZ_DUE, i=990010532340206443, n=Universität Düsseldorf}], 983  =[{a== SchwG183, 9=LOCAL, M=49HBZ_DUE}, {a=hist534, 9=LOCAL, M=49HBZ_DUE}], 996  =[{g=Liechtenstein, 0=(DE-588)4035665-6, 9=LOCAL, M=49HBZ_DUE}, {s=Dokumentation, 0=(DE-588)4012656-0, 9=LOCAL, M=49HBZ_DUE}, {a=Fürst, 9=LOCAL, M=49HBZ_DUE}, {a=liechtensteinisch, 9=LOCAL, M=49HBZ_DUE}], MNG  ={c=System, f=HBZ, i=marc21, h=20220602082800.0, k=01, e=false, d=2022-06-02 08:32:02 Europe/Berlin, g=002677226-HBZ01, j=60, a=import, b=2021-04-05 17:35:17 Europe/Berlin}, H528 ={b=X0001, c=00, h=nc/i7971, 8=22243747910006443}, HOL  ={b=2021-04-01 09:18:00, 8=22243747910006443, M=49HBZ_DUE, g=false, a=import}, ITM  ={H=22243747910006443, x=00, f=BOOK, v=00, p=11, X=System, U=1999-12-22 10:59:00, Y=2021-04-01 09:18:42, h=false, M=49HBZ_DUE, s=1, d=8, V=import, b=900964601, a=23243747890006443, c=nc/i7971, W=1999-12-22 11:59:00, u=X0001, w=X0001}, @context=http://lobid.org/resources/context.jsonld, almaMmsId=990026772260206441, id=http://lobid.org/resources/990026772260206441#!, urn[]=[], hbzId=HT007077628, @isbn[]=[], isbn[]=[], issn[]=[], ismn[]=[]}
 WARN [Thread-3] (Metafix.java:421) - Error while executing Fix expression (at file:/home/sol/git/lobid-resources-alma-fix/src/main/resources/alma/fix/identifiers.fix, line 113): do list(path:"0242?", "var":"$i") copy_field("$i.a", "ismn[].$append") end
java.lang.IllegalStateException: No match available

TobiasNx commented 2 years ago

[ ] urn[].$append

 INFO [Thread-4] (Metafix.java:414) - Current record: {leader=01494ngm a2200373za 4500, 006=m     o  c        , 007=[vz uzazuu, cr una---unuuu], 008=150908p20152007cau100        o   vlund d, 001=991025930244606442, 02852={a=1120119, b=Kanopy}, 035  =[{a=(OCoLC)921955872, 0=(uri) http://www.worldcat.org/oclc/921955872}, {a=(CaSfKAN)kan1120119}], 040  ={a=VDU, b=eng, c=VDU}, 24500={a=Love's Harvest}, 264 1={a=[San Francisco, California, USA] :, b=Kanopy Streaming,, c=2015.}, 300  ={a=1 online resource (streaming video file)}, 306  ={a=Duration: 100 minutes}, 336  ={a=tdi, b=tdi, 2=rdacontent}, 337  ={b=c, 2=rdamedia}, 338  ={b=[cr, cr], 2=rdacarrier}, 500  =[{a=Title from title frames.}, {a=In Process Record.}], 518  ={a=Originally produced by Rebel Films in 2007.}, 520  ={a=A series of four beautiful and intimate stories about organic farmers toiling hard for their belief in organic goodness and the life it provides. Director/writer/camera - Brian McKenzie Producer - Jen McMahon Editor - Bergan O'Brien  Award Australian Directors Guild Award - Best Direction in A documentary - 2007}, 538  ={a=Mode of access: World Wide Web.}, 546  ={a=In English}, 653  =[{a=Australian and Indigenous Studies}, {a=Environmental Sciences}], 7001 ={a=McKenzie, Brian,, e=filmmaker}, 7102 ={a=Kanopy (Firm)}, 85640={u=https://bielefeld.kanopy.com/node/120120, z=A Kanopy streaming video}, 85642={z=Cover Image, u=https://www.kanopy.com/node/120120/external-image}, 9801 ={a=ZDB-177-KAN}, MBD  ={M=49HBZ_BIE, i=991025930244606442, n=Universität Bielefeld}, MNG  ={c=System, f=OTHER, i=marc21, k=01, e=false, d=2022-06-07 18:14:53 Europe/Berlin, g=kan1120119, j=00, a=System, b=2022-06-07 18:14:55 Europe/Berlin}, POR  ={x=System, a=53344020730006442, M=49HBZ_BIE, p=61334816600006442, q=Kanopy, c=static, B=17.131 Titel, l=A Kanopy streaming video, y=2022-06-07 18:15:10 Europe/Berlin, w=2022-06-07 18:15:04 Europe/Berlin, D=https://eu04.alma.exlibrisgroup.com/view/uresolver/49HBZ_NETWORK/openurl?u.ignore_date_coverage=true&portfolio_pid=53344020730006442&Force_direct=true, v=System, z=2022-06-07 16:15:04, i=false, g=UB_BI, t=BIE-8465, S=62343002520006442, d=https://hbz-network.userservices.exlibrisgroup.com/view/uresolver/49HBZ_NETWORK/openurl?u.ignore_date_coverage=true&rft.mms_id=991025930244606442, b=Available, f=VIDEO, e=https://bielefeld.kanopy.com/node/120120, 8=53344020730006442}, @context=http://lobid.org/resources/context.jsonld, almaMmsId=991025930244606442, id=http://lobid.org/resources/991025930244606442#!, urn[]=[]}
 INFO [Thread-5] (Metafix.java:414) - Current record: {leader=01879ngm a2200385ua 4500, 006=m     o  c        , 007=[cr una---unuuu, vz uzazuu], 008=160826p20162006cau023        o   vlund d, 001=991025930286006442, 02852={a=1089373, b=Kanopy}, 035  =[{a=(OCoLC)956893319, 0=(uri) http://www.worldcat.org/oclc/956893319}, {a=(CaSfKAN)kan1089373}], 040  ={a=VDU, b=eng, c=VDU}, 24500={a=Lockie Leonard Series 1. Episode 13: It's not You, it's Me}, 264 1=[{b=Australian Children's Television Foundation,, c=2006.}, {a=[San Francisco, California, USA] :, b=Kanopy Streaming,, c=2016.}], 300  ={a=1 online resource (streaming video file) (24 minutes):, b=digital, .flv file, sound}, 306  ={a=002354}, 336  ={a=tdi, b=tdi, 2=rdacontent}, 337  ={b=[c, c], 2=rdamedia}, 338  ={b=[cr, cr], 2=rdacarrier}, 500  =[{a=Filmclip}, {a=Title from title frames.}, {a=In Process Record.}], 518  ={a=Originally produced by Australian Children's Television Foundation in 2006.}, 520  ={a=Lockie has some problems.  First, he's decided to break up with Dot.  It's just figuring out how that's tricky.  Meanwhile, Mum is acting weird.  She's been staring at the TV for days, when it's not even tuned to a channel.  Phillip's not impressed - he's missing his favourite superhero cartoon, Captain Chicken.   And to top things off, Egg and Lockie's quest to save the river is going nowhere...    Lockie has got some serious work to do.   And he's going to have to find his inner Captain Chicken to sort it all out.}, 538  ={a=Mode of access: World Wide Web.}, 546  ={a=In English}, 653  ={a=Australian Studies}, 7102 =[{a=Australian Children's Television Foundation (Firm),, 4=dst}, {a=Kanopy (Firm),, 4=dst}], 85640={u=https://bielefeld.kanopy.com/node/89374, z=A Kanopy streaming video}, 85642={z=Cover Image, u=https://www.kanopy.com/node/89374/external-image}, 9801 ={a=ZDB-177-KAN}, MBD  ={M=49HBZ_BIE, i=991025930286006442, n=Universität Bielefeld}, MNG  ={c=System, f=OTHER, i=marc21, k=01, e=false, d=2022-06-07 18:14:09 Europe/Berlin, g=kan1089373, j=00, a=System, b=2022-06-07 18:14:11 Europe/Berlin}, POR  ={x=System, a=53344020580006442, M=49HBZ_BIE, p=61334816600006442, q=Kanopy, c=static, B=17.131 Titel, l=A Kanopy streaming video, y=2022-06-07 18:14:20 Europe/Berlin, w=2022-06-07 18:14:16 Europe/Berlin, D=https://eu04.alma.exlibrisgroup.com/view/uresolver/49HBZ_NETWORK/openurl?u.ignore_date_coverage=true&portfolio_pid=53344020580006442&Force_direct=true, v=System, z=2022-06-07 16:14:16, i=false, g=UB_BI, t=BIE-8465, S=62343002520006442, d=https://hbz-network.userservices.exlibrisgroup.com/view/uresolver/49HBZ_NETWORK/openurl?u.ignore_date_coverage=true&rft.mms_id=991025930286006442, b=Available, f=VIDEO, e=https://bielefeld.kanopy.com/node/89374, 8=53344020580006442}, @context=http://lobid.org/resources/context.jsonld, almaMmsId=991025930286006442, id=http://lobid.org/resources/991025930286006442#!, urn[]=[]}
 WARN [Thread-4] (Metafix.java:421) - Error while executing Fix expression (at file:/home/sol/git/lobid-resources-alma-fix/src/main/resources/alma/fix/identifiers.fix, line 23): do list(path: "0247?", "var": "$i") if any_equal("$i.2","urn") copy_field("$i.a","urn[].$append") end end
java.lang.IllegalStateException: Key '0' already used
...
 WARN [Thread-5] (Metafix.java:421) - Error while executing Fix expression (at file:/home/sol/git/lobid-resources-alma-fix/src/main/resources/alma/fix/identifiers.fix, line 23): do list(path: "0247?", "var": "$i") if any_equal("$i.2","urn") copy_field("$i.a","urn[].$append") end end
java.lang.NullPointerException

~This is weird. Thread 4 and 5 state an error for the same record!~ Looked wrong They are different records. But it is the same error.

* [ ]  `@title`

INFO [Thread-4] (Metafix.java:414) - Current record: {leader=01494ngm a2200373za 4500, 006=m     o  c        , 007=[vz uzazuu, cr una---unuuu], 008=150908p20152007cau100        o   vlund d, 001=991025930244606442, 02852={a=1120119, b=Kanopy}, 035  =[{a=(OCoLC)921955872, 0=(uri) http://www.worldcat.org/oclc/921955872}, {a=(CaSfKAN)kan1120119}], 040  ={a=VDU, b=eng, c=VDU}, 24500={a=Love's Harvest}, 264 1={a=[San Francisco, California, USA] :, b=Kanopy Streaming,, c=2015.}, 300  ={a=1 online resource (streaming video file)}, 306  ={a=Duration: 100 minutes}, 336  ={a=tdi, b=tdi, 2=rdacontent}, 337  ={b=c, 2=rdamedia}, 338  ={b=[cr, cr], 2=rdacarrier}, 500  =[{a=Title from title frames.}, {a=In Process Record.}], 518  ={a=Originally produced by Rebel Films in 2007.}, 520  ={a=A series of four beautiful and intimate stories about organic farmers toiling hard for their belief in organic goodness and the life it provides. Director/writer/camera - Brian McKenzie Producer - Jen McMahon Editor - Bergan O'Brien  Award Australian Directors Guild Award - Best Direction in A documentary - 2007}, 538  ={a=Mode of access: World Wide Web.}, 546  ={a=In English}, 653  =[{a=Australian and Indigenous Studies}, {a=Environmental Sciences}], 7001 ={a=McKenzie, Brian,, e=filmmaker}, 7102 ={a=Kanopy (Firm)}, 85640={u=https://bielefeld.kanopy.com/node/120120, z=A Kanopy streaming video}, 85642={z=Cover Image, u=https://www.kanopy.com/node/120120/external-image}, 9801 ={a=ZDB-177-KAN}, MBD  ={M=49HBZ_BIE, i=991025930244606442, n=Universität Bielefeld}, MNG  ={c=System, f=OTHER, i=marc21, k=01, e=false, d=2022-06-07 18:14:53 Europe/Berlin, g=kan1120119, j=00, a=System, b=2022-06-07 18:14:55 Europe/Berlin}, POR  ={x=System, a=53344020730006442, M=49HBZ_BIE, p=61334816600006442, q=Kanopy, c=static, B=17.131 Titel, l=A Kanopy streaming video, y=2022-06-07 18:15:10 Europe/Berlin, w=2022-06-07 18:15:04 Europe/Berlin, D=https://eu04.alma.exlibrisgroup.com/view/uresolver/49HBZ_NETWORK/openurl?u.ignore_date_coverage=true&portfolio_pid=53344020730006442&Force_direct=true, v=System, z=2022-06-07 16:15:04, i=false, g=UB_BI, t=BIE-8465, S=62343002520006442, d=https://hbz-network.userservices.exlibrisgroup.com/view/uresolver/49HBZ_NETWORK/openurl?u.ignore_date_coverage=true&rft.mms_id=991025930244606442, b=Available, f=VIDEO, e=https://bielefeld.kanopy.com/node/120120, 8=53344020730006442}, @context=http://lobid.org/resources/context.jsonld, almaMmsId=991025930244606442, id=http://lobid.org/resources/991025930244606442#!, urn[]=[], @isbn[]=[], isbn[]=[], issn[]=[], ismn[]=[], doi[]=[], oclcNumber[]=[921955872]}
WARN [Thread-4] (Metafix.java:421) - Error while executing Fix expression (at file:/home/sol/git/lobid-resources-alma-fix/src/main/resources/alma/fix/titleRelatedFields.fix, line 9): copy_field("245??.a","@title")
java.lang.NullPointerException

* [ ]  `alternativeTitle[].$append`

 INFO [Thread-4] (Metafix.java:414) - Current record: {leader=01494ngm a2200373za 4500, 006=m     o  c        , 007=[vz uzazuu, cr una---unuuu], 008=150908p20152007cau100        o   vlund d, 001=991025930244606442, 02852={a=1120119, b=Kanopy}, 035  =[{a=(OCoLC)921955872, 0=(uri) http://www.worldcat.org/oclc/921955872}, {a=(CaSfKAN)kan1120119}], 040  ={a=VDU, b=eng, c=VDU}, 24500={a=Love's Harvest}, 264 1={a=[San Francisco, California, USA] :, b=Kanopy Streaming,, c=2015.}, 300  ={a=1 online resource (streaming video file)}, 306  ={a=Duration: 100 minutes}, 336  ={a=tdi, b=tdi, 2=rdacontent}, 337  ={b=c, 2=rdamedia}, 338  ={b=[cr, cr], 2=rdacarrier}, 500  =[{a=Title from title frames.}, {a=In Process Record.}], 518  ={a=Originally produced by Rebel Films in 2007.}, 520  ={a=A series of four beautiful and intimate stories about organic farmers toiling hard for their belief in organic goodness and the life it provides. Director/writer/camera - Brian McKenzie Producer - Jen McMahon Editor - Bergan O'Brien  Award Australian Directors Guild Award - Best Direction in A documentary - 2007}, 538  ={a=Mode of access: World Wide Web.}, 546  ={a=In English}, 653  =[{a=Australian and Indigenous Studies}, {a=Environmental Sciences}], 7001 ={a=McKenzie, Brian,, e=filmmaker}, 7102 ={a=Kanopy (Firm)}, 85640={u=https://bielefeld.kanopy.com/node/120120, z=A Kanopy streaming video}, 85642={z=Cover Image, u=https://www.kanopy.com/node/120120/external-image}, 9801 ={a=ZDB-177-KAN}, MBD  ={M=49HBZ_BIE, i=991025930244606442, n=Universität Bielefeld}, MNG  ={c=System, f=OTHER, i=marc21, k=01, e=false, d=2022-06-07 18:14:53 Europe/Berlin, g=kan1120119, j=00, a=System, b=2022-06-07 18:14:55 Europe/Berlin}, POR  ={x=System, a=53344020730006442, M=49HBZ_BIE, p=61334816600006442, q=Kanopy, c=static, B=17.131 Titel, l=A Kanopy streaming video, y=2022-06-07 18:15:10 Europe/Berlin, w=2022-06-07 18:15:04 Europe/Berlin, D=https://eu04.alma.exlibrisgroup.com/view/uresolver/49HBZ_NETWORK/openurl?u.ignore_date_coverage=true&portfolio_pid=53344020730006442&Force_direct=true, v=System, z=2022-06-07 16:15:04, i=false, g=UB_BI, t=BIE-8465, S=62343002520006442, d=https://hbz-network.userservices.exlibrisgroup.com/view/uresolver/49HBZ_NETWORK/openurl?u.ignore_date_coverage=true&rft.mms_id=991025930244606442, b=Available, f=VIDEO, e=https://bielefeld.kanopy.com/node/120120, 8=53344020730006442}, @context=http://lobid.org/resources/context.jsonld, almaMmsId=991025930244606442, id=http://lobid.org/resources/991025930244606442#!, oclcNumber[]=[921955872], title=, alternativeTitle[]=[]}
 WARN [Thread-4] (Metafix.java:421) - Error while executing Fix expression (at file:/home/sol/git/lobid-resources-alma-fix/src/main/resources/alma/fix/titleRelatedFields.fix, line 81): copy_field("246?[ 345678].a","alternativeTitle[].$append")
java.lang.IllegalStateException: No match available
  at java.util.regex.Matcher.start(Matcher.java:343)

* [ ]  `@titleOfSubSeries_p.$append`

 INFO [Thread-6] (Metafix.java:414) - Current record: {leader=01655ngm a2200409ua 4500, 006=m     o  c        , 007=[cr una---unuuu, vz uzazuu], 008=160826p20162011cau026        o   vlund d, 001=991025930295506442, 02852={a=1089337, b=Kanopy}, 035  =[{a=(OCoLC)1035144056, 0=(uri) http://www.worldcat.org/oclc/1035144056}, {a=(CaSfKAN)kan1089337}], 040  ={a=VDU, b=eng, c=VDU}, 24504={a=The March of Time Corporate Films and Public Service Announcements. My Son's Dad}, 264 1=[{b=Australian Children's Television Foundation,, c=2011.}, {a=[San Francisco, California, USA] :, b=Kanopy Streaming,, c=2016.}], 300  ={a=1 online resource (streaming video file) (26 minutes):, b=digital, .flv file, sound}, 306  ={a=002600}, 336  ={a=tdi, b=tdi, 2=rdacontent}, 337  ={b=[c, c], 2=rdamedia}, 338  ={b=[cr, cr], 2=rdacarrier}, 500  =[{a=Filmclip}, {a=Title from title frames.}, {a=In Process Record.}], 518  ={a=Originally produced by Australian Children's Television Foundation in 2011.}, 520  ={a=Busy Christian white-collar father takes time to spend with son at YMCA. Tour of Y brings back memories of childhood and importance of YMCA in his life.  Son arranges ostracizing lessons at camp for misbehaving bully}, 538  ={a=Mode of access: World Wide Web.}, 546  ={a=In English}, 653  =[{a=Film Studies}, {a=History - Modern}, {a=Jewish Studies}], 7102 =[{a=Australian Children's Television Foundation (Firm),, 4=dst}, {a=Kanopy (Firm),, 4=dst}], 85640={u=https://bielefeld.kanopy.com/node/89338, z=A Kanopy streaming video}, 85642={z=Cover Image, u=https://www.kanopy.com/node/89338/external-image}, 9801 ={a=ZDB-177-KAN}, MBD  ={M=49HBZ_BIE, i=991025930295506442, n=Universität Bielefeld}, MNG  ={c=System, f=OTHER, i=marc21, k=01, e=false, d=2022-06-07 18:14:09 Europe/Berlin, g=kan1089337, j=00, a=System, b=2022-06-07 18:14:11 Europe/Berlin}, POR  ={x=System, a=53344020430006442, M=49HBZ_BIE, p=61334816600006442, q=Kanopy, c=static, B=17.131 Titel, l=A Kanopy streaming video, y=2022-06-07 18:14:20 Europe/Berlin, w=2022-06-07 18:14:16 Europe/Berlin, D=https://eu04.alma.exlibrisgroup.com/view/uresolver/49HBZ_NETWORK/openurl?u.ignore_date_coverage=true&portfolio_pid=53344020430006442&Force_direct=true, v=System, z=2022-06-07 16:14:16, i=false, g=UB_BI, t=BIE-8465, S=62343002520006442, d=https://hbz-network.userservices.exlibrisgroup.com/view/uresolver/49HBZ_NETWORK/openurl?u.ignore_date_coverage=true&rft.mms_id=991025930295506442, b=Available, f=VIDEO, e=https://bielefeld.kanopy.com/node/89338, 8=53344020430006442}, @context=http://lobid.org/resources/context.jsonld, almaMmsId=991025930295506442, id=http://lobid.org/resources/991025930295506442#!, urn[]=[], @isbn[]=[], isbn[]=[], issn[]=[], ismn[]=[], doi[]=[], oclcNumber[]=[1035144056], @titleOfSubSeries_n=, @titleOfSubSeries_p=[]}
 WARN [Thread-6] (Metafix.java:421) - Error while executing Fix expression (at file:/home/sol/git/lobid-resources-alma-fix/src/main/resources/alma/fix/titleRelatedFields.fix, line 39): do list(path:"245??.p", "var":"$i") copy_field("$i","@titleOfSubSeries_p.$append") end
java.lang.NullPointerException

____-

* [ ]  `contribution[].$last.role.id`

 INFO [Thread-4] (Metafix.java:414) - Current record: {leader=00862nam#a2200313#cb4500, 005=20210409162606.0, 007=tu, 008=180115|1975####xx############|||#|#####c, 003=DE-605, 001=990222755220206441, 0167 ={a=1075143759, 2=OCoLC}, 035  =[{a=(DE-605)TT003819122}, {a=(OCoLC)1075143759, 0=(uri) http://www.worldcat.org/oclc/1075143759}], 040  ={a=B 23, b=ger, c=B 23, e=rakwb}, 080  =[{a=551.5}, {a=06.05}, {a=(100)}, {a=061.3}, {a=1975}, {a=Genf}], 1102 ={a=WMO, 0=[(DE-588)2034291-3, (uri) https://portal.dnb.de/opac.htm?method=simpleSearch&cqlMode=true&query=idn=004901681, (uri) http://viaf.org/viaf/sourceID/DNB|004901681], 4=aut}, 24510={a=SEVENTH WORLD METEOROLOGICAL CONGRESS. GENEVA, 28 APRIL - 23 MAY 1975. ABRIDGED REPORT WITH RESOLUTIONS.}, 264 1={a=GENEVA, b=(SELBSTVERL.), c=1975}, 300  ={a=XXVII, 255 S.}, 4901 ={a=WMO, v=416}, 830 0={a=WMO, w=(DE-605)HT001231887, v=416, 9=O:1, 0=[(uri) https://portal.dnb.de/opac.htm?method=simpleSearch&cqlMode=true&query=idn=004901681, (uri) http://viaf.org/viaf/sourceID/DNB|004901681]}, 9640s=[{F=030, A=a|1uc||||||27}, {F=050, A=a|||||||||||||}, {F=051, A=s|||||||}], MBD  ={M=49HBZ_NETWORK, i=990222755220206441, n=HBZ Network}, MNG  ={c=System, f=ILS, i=marc21, k=01, e=false, d=2022-01-20 05:43:32 Europe/Berlin, g=022275522-HBZ01, j=60, a=import, b=2021-04-05 10:07:13 Europe/Berlin}, @context=http://lobid.org/resources/context.jsonld, almaMmsId=990222755220206441, id=http://lobid.org/resources/990222755220206441#!, hbzId=TT003819122, oclcNumber[]=[1075143759], @title=SEVENTH WORLD METEOROLOGICAL CONGRESS. GENEVA, 28 APRIL - 23 MAY 1975. ABRIDGED REPORT WITH RESOLUTIONS, title=SEVENTH WORLD METEOROLOGICAL CONGRESS. GENEVA, 28 APRIL - 23 MAY 1975. ABRIDGED REPORT WITH RESOLUTIONS, alternativeTitle[]=[], otherTitleInformation[]=[], edition[]=[], publication[]=[{test=, publicationHistory=1975, startDate=1975, type[]=[PublicationEvent], location[]=[GENEVA], publishedBy[]=[(SELBSTVERL.)]}], describedBy={id=http://lobid.org/resources/990222755220206441, label=Webseite der hbz-Ressource 990222755220206441, type[]=[BibliographicDescription], dateCreated=20210405, dateModified=20220120, inDataset={id=http://lobid.org/resources/dataset#!, label=lobid-resources – Der hbz-Verbundkatalog als Linked Open Data}, resultOf={type[]=[CreateAction], endTime=0000-00-00T00:00:00, instrument={id=https://github.com/hbz/lobid-resources, type[]=[SoftwareApplication], label=Software lobid-resources}, object={id=https://lobid.org/hbz01/990222755220206441, type[]=[DataFeedItem], label=hbz-Ressource 990222755220206441 im Exportformat MARC21 XML, inDataset={id=https://datahub.io/dataset/hbz_unioncatalog, label=hbz_unioncatalog}}}, license[]=[{id=http://creativecommons.org/publicdomain/zero/1.0, label=Creative Commons-Lizenz CC0 1.0 Universal}], sourceOrganization={id=http://lobid.org/organisations/DE-B23#!, label=lobid Organisation}, provider={id=http://lobid.org/organisations/DE-B23#!, label=lobid Organisation}}, sameAs[]=[{id=http://hub.culturegraph.org/resource/HBZ-TT003819122, label=Culturegraph Ressource}, {id=http://worldcat.org/oclc/1075143759, label=OCLC Ressource}], supplement[]=[], isPartOf[]=[{type[]=[IsPartOfRelation], hasSuperordinate[]=[{test=, id=http://lobid.org/resources/HT001231887#!, label=WMO}], numbering=416}], containedIn[]=[], primaryForm[]=[], tableOfContents[]=[], description[]=[], seeAlso[]=[], fulltextOnline[]=[], related[]=[], inCollection[]=[], similar[]=[], predecessor[]=[], succesor[]=[], @language=[], @008-lang=####c, language[]=[], note[]=[], abstract[]=[], thesisInformation[]=[], bibliographicCitation=, natureOfContent[]=[], subject[]=[], subjectAltLabel[]=[], spatial[]=[], hasItem[]=[{id=https://lobid.org/item/990222755220206441, type[]=[Item, MBD], heldBy={id=http://lobid.org/organisations/DE-605#!}}], medium[]=[{label=Print, id=http://rdaregistry.info/termList/RDAproductionMethod/1010}], @leaderPos06-07=am, @leaderTyp=Book, @leaderTyp+008=Book180115|1975####xx############|||#|#####c, type[]=[BibliographicResource, Book], responsibilityStatement[]=[], contribution[]=[]}
 WARN [Thread-4] (Metafix.java:421) - Error while executing Fix expression (at file:/home/sol/git/lobid-resources-alma-fix/src/main/resources/alma/fix/contribution.fix, line 184): do list(path:"110[012] ", "var":"$i") unless exists("$i.M") unless exists("$i.a.1") unless exists("$i.4") if any_match("type[]","ArchivedWebPage|Miscellaneous|Bibliography|Statistics|Legislation|PublishedScore|Game|Image|Map|Standard") add_field("$i.4","cre") elsif any_match("type[]","Periodical|Collection|Series|Newspaper|Journal|PublicationIssue|EditedVolume|Proceedings|Festschrift|ReferenceSource") add_field("$i.4","edt") elsif any_match("type[]","Book|MultiVolumeBook|Article|Thesis|OfficialPublication|Schoolbook|Biography|Report") add_field("$i.4","aut") else add_field("$i.4","cre") end end do list(path: "$i.4", "var":"$j") set_hash("contribution[].$append.agent") do list(path:"$i.0","var":"$k") if all_match("$k", "^\\(DE-588\\).*$") paste("contribution[].$last.agent.gndIdentifier","$k") paste("contribution[].$last.agent.id","$k") end end paste("contribution[].$last.agent.label", "$i.a", "$i.b", join_char: ". ") set_array("contribution[].$last.agent.type[]","CorporateBody") copy_field("$j","contribution[].$last.role.id") end end end end
java.util.ConcurrentModificationException

__-

* [ ]` ismn[].$append`

 INFO [Thread-3] (Metafix.java:414) - Current record: {leader=00851nam#a2200253#c#4500, 005=20220602082800.0, 007=tu, 008=960226|1966####lh############|||#|#ger#c, 003=DE-605, 001=990026772260206441, 0167 ={a=601785568, 2=OCoLC}, 035  =[{a=(DE-605)HT007077628}, {a=(OCoLC)601785568, 0=(uri) http://www.worldcat.org/oclc/601785568}], 040  ={c=DE-605, a=61, b=ger, d=[385, DE-605], e=rakwb}, 041  ={a=ger}, 044  ={c=XA-LI}, 24500={a=Fürstentum Liechtenstein, b=eine Dokumentation, c=bearb. von Walter Kranz}, 264 1={a=Vaduz, b=Presse- u. Informationsstelle d. Fürstentums Liechtenstein, c=1966}, 300  ={a=142 S. : Ill.}, 85642=[{m=V:DE-605;X:migimw, q=application/pdf, u=http://digitale-objekte.hbz-nrw.de/storage/2013/06/26/file_6/5166565.pdf, 3=Inhaltsverzeichnis, 9=O:PKN}, {m=V:DE-605;X:migimw, q=application/pdf, u=http://digitale-objekte.hbz-nrw.de/storage/2013/06/26/file_6/5166565.pdf, 3=Inhaltsverzeichnis, 9=O:PKN, M=49HBZ_DUE}], 9640s=[{F=051, A=m|||||||}, {F=030, A=a|1uc||||||37}, {F=050, A=a|||||||||||||}], 9640i={I=marc-FMT via MAB2MARC, F=FMT, A=BK}, 7001 ={a=Kranz, Walter, 4=oth}, MBD  =[{M=49HBZ_NETWORK, i=990026772260206441, n=HBZ Network}, {M=49HBZ_DUE, i=990010532340206443, n=Universität Düsseldorf}], 983  =[{a== SchwG183, 9=LOCAL, M=49HBZ_DUE}, {a=hist534, 9=LOCAL, M=49HBZ_DUE}], 996  =[{g=Liechtenstein, 0=(DE-588)4035665-6, 9=LOCAL, M=49HBZ_DUE}, {s=Dokumentation, 0=(DE-588)4012656-0, 9=LOCAL, M=49HBZ_DUE}, {a=Fürst, 9=LOCAL, M=49HBZ_DUE}, {a=liechtensteinisch, 9=LOCAL, M=49HBZ_DUE}], MNG  ={c=System, f=HBZ, i=marc21, h=20220602082800.0, k=01, e=false, d=2022-06-02 08:32:02 Europe/Berlin, g=002677226-HBZ01, j=60, a=import, b=2021-04-05 17:35:17 Europe/Berlin}, H528 ={b=X0001, c=00, h=nc/i7971, 8=22243747910006443}, HOL  ={b=2021-04-01 09:18:00, 8=22243747910006443, M=49HBZ_DUE, g=false, a=import}, ITM  ={H=22243747910006443, x=00, f=BOOK, v=00, p=11, X=System, U=1999-12-22 10:59:00, Y=2021-04-01 09:18:42, h=false, M=49HBZ_DUE, s=1, d=8, V=import, b=900964601, a=23243747890006443, c=nc/i7971, W=1999-12-22 11:59:00, u=X0001, w=X0001}, @context=http://lobid.org/resources/context.jsonld, almaMmsId=990026772260206441, id=http://lobid.org/resources/990026772260206441#!, urn[]=[], hbzId=HT007077628, @isbn[]=[], isbn[]=[], issn[]=[], ismn[]=[]}
 WARN [Thread-3] (Metafix.java:421) - Error while executing Fix expression (at file:/home/sol/git/lobid-resources-alma-fix/src/main/resources/alma/fix/identifiers.fix, line 113): do list(path:"0242?", "var":"$i") copy_field("$i.a", "ismn[].$append") end
java.lang.IllegalStateException: No match available

I tested these errors with the stated records but I cannot reproduce any of the errors: #1423 and #1422

blackwinter commented 2 years ago

Some of these errors seem to indicate race conditions. Could it be that Metafix just isn't thread-safe?

If you then run the transformation with fewer records or a different number of threads, you might not trigger these race conditions (which are inherently difficult to reproduce).

dr0i commented 2 years ago

Yep , at least ConcurrentModificationException seems to proof that. This also might explain the (in this t"hreaded" context) experienced exponential run time (which also comes with, and may explain, the lot of allocated MEM (going up to >14GB )).

blackwinter commented 2 years ago

Can you open an issue for potential race conditions and provide stack traces for the ones you encountered? IllegalStateException: Key '0' already used might be another (semi-)obvious candidate.

dr0i commented 2 years ago

Thx @blackwinter , opened https://github.com/metafacture/metafacture-fix/issues/255.

dr0i commented 2 years ago

Closing this one.

hbz / lobid-resources

ALMA-Fix: Identify errors from log #1409