metafacture / metafacture-fix

Work in progress towards an implementation of the Fix language for Metafacture
Apache License 2.0
6 stars 2 forks source link

Behavior of bind list is not a correct if array provided contains only one item #331

Open aquast opened 1 year ago

aquast commented 1 year ago

If I use the do list command with an array of one item the result differs from an array of n items.

It looks like the command can not be executed correctly with an array of one.

Please see attached files

Attachments.zip

TobiasNx commented 1 year ago

I had a quick look at the transformation. Try to use the list-bind like this:

input:

  "test" : [ {
    "key" : [ "value", "more_value", "even_more_value" ],
    "foo" : "bar1",
    "test": "test1"
  }, {
    "key" : [ "less_value", "little_value", "no_value" ],
    "foo" : "bar2",
    "test": "test2"
  } ]
do list(path:"test[]","var":"$i")
  move_field("$i.foo","$i.dc:foo")
end

output:

  "test" : [ {
    "key" : [ "value", "more_value", "even_more_value" ],
    "dc:foo" : "bar1",
    "test": "test1"
  }, {
    "key" : [ "less_value", "little_value", "no_value" ],
    "dc:foo" : "bar2",
    "test": "test2"
  } ]

You do not have root access if you do not use the "var" option. (Hint: unfortunatly the only option in fix that needs " is var)

TobiasNx commented 1 year ago

@aquast I tried to fix your workflow:

https://metafacture.org/playground/?flux=%22https%3A//raw.githubusercontent.com/TobiasNx/metafacture_workflows/master/EDM2ESJsonBulk/Attachments/example1.xml%22%0A%7C+open-http%0A%7C+decode-xml%0A%7C+handle-generic-xml%28emitnamespace%3D%22true%22%29%0A%7C+fix%28transformationFile%29%0A%7C+encode-yaml%0A%7C+print%0A%3B&transformation=copy_field%28%22header.identifier.value%22%2C+%22id%22%29%0Aremove_field%28%22header%22%29%0A%0Amove_field%28%22metadata.ns1%3ARDF.ns2%3AProvidedCHO%22%2C+%22rdf%3ARDF.edm%3AProvidedCHO%5B%5D%22%29%0A%0Ado+list%28path%3A+%22rdf%3ARDF.edm%3AProvidedCHO%5B%5D%22%2C%22var%22%3A%22%24i%22%29%0A++set_array%28%22%24i.dc%3Atitle%5B%5D%22%29%0A++set_array%28%22%24i.dc%3Atitle%5B%5D.%24append%22%29%0A++move_field%28%22%24i.ns3%3Atitle.value%22%2C+%22%24i.dc%3Atitle%5B%5D.%24append%22%29%0A++remove_field%28%22%24i.ns3%3Atitle%22%29%0A%0A++set_array%28%22%24i.dcterms%3Aalternative%5B%5D%22%29%0A++move_field%28%22%24i.dcterms%3Aalternative.value%22%2C+%22%24i.dcterms%3Aalternative%5B%5D.%24append%22%29%0A++remove_field%28%22%24i.dcterms%3Aalternative%22%29%0A++set_array%28%22%24i.dc%3Acreator%5B%5D%22%29%0A++move_field%28%22%24i.ns3%3Acreator.value%22%2C+%22%24i.dc%3Acreator%5B%5D.%24append%22%29%0A++remove_field%28%22%24i.ns3%3Acreator%22%29%0A++set_array%28%22%24i.dc%3Acontributor%5B%5D%22%29%0A++move_field%28%22%24i.ns3%3Acontributor.value%22%2C+%22%24i.dc%3Acontributor%5B%5D.%24append%22%29%0A++remove_field%28%22%24i.ns3%3Acontributor%22%29%0A++set_array%28%22%24i.dc%3Adescription%5B%5D%22%29%0A++move_field%28%22%24i.ns3%3Adescription.value%22%2C+%22%24i.dc%3Adescription%5B%5D.%24append%22%29%0A++remove_field%28%22%24i.ns3%3Adescription%22%29%0A++set_array%28%22%24i.dc%3Asubject%5B%5D%22%29%0A++move_field%28%22%24i.ns3%3Asubject.value%22%2C+%22%24i.dc%3Asubject%5B%5D.%24append%22%29%0A++remove_field%28%22%24i.ns3%3Asubject%22%29%0A++set_array%28%22%24i.dc%3Acoverage%5B%5D%22%29%0A++move_field%28%22%24i.ns3%3Acoverage.value%22%2C+%22%24i.dc%3Acoverage%5B%5D.%24append%22%29%0A++remove_field%28%22%24i.ns3%3Acoverage%22%29%0A++set_array%28%22%24i.dc%3Apublisher%5B%5D%22%29%0A++move_field%28%22%24i.ns3%3Apublisher.value%22%2C+%22%24i.dc%3Apublisher%5B%5D.%24append%22%29%0A++remove_field%28%22%24i.ns3%3Apublisher%22%29%0A++move_field%28%22%24i.ns3%3ArightsHolder.value%22+%2C%22%24i.dc%3ArightsHolder%22%29++%0A++remove_field%28%22%24i.ns3%3ArightsHolder%22%29%0A++set_array%28%22%24i.dcterms%3Aextent%5B%5D%22%29%0A++move_field%28%22%24i.dcterms%3Aextent.value%22%2C+%22%24i.dcterms%3Aextent%5B%5D.%24append%22%29%0A++remove_field%28%22%24i.ns4%3Aextent%22%29%0A++set_array%28%22%24i.dc%3Alanguage%5B%5D%22%29%0A++move_field%28%22%24i.ns3%3Alanguage.value%22%2C+%22%24i.dc%3Alanguage%5B%5D.%24append%22%29%0A++remove_field%28%22%24i.ns3%3Alanguage%22%29%0A++set_array%28%22%24i.dcterms%3Aprovenance%5B%5D%22%29%0A++move_field%28%22%24i.ns4%3Aprovenance.value%22%2C+%22%24i.dcterms%3Aprovenance%5B%5D.%24append%22%29%0A++remove_field%28%22%24i.ns4%3Aprovenance%22%29%0A++set_array%28%22%24i.dcterms%3Amedium%5B%5D%22%29%0A++move_field%28%22%24i.ns4%3Amedium.value%22%2C+%22%24i.dcterms%3Amedium%5B%5D.%24append%22%29%0A++remove_field%28%22%24i.dcterms%3Amedium%22%29%0A++set_array%28%22%24i.dcterms%3AtableOfContent%5B%5D%22%29%0A++move_field%28%22%24i.ns4%3AtableOfContent.value%22%2C+%22%24i.dcterms%3AtableOfContent%5B%5D.%24append%22%29%0A++remove_field%28%22%24i.ns4%3AtableOfContent%22%29%0A++%0A%0A++move_field%28%22%24i.ns3%3Aidentifier.value%22+%2C%22%24i.dc%3Aidentifier%5B%5D%22%29%0A++remove_field%28%22%24i.ns3%3Aidentifier%22%29%0A++set_array%28%22%24i.dc%3Aformat%5B%5D%22%29%0A++move_field%28%22%24i.ns3%3Aformat.value%22%2C+%22%24i.dc%3Aformat%5B%5D.%24append%22%29%0A++remove_field%28%22%24i.ns3%3Aformat%22%29%0A++set_array%28%22%24i.dcterms%3AhasFormat%5B%5D%22%29%0A++move_field%28%22%24i.dcterms%3AhasFormat.value%22%2C+%22%24i.dcterms%3AhasFormat%5B%5D.%24append%22%29%0A++remove_field%28%22%24i.ns4%3AhasFormatvalue%22%29%0A++set_array%28%22%24i.dc%3Arights%5B%5D%22%29%0A++move_field%28%22%24i.ns3%3Arights.value%22%2C+%22%24i.dc%3Arights%5B%5D.%24append%22%29%0A++remove_field%28%22%24i.ns3%3Arights%22%29%0A++set_array%28%22%24i.dcterms%3AisPartOf%5B%5D%22%29%0A++move_field%28%22%24i.ns4%3AisPartOf.value%22%2C+%22%24i.dcterms%3AisPartOf%5B%5D.%24append%22%29%0A++remove_field%28%22%24i.ns4%3AisPartOf%22%29%0A++move_field%28%22%24i.ns2%3Atype.value%22+%2C%22%24i.edm%3Atype%22%29++%0A++remove_field%28%22%24i.ns2%3Atype%22%29%0A++move_field%28%22%24i.ns3%3Atype.value%22+%2C%22%24i.dc%3Atype%22%29++%0A++remove_field%28%22%24i.ns3%3Atype%22%29%0A++set_array%28%22%24i.dcterms%3Aspatial%5B%5D%22%29%0A++move_field%28%22%24i.ns4%3Aspatial.value%22+%2C%22%24i.dcterms%3Aspatial%5B%5D.%24append%22%29++%0A++remove_field%28%22%24i.ns4%3Aspatial%22%29%0A++set_array%28%22%24i.dcterms%3Atemporal%5B%5D%22%29%0A++move_field%28%22%24i.ns4%3Atemporal.value%22+%2C%22%24i.dcterms%3Atemporal%5B%5D.%24append%22%29++%0A++remove_field%28%22%24i.ns4%3Atemporal%22%29%0A++set_array%28%22%24i.dc_source%5B%5D%22%29%0A++move_field%28%22%24i.ns3%3Asource.value%22%2C+%22%24i.dc_source%5B%5D.%24append%22%29%0A++remove_field%28%22%24i.ns3%3Asource%22%29%0A%0A++set_array%28%22%24i.dc%3Adate%5B%5D%22%29%0A++move_field%28%22%24i.ns3%3Adate.value%22%2C+%22%24i.dc%3Adate%5B%5D.%24append%22%29%0A++remove_field%28%22%24i.ns3%3Adate%22%29%0A++set_array%28%22%24i.dcterms%3Acreated%5B%5D%22%29%0A++move_field%28%22%24i.ns4%3Acreated.value%22%2C+%22%24i.dcterms%3Acreated%5B%5D.%24append%22%29%0A++remove_field%28%22%24i.ns4%3Acreated%22%29%0A++set_array%28%22%24i.dcterms%3Aissued%5B%5D%22%29%0A++move_field%28%22%24i.ns4%3Aissued.value%22%2C+%22%24i.dcterms%3Aissued%5B%5D.%24append%22%29%0A++remove_field%28%22%24i.ns4%3Aissued%22%29%0A++%0A++remove_field%28%22%24i.ns4%3AhasPart%22%29%0A++remove_field%28%22%24i.ns2%3AhasType%22%29%0A++%0Aend%0A%0Amove_field%28%22metadata.ns1%3ARDF.ns10%3AAggregation%22%2C+%22rdf%3ARDF.ore%3AAggregation%5B%5D%22%29%0A%0Ado+list%28path%3A+%22rdf.ore%3AAggregation%5B%5D%22%2C+%22var%22%3A%22%24i%22%29%0A++set_array%28%22%24i.edm%3Arights%5B%5D%22%29%0A++move_field%28%22%24i.rights.resource%22%2C+%22%24i.edm%3Arights%5B%5D.%24append%22%29%0A++remove_field%28%22%24i.rights%22%29%0A++move_field%28%22%24i.dataProvider.1.value%22%2C+%22%24i.edm%3AdataProvider%22%29++%0A++remove_field%28%22%24i.dataProvider%22%29%0Aend%0A%0Avacuum%28%29%0A

If you want to create a new array, it is best to use a set_array and move the field with an $append into the new array.

aquast commented 1 year ago

Hi @TobiasNx ,

your proposal with "var" solved the issue for me. So maybe it's an issue of documentation rather than coding

Thank you : (thumbs up)

TobiasNx commented 1 year ago

Hi @TobiasNx ,

your proposal with "var" solved the issue for me. So maybe it's an issue of documentation rather than coding

Thank you : (thumbs up)

@aquast also have a look at my playground link. if you are working with newly created arrays, you should use set_array("newArray[]") and copy or move values like this into it: copy_field("oldField.value","newArray[].$append")

TobiasNx commented 1 year ago

@blackwinter: should we delete the documentation of the list-bind without variable since it is not very useful?

blackwinter commented 1 year ago

should we delete the documentation of the list-bind without variable since it is not very useful?

No, I don't think so. It's certainly limited as long as we don't support the . path operator, but there might still be legitimate use cases (which I don't recall off the top of my head).

aquast commented 1 year ago

Thank you for all your feedback.

If you asked me as a newbie user I'd like to have the list-bind without var with respect to my growing learners skills. Additionaly having a code example with "var":"value" as parameter would serve me better.

TobiasNx commented 1 year ago

@aquast have a look at, it seems that is should work with only one element:

https://metafacture.org/playground/?flux=inputFile%0A%7Copen-file%0A%7Cas-records%0A%7Cdecode-json%0A%7Cfix%28transformationFile%29%0A%7Cencode-yaml%0A%7Cprint%0A%3B&transformation=%23+Iterates+over+a+list+%28or+also+a+single+element+with+the+same+name%29.%0A%23+Only+the+current+element+is+accessible+in+this+case+%28as+the+root+element%29.%0A%0Ado+list%28path%3A%22author%5B%5D%22%29%0A++move_field%28%22@type%22%2C%22type%22%29%0Aend%0A%0A&data=%7B%0A++%22author%22%3A+%5B%0A++++%7B%0A++++++%22name%22%3A+%22Test+Team%22%2C%0A++++++%22@type%22%3A+%22Organization%22%0A++++%7D%2C%0A++++%7B%0A++++++%22name%22%3A+%22Maxi+Muster%22%2C%0A++++++%22@type%22%3A+%22Person%22%0A++++%7D%2C%0A++++%7B%0A++++++%22name%22%3A+%22Sally+Sample%22%2C%0A++++++%22@type%22%3A+%22Person%22%0A++++%7D%0A++%5D%0A%7D%0A%7B%0A++%22author%22%3A+%5B%0A++++%7B%0A++++++%22@type%22%3A+%22Person%22%2C%0A++++++%22name%22%3A+%22J%C3%BCrgen+Meta%22%0A++++%7D%0A++%5D%0A%7D%0A