fcrepo-exts / fcrepo-import-export

Apache License 2.0
15 stars 19 forks source link

Adds support for importing 5.x versions (mementos). Included in this… #128

Closed dbernstein closed 5 years ago

dbernstein commented 5 years ago

… commit is a fix that ensures that the versions flag is respected on import.

Resolves: https://jira.duraspace.org/browse/FCREPO-2982 https://jira.duraspace.org/browse/FCREPO-3002

There are integration tests covering this functionality.


To test manually:

  1. start up a repository
  2. create a container
    curl -X PUT -u fedoraAdmin:fedoraAdmin http://localhost:8080/rest/container
  3. Snap a version
    curl -X POST -u fedoraAdmin:fedoraAdmin http://localhost:8080/rest/container/fcr:versions
  4. Change the container
    curl -X PATCH -u fedoraAdmin:fedoraAdmin http://localhost:8080/rest/container --data-binary "INSERT { <> <http://example.org/test> \"something\" } WHERE {}" -H "Content-Type: application/sparql-update"
  5. Create a binary within the container
    curl -X PUT -u fedoraAdmin:fedoraAdmin http://localhost:8080/rest/container/binary --data-binary "hello world" -H "Content-Type: text/plain"
  6. Snap a version of the binary
    curl -X POST -u fedoraAdmin:fedoraAdmin http://localhost:8080/rest/container/binary/fcr:versions
  7. Snap a version of the binary metadata
    curl -X POST -u fedoraAdmin:fedoraAdmin http://localhost:8080/rest/container/binary/fcr:metadata/fcr:versions
  8. Change the binary
    curl -X PUT -u fedoraAdmin:fedoraAdmin http://localhost:8080/rest/container/binary --data-binary "hello world 2" -H "Content-Type: text/plain"
  9. Change the binary metadata
    curl -X PATCH -u fedoraAdmin:fedoraAdmin http://localhost:8080/rest/container/binary/fcr:metadata --data-binary "INSERT { <> <http://example.org/test> \"something\" } WHERE {}" -H "Content-Type: application/sparql-update"
  10. Perform export
    java -jar ~/code/fcrepo-import-export/target/fcrepo-import-export-0.4.0-SNAPSHOT.jar  --mode export --predicates "http://www.w3.org/ns/ldp#contains" --resource "http://localhost:8080/rest" --versions --binaries --rdfLang "text/turtle" --dir export-dir -u fedoraAdmin:fedoraAdmin
  11. Wipe and restart fresh fcrepo4 - NOTE: make sure you are using the -Dfcrepo.properties.management=relaxed flag when starting Fedora.
  12. Perform import
    java -jar ~/code/fcrepo-import-export/target/fcrepo-import-export-0.4.0-SNAPSHOT.jar  --mode import --predicates "http://www.w3.org/ns/ldp#contains" --resource "http://localhost:8080/rest" --versions --binaries --rdfLang "text/turtle" --dir export-dir -u fedoraAdmin:fedoraAdmin
  13. Verify that each of the resources match what is expected.
bseeger commented 5 years ago

Hi Danny - great instructions to test with above! When running this against the 5.1.0- RC using the jetty server, I'm seeing this stack trace from the import/export tool:

INFO 14:48:15.474 (Exporter) Exported http://localhost:8080/rest/container/binary/fcr:versions/20190710184436 to /private/tmp/fcrepo-ie/export/rest/container/binary/fcr%3Aversions/20190710184436.binary INFO 14:48:15.474 (Exporter) Already exported http://localhost:8080/rest/container/binary/fcr:metadata java.lang.RuntimeException: Export operation failed: unexpected status 400 for http://localhost:8080/rest/container/binary/fcr:versions/fcr:acl at org.fcrepo.importexport.common.TransferProcess.checkValidResponse(TransferProcess.java:203) at org.fcrepo.importexport.exporter.Exporter.export(Exporter.java:269) at org.fcrepo.importexport.exporter.Exporter.export(Exporter.java:289) at org.fcrepo.importexport.exporter.Exporter.exportVersions(Exporter.java:525) at org.fcrepo.importexport.exporter.Exporter.exportBinary(Exporter.java:332) at org.fcrepo.importexport.exporter.Exporter.export(Exporter.java:277) at org.fcrepo.importexport.exporter.Exporter.exportMembers(Exporter.java:431) at org.fcrepo.importexport.exporter.Exporter.exportRdf(Exporter.java:387) at org.fcrepo.importexport.exporter.Exporter.export(Exporter.java:280) at org.fcrepo.importexport.exporter.Exporter.exportMembers(Exporter.java:431) at org.fcrepo.importexport.exporter.Exporter.exportRdf(Exporter.java:387) at org.fcrepo.importexport.exporter.Exporter.export(Exporter.java:280) at org.fcrepo.importexport.exporter.Exporter.run(Exporter.java:227) at org.fcrepo.importexport.ImportExportDriver.run(ImportExportDriver.java:58) at org.fcrepo.importexport.ImportExportDriver.main(ImportExportDriver.java:47)

And from fcrepo:

INFO 14:48:15.390 (FedoraLdp) HEAD for: container/binary INFO 14:48:15.451 (FedoraLdp) HEAD for: container/binary/fcr:versions/20190710184436 INFO 14:48:15.467 (FedoraLdp) GET resource 'container/binary/fcr:versions/20190710184436' INFO 14:48:15.485 (FedoraLdp) HEAD for: container/binary/fcr:versions/20190710184436 INFO 14:48:15.544 (FedoraAcl) GET resource 'container/binary/fcr:versions/' ERROR 14:48:15.544 (RepositoryExceptionMapper) Caught a repository exception: Error converting "fcr:versions" from String to a Name

What I've distilled this down to is this call failing: curl http://localhost:8080/rest/container/fcr:versions/fcr:acl Error converting "fcr:versions" from String to a Name

bseeger commented 5 years ago

Whoops! I was using RC-1 and not RC-2. RC-2 doesn't have the issue (as expected).

bseeger commented 5 years ago

This tests out well and looks good to me. I left a few fairly minor code comments.