fcrepo-exts / migration-utils

An in-development framework for managing data migrations from previous versions to 4.x.
Apache License 2.0
13 stars 29 forks source link

resource is locked #15

Closed glenrobson closed 3 months ago

glenrobson commented 9 years ago

Hi,

I'm trying to migrate 6 objects from Fedora3 to Fedora4. I've copied the object XML and datastreams onto a Fedora 4 server matching the directory layout from Fedora 3. I've configured the migration-bean.xml to use nativeFoxmlDirectoryObjectSource and legacyIDResolver and when I ran the consoleLoggingStreamingFedoraObjectHandler it gave me the following:

WARN 16:33:57.917 (InternalIDResolver) Index exists at "/usr/local/fedora3.3/data/index" and will be used.  To clear index, simply delete this directory and re-run the application.
INFO 16:33:58.942 (Migrator) Processing "llgc-id:3143635"...
llgc-id:3143635
  Properties
    info:fedora/fedora-system:def/model#state = Active
    info:fedora/fedora-system:def/model#label = Evening express
    info:fedora/fedora-system:def/model#ownerId = 
    info:fedora/fedora-system:def/model#createdDate = 2013-02-12T12:55:44.079Z
    info:fedora/fedora-system:def/view#lastModifiedDate = 2015-02-12T12:48:05.156Z
  AUDIT version AUDIT.0
  RELS-EXT version RELS-EXT.0
  RELS-EXT version RELS-EXT.1
  RELS-EXT version RELS-EXT.2

... removed lines for RELS-EXT versions up until: 

  RELS-EXT version RELS-EXT.202
  RELS-EXT version RELS-EXT.203
  METS version METS.0
  METS version METS.1

  ... removed lines for METS versions up until: 

  METS version METS.120
  METS version METS.121
  DC version DC.0
  POLICY version POLICY.0
  RIGHTS version RIGHTS.0
  ISSUE_POLICY version ISSUE_POLICY.0
  PAGE_POLICY version PAGE_POLICY.0
llgc-id:3143635 parsed in 2179ms.
INFO 16:34:01.121 (Migrator) Processing "llgc-id:3143640"...
llgc-id:3143640
  Properties
    info:fedora/fedora-system:def/model#state = Active
    info:fedora/fedora-system:def/model#label = [4]
    info:fedora/fedora-system:def/model#ownerId = 
    info:fedora/fedora-system:def/model#createdDate = 2013-02-12T12:50:40.831Z
    info:fedora/fedora-system:def/view#lastModifiedDate = 2013-06-05T16:21:05.203Z
  AUDIT version AUDIT.0
  RELS-EXT version RELS-EXT.0
  METS version METS.0
  METS version METS.1
  DC version DC.0
  RIGHTS version RIGHTS.0
  POLICY version POLICY.0
  archive version archive.0
  jp2 version jp2.0
  coordinates version coordinates.0
llgc-id:3143640 parsed in 14ms.
INFO 16:34:01.135 (Migrator) Processing "llgc-id:3143637"...
llgc-id:3143637
  Properties
    info:fedora/fedora-system:def/model#state = Active
    info:fedora/fedora-system:def/model#label = [1]
    info:fedora/fedora-system:def/model#ownerId = 
    info:fedora/fedora-system:def/model#createdDate = 2013-02-12T12:50:39.558Z
    info:fedora/fedora-system:def/view#lastModifiedDate = 2013-06-05T16:20:26.464Z
  AUDIT version AUDIT.0
  RELS-EXT version RELS-EXT.0
  METS version METS.0
  METS version METS.1
  DC version DC.0
  RIGHTS version RIGHTS.0
  POLICY version POLICY.0
  archive version archive.0
  jp2 version jp2.0
  coordinates version coordinates.0
llgc-id:3143637 parsed in 40ms.
INFO 16:34:01.175 (Migrator) Processing "llgc-id:3143638"...
llgc-id:3143638
  Properties
    info:fedora/fedora-system:def/model#state = Active
    info:fedora/fedora-system:def/model#label = [2]
    info:fedora/fedora-system:def/model#ownerId = 
    info:fedora/fedora-system:def/model#createdDate = 2013-02-12T12:50:40.019Z
    info:fedora/fedora-system:def/view#lastModifiedDate = 2013-06-05T16:20:25.175Z
  AUDIT version AUDIT.0
  RELS-EXT version RELS-EXT.0
  METS version METS.0
  METS version METS.1
  DC version DC.0
  RIGHTS version RIGHTS.0
  POLICY version POLICY.0
  archive version archive.0
  jp2 version jp2.0
  coordinates version coordinates.0
llgc-id:3143638 parsed in 30ms.
INFO 16:34:01.205 (Migrator) Processing "llgc-id:3143639"...
llgc-id:3143639
  Properties
    info:fedora/fedora-system:def/model#state = Active
    info:fedora/fedora-system:def/model#label = [3]
    info:fedora/fedora-system:def/model#ownerId = 
    info:fedora/fedora-system:def/model#createdDate = 2013-02-12T12:50:40.439Z
    info:fedora/fedora-system:def/view#lastModifiedDate = 2013-06-05T16:23:55.407Z
  AUDIT version AUDIT.0
  RELS-EXT version RELS-EXT.0
  METS version METS.0
  METS version METS.1
  DC version DC.0
  RIGHTS version RIGHTS.0
  POLICY version POLICY.0
  archive version archive.0
  jp2 version jp2.0
  coordinates version coordinates.0
llgc-id:3143639 parsed in 7ms.
INFO 16:34:01.212 (Migrator) Processing "llgc-id:3143636"...
llgc-id:3143636
  Properties
    info:fedora/fedora-system:def/model#state = Active
    info:fedora/fedora-system:def/model#label = 1891-04-01
    info:fedora/fedora-system:def/model#ownerId = 
    info:fedora/fedora-system:def/model#createdDate = 2013-02-12T12:50:40.925Z
    info:fedora/fedora-system:def/view#lastModifiedDate = 2014-11-06T00:46:23.896Z
  AUDIT version AUDIT.0
  RELS-EXT version RELS-EXT.0
  RELS-EXT version RELS-EXT.1
  METS version METS.0
  METS version METS.1
  DC version DC.0
  RIGHTS version RIGHTS.0
  POLICY version POLICY.0
  PAGE_POLICY version PAGE_POLICY.0
llgc-id:3143636 parsed in 16ms.

but when I try and run with objectAbstraction instead of consoleLoggingStreamingFedoraObjectHandler I get:

[INFO] --- exec-maven-plugin:1.4.0:java (default-cli) @ migration-utils ---
WARN 16:42:16.613 (InternalIDResolver) Index exists at "/usr/local/fedora3.3/data/index" and will be used.  To clear index, simply delete this directory and re-run the application.
INFO 16:42:17.313 (Migrator) Processing "llgc-id:3143635"...
DEBUG 16:42:19.399 (BasicObjectVersionHandler) Considering object llgc-id:3143635 version at 2013-02-12T12:55:44.079Z.
DEBUG 16:42:26.007 (BasicObjectVersionHandler) Considering changed datastream version AUDIT.0
DEBUG 16:42:28.136 (BasicObjectVersionHandler) Considering object llgc-id:3143635 version at 2013-02-12T12:55:44.100Z.
DEBUG 16:42:28.137 (BasicObjectVersionHandler) Considering changed datastream version RELS-EXT.0
ERROR 16:42:29.048 (FedoraResourceImpl) resource http://turin.llgc.org.uk:8080/fcrepo-webapp-4.1.0/rest/migrated-fedora3/llgc-id/31/43/63/5 is locked
[WARNING] 
java.lang.reflect.InvocationTargetException
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:606)
    at org.codehaus.mojo.exec.ExecJavaMojo$1.run(ExecJavaMojo.java:293)
    at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.RuntimeException: org.fcrepo.client.FedoraException: resource is locked: http://turin.llgc.org.uk:8080/fcrepo-webapp-4.1.0/rest/migrated-fedora3/llgc-id/31/43/63/5
    at org.fcrepo.migration.handlers.BasicObjectVersionHandler.updateResourceProperties(BasicObjectVersionHandler.java:513)
    at org.fcrepo.migration.handlers.BasicObjectVersionHandler.updateObjectProperties(BasicObjectVersionHandler.java:225)
    at org.fcrepo.migration.handlers.BasicObjectVersionHandler.processObjectVersions(BasicObjectVersionHandler.java:157)
    at org.fcrepo.migration.handlers.VersionAbstractionFedoraObjectHandler.processObject(VersionAbstractionFedoraObjectHandler.java:95)
    at org.fcrepo.migration.handlers.ObjectAbstractionStreamingFedoraObjectHandler.completeObject(ObjectAbstractionStreamingFedoraObjectHandler.java:67)
    at org.fcrepo.migration.foxml11.Foxml11InputStreamFedoraObjectProcessor.processObject(Foxml11InputStreamFedoraObjectProcessor.java:124)
    at org.fcrepo.migration.Migrator.run(Migrator.java:101)
    at org.fcrepo.migration.Migrator.main(Migrator.java:37)
    ... 6 more
Caused by: org.fcrepo.client.FedoraException: resource is locked: http://turin.llgc.org.uk:8080/fcrepo-webapp-4.1.0/rest/migrated-fedora3/llgc-id/31/43/63/5
    at org.fcrepo.client.impl.FedoraResourceImpl.updateProperties(FedoraResourceImpl.java:187)
    at org.fcrepo.migration.handlers.BasicObjectVersionHandler.updateResourceProperties(BasicObjectVersionHandler.java:510)
    ... 13 more
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 20.930s
[INFO] Finished at: Fri May 15 16:42:29 BST 2015
[INFO] Final Memory: 25M/71M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.codehaus.mojo:exec-maven-plugin:1.4.0:java (default-cli) on project migration-utils: An exception occured while executing the Java class. null: InvocationTargetException: org.fcrepo.client.FedoraException: resource is locked: http://turin.llgc.org.uk:8080/fcrepo-webapp-4.1.0/rest/migrated-fedora3/llgc-id/31/43/63/5 -> [Help 1]
[ERROR] 
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
[gmr@localhost migration-utils-master]$

It seems to create the audit datastream but stop at the first RELS-EXT version (Note we have a lot of RELS-EXTs and METS versions in this object).

Sorry if I've missed something obvious. I'm using Fedora version 4.1.0 and the migration utils downloaded yesterday from the master.

Thanks

Glen

glenrobson commented 9 years ago

Hi,

I tried running the same migration on the latest version of Fedora (4.1.1) and got a different error. Running the migration this time processed the objects in a different order so I don't know if I've come across another problem before reaching an issue with llgc-id:3143635 (which is the object with lots of versions).

Running the migration on 4.1.1 the first object to be processed is llgc-id:3143637 and the migration log looks as follows:

INFO 00:20:51.862 (InternalIDResolver) No index directory specified.  Creating temporary index at "/var/folders/9c/j9t_fxld66jbtyq8bx_kmwwd5k5799/T/tempfile8874507445491749144basedir/index".
INFO 00:20:52.154 (InternalIDResolver) Builidng an index of all the datastreams in "src/test/resources/legacyFS/datastreams"...
INFO 00:20:53.168 (Migrator) Processing "llgc-id:3143637"...
DEBUG 00:20:53.315 (BasicObjectVersionHandler) Considering object llgc-id:3143637 version at 2013-02-12T12:50:39.558Z.
DEBUG 00:20:56.034 (BasicObjectVersionHandler) Considering changed datastream version AUDIT.0
DEBUG 00:20:57.799 (BasicObjectVersionHandler) Considering object llgc-id:3143637 version at 2013-02-12T12:50:39.569Z.
DEBUG 00:20:57.800 (BasicObjectVersionHandler) Considering changed datastream version RELS-EXT.0
DEBUG 00:20:58.945 (BasicObjectVersionHandler) Considering changed datastream version METS.0
DEBUG 00:20:59.527 (BasicObjectVersionHandler) Considering changed datastream version DC.0
DEBUG 00:20:59.592 (BasicObjectVersionHandler) Adding http://purl.org/dc/elements/1.1/identifier value llgc-id:3143637
DEBUG 00:20:59.592 (BasicObjectVersionHandler) Adding http://purl.org/dc/elements/1.1/title value [1]
DEBUG 00:20:59.592 (BasicObjectVersionHandler) Considering changed datastream version RIGHTS.0
DEBUG 00:20:59.835 (BasicObjectVersionHandler) Considering changed datastream version POLICY.0
DEBUG 00:21:00.073 (BasicObjectVersionHandler) Considering changed datastream version archive.0
DEBUG 00:21:00.310 (BasicObjectVersionHandler) Considering changed datastream version jp2.0
ERROR 00:21:00.726 (FedoraResourceImpl) error updating resource http://localhost:8888/rest/migrated-fedora3/llgc-id/31/43/63/7: 400 Bad Request
[WARNING] 
java.lang.reflect.InvocationTargetException
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:606)
    at org.codehaus.mojo.exec.ExecJavaMojo$1.run(ExecJavaMojo.java:293)
    at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.RuntimeException: org.fcrepo.client.FedoraException: error updating resource http://localhost:8888/rest/migrated-fedora3/llgc-id/31/43/63/7: 400 Bad Request
    at org.fcrepo.migration.handlers.BasicObjectVersionHandler.updateResourceProperties(BasicObjectVersionHandler.java:520)
    at org.fcrepo.migration.handlers.BasicObjectVersionHandler.updateObjectProperties(BasicObjectVersionHandler.java:232)
    at org.fcrepo.migration.handlers.BasicObjectVersionHandler.processObjectVersions(BasicObjectVersionHandler.java:164)
    at org.fcrepo.migration.handlers.VersionAbstractionFedoraObjectHandler.processObject(VersionAbstractionFedoraObjectHandler.java:95)
    at org.fcrepo.migration.handlers.ObjectAbstractionStreamingFedoraObjectHandler.completeObject(ObjectAbstractionStreamingFedoraObjectHandler.java:67)
    at org.fcrepo.migration.foxml11.Foxml11InputStreamFedoraObjectProcessor.processObject(Foxml11InputStreamFedoraObjectProcessor.java:129)
    at org.fcrepo.migration.Migrator.run(Migrator.java:101)
    at org.fcrepo.migration.Migrator.main(Migrator.java:37)
    ... 6 more
Caused by: org.fcrepo.client.FedoraException: error updating resource http://localhost:8888/rest/migrated-fedora3/llgc-id/31/43/63/7: 400 Bad Request
    at org.fcrepo.client.impl.FedoraResourceImpl.updateProperties(FedoraResourceImpl.java:191)
    at org.fcrepo.migration.handlers.BasicObjectVersionHandler.updateResourceProperties(BasicObjectVersionHandler.java:517)
    ... 13 more
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 15.144 s
[INFO] Finished at: 2015-05-17T00:21:00+01:00
[INFO] Final Memory: 30M/620M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.codehaus.mojo:exec-maven-plugin:1.4.0:java (default-cli) on project migration-utils: An exception occured while executing the Java class. null: InvocationTargetException: org.fcrepo.client.FedoraException: error updating resource http://localhost:8888/rest/migrated-fedora3/llgc-id/31/43/63/7: 400 Bad Request -> [Help 1]
[ERROR] 
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException

It has loaded the following datastreams:

but it looks like it failed on the coordinates datastream which is a managed ALTO XML file. The analysis for this object is below:

INFO 00:19:34.370 (Migrator) Processing "llgc-id:3143637"...
llgc-id:3143637
  Properties
    info:fedora/fedora-system:def/model#state = Active
    info:fedora/fedora-system:def/model#label = [1]
    info:fedora/fedora-system:def/model#ownerId = 
    info:fedora/fedora-system:def/model#createdDate = 2013-02-12T12:50:39.558Z
    info:fedora/fedora-system:def/view#lastModifiedDate = 2013-06-05T16:20:26.464Z
  AUDIT version AUDIT.0
  RELS-EXT version RELS-EXT.0
  METS version METS.0
  METS version METS.1
  DC version DC.0
  RIGHTS version RIGHTS.0
  POLICY version POLICY.0
  archive version archive.0
  jp2 version jp2.0
  coordinates version coordinates.0

It seems to have loaded the METS but only the first version (unless I'm missing something from the log).

Should I be adding this to JIRA or the fedora tech list rather than github?

Thanks

Glen

glenrobson commented 9 years ago

here is the relevant fedora (4.1.1) log for the error above:

INFO 00:20:53.570 (FedoraLdp) HEAD for: migrated-fedora3/llgc-id/31/43/63/7
INFO 00:20:54.192 (FedoraLdp) PUT resource 'migrated-fedora3/llgc-id/31/43/63/7'
INFO 00:20:54.358 (FedoraLdp) GET resource 'migrated-fedora3/llgc-id/31/43/63/7'
INFO 00:20:56.083 (FedoraLdp) PUT resource 'migrated-fedora3/llgc-id/31/43/63/7/AUDIT'
INFO 00:20:56.178 (FedoraLdp) GET resource 'migrated-fedora3/llgc-id/31/43/63/7/AUDIT/fcr:metadata'
INFO 00:20:56.563 (FedoraLdp) PATCH for 'migrated-fedora3/llgc-id/31/43/63/7/AUDIT/fcr:metadata'
INFO 00:20:57.273 (FedoraLdp) GET resource 'migrated-fedora3/llgc-id/31/43/63/7/AUDIT/fcr:metadata'
INFO 00:20:57.679 (FedoraVersioning) Request to add version 'imported-version-0' for 'migrated-fedora3/llgc-id/31/43/63/7'
INFO 00:20:57.861 (FedoraLdp) HEAD for: migrated-fedora3/model/sc/if/-n/ew/sp/ap/er/s-/pa/ge
INFO 00:20:57.936 (FedoraLdp) PUT resource 'migrated-fedora3/model/sc/if/-n/ew/sp/ap/er/s-/pa/ge'
INFO 00:20:58.024 (FedoraLdp) GET resource 'migrated-fedora3/model/sc/if/-n/ew/sp/ap/er/s-/pa/ge'
INFO 00:20:58.197 (FedoraLdp) HEAD for: migrated-fedora3/llgc-id/31/43/63/6
INFO 00:20:58.248 (FedoraLdp) PUT resource 'migrated-fedora3/llgc-id/31/43/63/6'
INFO 00:20:58.307 (FedoraLdp) GET resource 'migrated-fedora3/llgc-id/31/43/63/6'
INFO 00:20:58.396 (FedoraLdp) HEAD for: migrated-fedora3/collection/sc/if/_a/pn
INFO 00:20:58.451 (FedoraLdp) PUT resource 'migrated-fedora3/collection/sc/if/_a/pn'
INFO 00:20:58.503 (FedoraLdp) GET resource 'migrated-fedora3/collection/sc/if/_a/pn'
INFO 00:20:58.592 (FedoraLdp) HEAD for: migrated-fedora3/collection/sc/if
INFO 00:20:58.672 (FedoraLdp) HEAD for: migrated-fedora3/collection/di/gi/ti/sa/ti/on
INFO 00:20:58.728 (FedoraLdp) PUT resource 'migrated-fedora3/collection/di/gi/ti/sa/ti/on'
INFO 00:20:58.884 (FedoraLdp) GET resource 'migrated-fedora3/collection/di/gi/ti/sa/ti/on'
INFO 00:20:58.985 (FedoraLdp) PUT resource 'migrated-fedora3/llgc-id/31/43/63/7/METS'
INFO 00:20:59.085 (FedoraLdp) GET resource 'migrated-fedora3/llgc-id/31/43/63/7/METS/fcr:metadata'
INFO 00:20:59.252 (FedoraLdp) PATCH for 'migrated-fedora3/llgc-id/31/43/63/7/METS/fcr:metadata'
INFO 00:20:59.402 (FedoraLdp) GET resource 'migrated-fedora3/llgc-id/31/43/63/7/METS/fcr:metadata'
INFO 00:20:59.635 (FedoraLdp) PUT resource 'migrated-fedora3/llgc-id/31/43/63/7/RIGHTS'
INFO 00:20:59.714 (FedoraLdp) GET resource 'migrated-fedora3/llgc-id/31/43/63/7/RIGHTS/fcr:metadata'
INFO 00:20:59.884 (FedoraLdp) PUT resource 'migrated-fedora3/llgc-id/31/43/63/7/POLICY'
INFO 00:20:59.968 (FedoraLdp) GET resource 'migrated-fedora3/llgc-id/31/43/63/7/POLICY/fcr:metadata'
INFO 00:21:00.128 (FedoraLdp) PUT resource 'migrated-fedora3/llgc-id/31/43/63/7/archive'
INFO 00:21:00.206 (FedoraLdp) GET resource 'migrated-fedora3/llgc-id/31/43/63/7/archive/fcr:metadata'
INFO 00:21:00.386 (FedoraLdp) PUT resource 'migrated-fedora3/llgc-id/31/43/63/7/jp2'
INFO 00:21:00.459 (FedoraLdp) GET resource 'migrated-fedora3/llgc-id/31/43/63/7/jp2/fcr:metadata'
INFO 00:21:00.603 (FedoraLdp) PATCH for 'migrated-fedora3/llgc-id/31/43/63/7'
glenrobson commented 9 years ago

Set the debug to trace and added a printStackTrace to BasicObjectVersionHandler.updateResourceProperties exception handler and got the following:

TRACE 01:01:29.869 (BasicObjectVersionHandler) SPARQL: PREFIX  fedoraaccess: <http://fedora.info/definitions/1/0/access/>
PREFIX  fedora3model: <info:fedora/fedora-system:def/model#>
PREFIX  dcterms: <http://purl.org/dc/terms/>

DELETE WHERE 
{
  <> <http://dev.llgc.org.uk/digitisation/identifiers/handle> ?o0 .
  <> <http://www.openarchives.org/OAI/2.0/itemID> ?o1 .
  <> fedora3model:hasModel ?o2 .
  <> <info:fedora/fedora-system:def/relations-external#isPartOf> ?o3 .
  <> <http://dev.llgc.org.uk/digitisation/identifiers/nlw_id> ?o4 .
  <> <info:fedora/fedora-system:def/relations-external#isMemberOfCollection> ?o5 .
  <> <info:fedora/fedora-system:def/relations-external#isMemberOfCollection> ?o6 .
  <> <info:fedora/fedora-system:def/relations-external#isMemberOfCollection> ?o7 .
  <> <http://purl.org/dc/elements/1.1/identifier> ?o .
  <> <http://purl.org/dc/elements/1.1/title> ?o .
} ;
INSERT DATA {
  <> <http://dev.llgc.org.uk/digitisation/identifiers/handle> "10107/3143637" .
  <> <http://www.openarchives.org/OAI/2.0/itemID> "oai:llgc.org.uk:llgc-id:3143637" .
  <> fedora3model:hasModel <http://localhost:8888/rest/migrated-fedora3/model/sc/if/-n/ew/sp/ap/er/s-/pa/ge> .
  <> <info:fedora/fedora-system:def/relations-external#isPartOf> <http://localhost:8888/rest/migrated-fedora3/llgc-id/31/43/63/6> .
  <> <http://dev.llgc.org.uk/digitisation/identifiers/nlw_id> "apnag00100001" .
  <> <info:fedora/fedora-system:def/relations-external#isMemberOfCollection> <http://localhost:8888/rest/migrated-fedora3/collection/sc/if/_a/pn> .
  <> <info:fedora/fedora-system:def/relations-external#isMemberOfCollection> <http://localhost:8888/rest/migrated-fedora3/collection/sc/if> .
  <> <info:fedora/fedora-system:def/relations-external#isMemberOfCollection> <http://localhost:8888/rest/migrated-fedora3/collection/di/gi/ti/sa/ti/on> .
  <> <http://purl.org/dc/elements/1.1/identifier> "llgc-id:3143637" .
  <> <http://purl.org/dc/elements/1.1/title> "[1]" .
}

ERROR 01:01:30.007 (FedoraResourceImpl) error updating resource http://localhost:8888/rest/migrated-fedora3/llgc-id/31/43/63/7: 400 Bad Request
org.fcrepo.client.FedoraException: error updating resource http://localhost:8888/rest/migrated-fedora3/llgc-id/31/43/63/7: 400 Bad Request
    at org.fcrepo.client.impl.FedoraResourceImpl.updateProperties(FedoraResourceImpl.java:191)
    at org.fcrepo.migration.handlers.BasicObjectVersionHandler.updateResourceProperties(BasicObjectVersionHandler.java:517)
    at org.fcrepo.migration.handlers.BasicObjectVersionHandler.updateObjectProperties(BasicObjectVersionHandler.java:232)
    at org.fcrepo.migration.handlers.BasicObjectVersionHandler.processObjectVersions(BasicObjectVersionHandler.java:164)
    at org.fcrepo.migration.handlers.VersionAbstractionFedoraObjectHandler.processObject(VersionAbstractionFedoraObjectHandler.java:95)
    at org.fcrepo.migration.handlers.ObjectAbstractionStreamingFedoraObjectHandler.completeObject(ObjectAbstractionStreamingFedoraObjectHandler.java:67)
    at org.fcrepo.migration.foxml11.Foxml11InputStreamFedoraObjectProcessor.processObject(Foxml11InputStreamFedoraObjectProcessor.java:129)
    at org.fcrepo.migration.Migrator.run(Migrator.java:101)
    at org.fcrepo.migration.Migrator.main(Migrator.java:37)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:606)
    at org.codehaus.mojo.exec.ExecJavaMojo$1.run(ExecJavaMojo.java:293)
    at java.lang.Thread.run(Thread.java:745)
glenrobson commented 9 years ago

pasting the sparql into the update box in the Fedora4 web interface and clicking update returns the following error:

 Bad Request

 Only referenceable nodes may be the value of reference properties 
mikedurbin commented 9 years ago

Thanks for all the helpful debugging information! The last statement seems the most helpful in that it's pointing out that the RDF update is bad because it references other resources (in fedora 4) that don't exist. We did some work a couple weeks ago to fix a similar issue, but perhaps there's a case we missed.

Placeholders for the following objects (referenced in RELS-EXT) should have been made when processing the RELS-EXT here.

Could you verify that the referenced objects exist? They will be empty and have no versions if they were made as placeholders to support creating links before they were migrated.

http://localhost:8888/rest/migrated-fedora3/model/sc/if/-n/ew/sp/ap/er/s-/pa/ge http://localhost:8888/rest/migrated-fedora3/llgc-id/31/43/63/6 http://localhost:8888/rest/migrated-fedora3/collection/sc/if/_a/pn http://localhost:8888/rest/migrated-fedora3/collection/sc/if http://localhost:8888/rest/migrated-fedora3/collection/di/gi/ti/sa/ti/on

glenrobson commented 9 years ago

Thanks for looking into this and for the explanation of the error (the error makes a lot more sense now you've explained it).

All of those links above do resolve to objects and the only one that looks a bit strange is: http://localhost:8888/rest/migrated-fedora3/collection/sc/if which does have the following child: http://localhost:8888/rest/migrated-fedora3/collection/sc/if/_a/pn I've haven't included the collection:scif_apn or collection:scif objects in the foxml directory so I'm guessing this relationship has been created by the SimpleIDMapper class. Would this child/parent relationship cause the problem above?

I was thinking of creating a new MigrationIDMapper that recognises if the characters after : in the PID are not numeric and if so just create the path with the word in it for example the two collections above would be:

http://localhost:8888/rest/migrated-fedora3/collection/scif http://localhost:8888/rest/migrated-fedora3/collection/scif_apn

the vast majority of our content is llgc-id:number which I was going to leave as the current layout (split after two numbers) but our collections and content models have letters after the colon and would look nicer as full words. Do you think that would be a good idea? If the parent child relationship was the problem then this would also fix that.

Thanks

Glen

mikedurbin commented 9 years ago

Your approach for a new MigrationIDMapper is just the sort of customization I expected people to make. I hope it's quite easy to implement.

I've reproduced the error you got by creating a similar RELS-EXT. I'll track the work fixing this bug in our Jira instance here: https://jira.duraspace.org/browse/FCREPO-1547.

glenrobson commented 9 years ago

I've implemented a new MigrationIDMapper:

https://github.com/fcrepo4-labs/migration-utils/compare/master...glenrobson:nlw_changes

and it seems to have fixed the 400 error. I hope it isn't too lazy but I extended your ID Mapper and changed some of the variables and methods from private to protected so I only needed to add a pidToPath method.

On to another problem which Ill create a new issue for.

Surfrdan commented 3 months ago

Supported pathway is now Fedora 3 to 6. Clearing out stale issues