search

fcrepo-exts / migration-utils

An in-development framework for managing data migrations from previous versions to 4.x.
Apache License 2.0
13 stars 29 forks source link

Mutliple RELS-EXT versions causes Fedora4 to throw java.lang.OutOfMemoryError: GC overhead limit exceeded #18

Closed glenrobson closed 4 months ago

glenrobson commented 9 years ago

I'm trying to load an object using the migration utils which has 203 versions of RELS-EXT. Details of the migration are contained in #15. The object which caused the problems (llgc-id:3143635) was the last to be migrated and all of the other objects and datastreams were created quickly in Fedora 4.

Creating the RELS-EXT datastream versions for llgc-id:3143635 was slow (about a minute per version) to load and after loading the first 7 caused Fedora4 to throw a OutOfMemoryError. This object also has 102 versions of METS documents and the first 77 loaded quickly.

The output for the migration of this object is below:

INFO 00:55:23.302 (Migrator) Processing "llgc-id:3143635"...
DEBUG 00:55:26.199 (BasicObjectVersionHandler) Considering object llgc-id:3143635 version at 2013-02-12T12:55:44.079Z.
DEBUG 00:55:26.272 (BasicObjectVersionHandler) Considering changed datastream version AUDIT.0
DEBUG 00:55:26.585 (BasicObjectVersionHandler) Considering object llgc-id:3143635 version at 2013-02-12T12:55:44.100Z.
DEBUG 00:55:26.585 (BasicObjectVersionHandler) Considering changed datastream version RELS-EXT.0
DEBUG 00:55:26.959 (BasicObjectVersionHandler) Considering object llgc-id:3143635 version at 2013-02-12T12:55:44.101Z.
DEBUG 00:55:26.960 (BasicObjectVersionHandler) Considering changed datastream version METS.0
DEBUG 00:55:27.177 (BasicObjectVersionHandler) Considering changed datastream version DC.0
DEBUG 00:55:27.183 (BasicObjectVersionHandler) Adding http://purl.org/dc/elements/1.1/identifier value llgc-id:3143635
DEBUG 00:55:27.183 (BasicObjectVersionHandler) Adding http://purl.org/dc/elements/1.1/title value Evening express
DEBUG 00:55:27.183 (BasicObjectVersionHandler) Considering changed datastream version POLICY.0
DEBUG 00:55:27.391 (BasicObjectVersionHandler) Considering changed datastream version RIGHTS.0
DEBUG 00:55:27.506 (BasicObjectVersionHandler) Considering changed datastream version ISSUE_POLICY.0
DEBUG 00:55:27.819 (BasicObjectVersionHandler) Considering changed datastream version PAGE_POLICY.0
DEBUG 00:55:28.331 (BasicObjectVersionHandler) Considering object llgc-id:3143635 version at 2013-02-13T11:25:39.805Z.
DEBUG 00:55:28.332 (BasicObjectVersionHandler) Considering changed datastream version METS.1
DEBUG 00:55:28.501 (BasicObjectVersionHandler) Considering object llgc-id:3143635 version at 2013-02-13T12:48:27.341Z.
DEBUG 00:55:28.502 (BasicObjectVersionHandler) Considering changed datastream version RELS-EXT.1
DEBUG 00:55:28.835 (BasicObjectVersionHandler) Considering object llgc-id:3143635 version at 2013-02-15T17:35:44.468Z.
DEBUG 00:55:28.835 (BasicObjectVersionHandler) Considering changed datastream version METS.2
DEBUG 00:55:29.017 (BasicObjectVersionHandler) Considering object llgc-id:3143635 version at 2013-02-15T17:37:42.742Z.
DEBUG 00:55:29.017 (BasicObjectVersionHandler) Considering changed datastream version METS.3
DEBUG 00:55:29.205 (BasicObjectVersionHandler) Considering object llgc-id:3143635 version at 2013-02-15T17:40:53.018Z.
DEBUG 00:55:29.205 (BasicObjectVersionHandler) Considering changed datastream version METS.4
DEBUG 00:55:29.425 (BasicObjectVersionHandler) Considering object llgc-id:3143635 version at 2013-02-15T17:47:06.648Z.
DEBUG 00:55:29.425 (BasicObjectVersionHandler) Considering changed datastream version METS.5
DEBUG 00:55:29.617 (BasicObjectVersionHandler) Considering object llgc-id:3143635 version at 2013-02-15T17:54:36.573Z.
DEBUG 00:55:29.618 (BasicObjectVersionHandler) Considering changed datastream version METS.6
DEBUG 00:55:29.822 (BasicObjectVersionHandler) Considering object llgc-id:3143635 version at 2013-02-15T18:01:05.996Z.
DEBUG 00:55:29.822 (BasicObjectVersionHandler) Considering changed datastream version METS.7
DEBUG 00:55:30.019 (BasicObjectVersionHandler) Considering object llgc-id:3143635 version at 2013-02-15T18:07:19.949Z.
DEBUG 00:55:30.019 (BasicObjectVersionHandler) Considering changed datastream version METS.8
DEBUG 00:55:30.226 (BasicObjectVersionHandler) Considering object llgc-id:3143635 version at 2013-02-15T18:11:55.388Z.
DEBUG 00:55:30.226 (BasicObjectVersionHandler) Considering changed datastream version METS.9
DEBUG 00:55:30.461 (BasicObjectVersionHandler) Considering object llgc-id:3143635 version at 2013-02-15T18:18:12.397Z.
DEBUG 00:55:30.461 (BasicObjectVersionHandler) Considering changed datastream version METS.10
DEBUG 00:55:30.669 (BasicObjectVersionHandler) Considering object llgc-id:3143635 version at 2013-02-15T18:23:58.472Z.
DEBUG 00:55:30.670 (BasicObjectVersionHandler) Considering changed datastream version METS.11
DEBUG 00:55:30.909 (BasicObjectVersionHandler) Considering object llgc-id:3143635 version at 2013-02-15T18:31:25.332Z.
DEBUG 00:55:30.910 (BasicObjectVersionHandler) Considering changed datastream version METS.12
DEBUG 00:55:31.115 (BasicObjectVersionHandler) Considering object llgc-id:3143635 version at 2013-02-15T18:34:39.529Z.
DEBUG 00:55:31.116 (BasicObjectVersionHandler) Considering changed datastream version METS.13
DEBUG 00:55:31.657 (BasicObjectVersionHandler) Considering object llgc-id:3143635 version at 2013-02-15T18:40:53.485Z.
DEBUG 00:55:31.657 (BasicObjectVersionHandler) Considering changed datastream version METS.14
DEBUG 00:55:33.333 (BasicObjectVersionHandler) Considering object llgc-id:3143635 version at 2013-02-15T18:48:32.543Z.
DEBUG 00:55:33.333 (BasicObjectVersionHandler) Considering changed datastream version METS.15
DEBUG 00:55:33.568 (BasicObjectVersionHandler) Considering object llgc-id:3143635 version at 2013-02-15T18:56:02.289Z.
DEBUG 00:55:33.570 (BasicObjectVersionHandler) Considering changed datastream version METS.16
DEBUG 00:55:33.868 (BasicObjectVersionHandler) Considering object llgc-id:3143635 version at 2013-02-15T19:02:46.197Z.
DEBUG 00:55:33.869 (BasicObjectVersionHandler) Considering changed datastream version METS.17
DEBUG 00:55:34.346 (BasicObjectVersionHandler) Considering object llgc-id:3143635 version at 2013-02-15T19:10:26.164Z.
DEBUG 00:55:34.346 (BasicObjectVersionHandler) Considering changed datastream version METS.18
DEBUG 00:55:34.553 (BasicObjectVersionHandler) Considering object llgc-id:3143635 version at 2013-02-15T19:16:15.040Z.
DEBUG 00:55:34.554 (BasicObjectVersionHandler) Considering changed datastream version METS.19
DEBUG 00:55:34.812 (BasicObjectVersionHandler) Considering object llgc-id:3143635 version at 2013-02-15T19:22:15.115Z.
DEBUG 00:55:34.823 (BasicObjectVersionHandler) Considering changed datastream version METS.20
DEBUG 00:55:35.092 (BasicObjectVersionHandler) Considering object llgc-id:3143635 version at 2013-02-15T19:28:42.752Z.
DEBUG 00:55:35.092 (BasicObjectVersionHandler) Considering changed datastream version METS.21
DEBUG 00:55:35.347 (BasicObjectVersionHandler) Considering object llgc-id:3143635 version at 2013-02-15T19:34:54.822Z.
DEBUG 00:55:35.348 (BasicObjectVersionHandler) Considering changed datastream version METS.22
DEBUG 00:55:35.677 (BasicObjectVersionHandler) Considering object llgc-id:3143635 version at 2013-02-15T19:40:13.000Z.
DEBUG 00:55:35.678 (BasicObjectVersionHandler) Considering changed datastream version METS.23
DEBUG 00:55:35.932 (BasicObjectVersionHandler) Considering object llgc-id:3143635 version at 2013-02-15T19:46:30.078Z.
DEBUG 00:55:35.933 (BasicObjectVersionHandler) Considering changed datastream version METS.24
DEBUG 00:55:36.175 (BasicObjectVersionHandler) Considering object llgc-id:3143635 version at 2013-02-15T19:53:00.831Z.
DEBUG 00:55:36.176 (BasicObjectVersionHandler) Considering changed datastream version METS.25
DEBUG 00:55:36.414 (BasicObjectVersionHandler) Considering object llgc-id:3143635 version at 2013-02-15T19:59:11.411Z.
DEBUG 00:55:36.414 (BasicObjectVersionHandler) Considering changed datastream version METS.26
DEBUG 00:55:36.672 (BasicObjectVersionHandler) Considering object llgc-id:3143635 version at 2013-02-15T20:05:37.764Z.
DEBUG 00:55:36.672 (BasicObjectVersionHandler) Considering changed datastream version METS.27
DEBUG 00:55:36.959 (BasicObjectVersionHandler) Considering object llgc-id:3143635 version at 2013-02-15T20:13:33.132Z.
DEBUG 00:55:36.959 (BasicObjectVersionHandler) Considering changed datastream version METS.28
DEBUG 00:55:37.231 (BasicObjectVersionHandler) Considering object llgc-id:3143635 version at 2013-02-15T20:19:40.976Z.
DEBUG 00:55:37.231 (BasicObjectVersionHandler) Considering changed datastream version METS.29
DEBUG 00:55:37.441 (BasicObjectVersionHandler) Considering object llgc-id:3143635 version at 2013-02-15T20:27:29.580Z.
DEBUG 00:55:37.442 (BasicObjectVersionHandler) Considering changed datastream version METS.30
DEBUG 00:55:37.675 (BasicObjectVersionHandler) Considering object llgc-id:3143635 version at 2013-02-15T20:30:06.115Z.
DEBUG 00:55:37.676 (BasicObjectVersionHandler) Considering changed datastream version RELS-EXT.2
DEBUG 00:55:38.196 (BasicObjectVersionHandler) Considering object llgc-id:3143635 version at 2013-02-15T20:32:52.379Z.
DEBUG 00:55:38.196 (BasicObjectVersionHandler) Considering changed datastream version METS.31
DEBUG 00:55:38.519 (BasicObjectVersionHandler) Considering object llgc-id:3143635 version at 2013-02-15T20:38:48.607Z.
DEBUG 00:55:38.520 (BasicObjectVersionHandler) Considering changed datastream version METS.32
DEBUG 00:55:38.764 (BasicObjectVersionHandler) Considering object llgc-id:3143635 version at 2013-02-15T20:44:04.388Z.
DEBUG 00:55:38.764 (BasicObjectVersionHandler) Considering changed datastream version METS.33
DEBUG 00:55:39.113 (BasicObjectVersionHandler) Considering object llgc-id:3143635 version at 2013-02-15T20:50:13.112Z.
DEBUG 00:55:39.114 (BasicObjectVersionHandler) Considering changed datastream version METS.34
DEBUG 00:55:39.413 (BasicObjectVersionHandler) Considering object llgc-id:3143635 version at 2013-02-15T20:58:54.923Z.
DEBUG 00:55:39.413 (BasicObjectVersionHandler) Considering changed datastream version METS.35
DEBUG 00:55:39.774 (BasicObjectVersionHandler) Considering object llgc-id:3143635 version at 2013-02-15T21:07:15.899Z.
DEBUG 00:55:39.774 (BasicObjectVersionHandler) Considering changed datastream version METS.36
DEBUG 00:55:40.047 (BasicObjectVersionHandler) Considering object llgc-id:3143635 version at 2013-02-15T21:12:02.838Z.
DEBUG 00:55:40.047 (BasicObjectVersionHandler) Considering changed datastream version METS.37
DEBUG 00:55:40.308 (BasicObjectVersionHandler) Considering object llgc-id:3143635 version at 2013-02-15T21:17:12.602Z.
DEBUG 00:55:40.310 (BasicObjectVersionHandler) Considering changed datastream version METS.38
DEBUG 00:55:40.614 (BasicObjectVersionHandler) Considering object llgc-id:3143635 version at 2013-02-15T21:23:19.307Z.
DEBUG 00:55:40.614 (BasicObjectVersionHandler) Considering changed datastream version METS.39
DEBUG 00:55:40.898 (BasicObjectVersionHandler) Considering object llgc-id:3143635 version at 2013-02-15T21:29:06.033Z.
DEBUG 00:55:40.899 (BasicObjectVersionHandler) Considering changed datastream version METS.40
DEBUG 00:55:41.209 (BasicObjectVersionHandler) Considering object llgc-id:3143635 version at 2013-02-15T21:37:00.485Z.
DEBUG 00:55:41.210 (BasicObjectVersionHandler) Considering changed datastream version METS.41
DEBUG 00:55:41.562 (BasicObjectVersionHandler) Considering object llgc-id:3143635 version at 2013-02-15T21:42:53.954Z.
DEBUG 00:55:41.562 (BasicObjectVersionHandler) Considering changed datastream version METS.42
DEBUG 00:55:41.867 (BasicObjectVersionHandler) Considering object llgc-id:3143635 version at 2013-02-15T21:50:20.239Z.
DEBUG 00:55:41.868 (BasicObjectVersionHandler) Considering changed datastream version METS.43
DEBUG 00:55:42.236 (BasicObjectVersionHandler) Considering object llgc-id:3143635 version at 2013-02-15T21:55:21.785Z.
DEBUG 00:55:42.237 (BasicObjectVersionHandler) Considering changed datastream version METS.44
DEBUG 00:55:42.516 (BasicObjectVersionHandler) Considering object llgc-id:3143635 version at 2013-02-15T22:00:59.153Z.
DEBUG 00:55:42.517 (BasicObjectVersionHandler) Considering changed datastream version METS.45
DEBUG 00:55:42.870 (BasicObjectVersionHandler) Considering object llgc-id:3143635 version at 2013-02-15T22:06:55.098Z.
DEBUG 00:55:42.870 (BasicObjectVersionHandler) Considering changed datastream version METS.46
DEBUG 00:55:43.301 (BasicObjectVersionHandler) Considering object llgc-id:3143635 version at 2013-02-15T22:14:21.727Z.
DEBUG 00:55:43.301 (BasicObjectVersionHandler) Considering changed datastream version METS.47
DEBUG 00:55:43.618 (BasicObjectVersionHandler) Considering object llgc-id:3143635 version at 2013-02-15T22:19:51.986Z.
DEBUG 00:55:43.618 (BasicObjectVersionHandler) Considering changed datastream version METS.48
DEBUG 00:55:43.892 (BasicObjectVersionHandler) Considering object llgc-id:3143635 version at 2013-02-15T22:26:07.331Z.
DEBUG 00:55:43.892 (BasicObjectVersionHandler) Considering changed datastream version METS.49
DEBUG 00:55:44.234 (BasicObjectVersionHandler) Considering object llgc-id:3143635 version at 2013-02-15T22:33:31.204Z.
DEBUG 00:55:44.234 (BasicObjectVersionHandler) Considering changed datastream version METS.50
DEBUG 00:55:44.537 (BasicObjectVersionHandler) Considering object llgc-id:3143635 version at 2013-02-15T22:39:16.632Z.
DEBUG 00:55:44.537 (BasicObjectVersionHandler) Considering changed datastream version METS.51
DEBUG 00:55:44.891 (BasicObjectVersionHandler) Considering object llgc-id:3143635 version at 2013-02-15T22:44:20.397Z.
DEBUG 00:55:44.891 (BasicObjectVersionHandler) Considering changed datastream version METS.52
DEBUG 00:55:45.216 (BasicObjectVersionHandler) Considering object llgc-id:3143635 version at 2013-02-15T22:48:45.754Z.
DEBUG 00:55:45.216 (BasicObjectVersionHandler) Considering changed datastream version METS.53
DEBUG 00:55:45.569 (BasicObjectVersionHandler) Considering object llgc-id:3143635 version at 2013-02-15T22:54:09.371Z.
DEBUG 00:55:45.569 (BasicObjectVersionHandler) Considering changed datastream version METS.54
DEBUG 00:55:46.308 (BasicObjectVersionHandler) Considering object llgc-id:3143635 version at 2013-02-15T23:01:35.811Z.
DEBUG 00:55:46.308 (BasicObjectVersionHandler) Considering changed datastream version METS.55
DEBUG 00:55:46.593 (BasicObjectVersionHandler) Considering object llgc-id:3143635 version at 2013-02-15T23:07:02.131Z.
DEBUG 00:55:46.594 (BasicObjectVersionHandler) Considering changed datastream version METS.56
DEBUG 00:55:46.984 (BasicObjectVersionHandler) Considering object llgc-id:3143635 version at 2013-02-15T23:13:14.642Z.
DEBUG 00:55:46.985 (BasicObjectVersionHandler) Considering changed datastream version METS.57
DEBUG 00:55:47.332 (BasicObjectVersionHandler) Considering object llgc-id:3143635 version at 2013-02-15T23:20:31.797Z.
DEBUG 00:55:47.332 (BasicObjectVersionHandler) Considering changed datastream version METS.58
DEBUG 00:55:47.666 (BasicObjectVersionHandler) Considering object llgc-id:3143635 version at 2013-02-15T23:26:03.744Z.
DEBUG 00:55:47.666 (BasicObjectVersionHandler) Considering changed datastream version METS.59
DEBUG 00:55:47.972 (BasicObjectVersionHandler) Considering object llgc-id:3143635 version at 2013-02-15T23:32:11.370Z.
DEBUG 00:55:47.973 (BasicObjectVersionHandler) Considering changed datastream version METS.60
DEBUG 00:55:48.413 (BasicObjectVersionHandler) Considering object llgc-id:3143635 version at 2013-02-15T23:37:33.135Z.
DEBUG 00:55:48.413 (BasicObjectVersionHandler) Considering changed datastream version METS.61
DEBUG 00:55:48.830 (BasicObjectVersionHandler) Considering object llgc-id:3143635 version at 2013-02-15T23:43:50.575Z.
DEBUG 00:55:48.831 (BasicObjectVersionHandler) Considering changed datastream version METS.62
DEBUG 00:55:49.211 (BasicObjectVersionHandler) Considering object llgc-id:3143635 version at 2013-02-15T23:49:18.250Z.
DEBUG 00:55:49.211 (BasicObjectVersionHandler) Considering changed datastream version METS.63
DEBUG 00:55:50.291 (BasicObjectVersionHandler) Considering object llgc-id:3143635 version at 2013-02-15T23:55:23.870Z.
DEBUG 00:55:50.292 (BasicObjectVersionHandler) Considering changed datastream version METS.64
DEBUG 00:55:50.856 (BasicObjectVersionHandler) Considering object llgc-id:3143635 version at 2013-02-16T00:02:46.898Z.
DEBUG 00:55:50.856 (BasicObjectVersionHandler) Considering changed datastream version METS.65
DEBUG 00:55:51.161 (BasicObjectVersionHandler) Considering object llgc-id:3143635 version at 2013-02-16T00:09:03.510Z.
DEBUG 00:55:51.162 (BasicObjectVersionHandler) Considering changed datastream version METS.66
DEBUG 00:55:51.480 (BasicObjectVersionHandler) Considering object llgc-id:3143635 version at 2013-02-16T00:15:01.620Z.
DEBUG 00:55:51.480 (BasicObjectVersionHandler) Considering changed datastream version METS.67
DEBUG 00:55:51.787 (BasicObjectVersionHandler) Considering object llgc-id:3143635 version at 2013-02-16T00:22:22.981Z.
DEBUG 00:55:51.787 (BasicObjectVersionHandler) Considering changed datastream version METS.68
DEBUG 00:55:52.172 (BasicObjectVersionHandler) Considering object llgc-id:3143635 version at 2013-02-16T00:30:00.326Z.
DEBUG 00:55:52.172 (BasicObjectVersionHandler) Considering changed datastream version METS.69
DEBUG 00:55:52.425 (BasicObjectVersionHandler) Considering object llgc-id:3143635 version at 2013-02-16T00:36:13.450Z.
DEBUG 00:55:52.425 (BasicObjectVersionHandler) Considering changed datastream version METS.70
DEBUG 00:55:53.010 (BasicObjectVersionHandler) Considering object llgc-id:3143635 version at 2013-02-16T00:43:48.832Z.
DEBUG 00:55:53.010 (BasicObjectVersionHandler) Considering changed datastream version METS.71
DEBUG 00:55:53.685 (BasicObjectVersionHandler) Considering object llgc-id:3143635 version at 2013-02-16T00:50:14.771Z.
DEBUG 00:55:53.685 (BasicObjectVersionHandler) Considering changed datastream version METS.72
DEBUG 00:55:54.233 (BasicObjectVersionHandler) Considering object llgc-id:3143635 version at 2013-02-16T00:56:03.364Z.
DEBUG 00:55:54.234 (BasicObjectVersionHandler) Considering changed datastream version METS.73
DEBUG 00:55:54.546 (BasicObjectVersionHandler) Considering object llgc-id:3143635 version at 2013-02-16T01:03:48.549Z.
DEBUG 00:55:54.546 (BasicObjectVersionHandler) Considering changed datastream version METS.74
DEBUG 00:55:54.906 (BasicObjectVersionHandler) Considering object llgc-id:3143635 version at 2013-02-16T01:09:42.328Z.
DEBUG 00:55:54.906 (BasicObjectVersionHandler) Considering changed datastream version METS.75
DEBUG 00:55:55.234 (BasicObjectVersionHandler) Considering object llgc-id:3143635 version at 2013-02-16T01:16:00.802Z.
DEBUG 00:55:55.235 (BasicObjectVersionHandler) Considering changed datastream version METS.76
DEBUG 00:55:55.883 (BasicObjectVersionHandler) Considering object llgc-id:3143635 version at 2013-02-16T01:22:27.047Z.
DEBUG 00:55:55.884 (BasicObjectVersionHandler) Considering changed datastream version METS.77
DEBUG 00:55:56.284 (BasicObjectVersionHandler) Considering object llgc-id:3143635 version at 2013-02-18T09:32:00.925Z.
DEBUG 00:55:56.285 (BasicObjectVersionHandler) Considering changed datastream version RELS-EXT.3
DEBUG 00:55:57.436 (BasicObjectVersionHandler) Considering object llgc-id:3143635 version at 2013-02-18T09:37:16.483Z.
DEBUG 00:55:57.436 (BasicObjectVersionHandler) Considering changed datastream version RELS-EXT.4
DEBUG 00:56:03.953 (BasicObjectVersionHandler) Considering object llgc-id:3143635 version at 2013-02-18T09:44:14.292Z.
DEBUG 00:56:03.954 (BasicObjectVersionHandler) Considering changed datastream version RELS-EXT.5
DEBUG 00:57:01.182 (BasicObjectVersionHandler) Considering object llgc-id:3143635 version at 2013-02-18T09:55:56.163Z.
DEBUG 00:57:01.182 (BasicObjectVersionHandler) Considering changed datastream version RELS-EXT.6
DEBUG 01:10:41.618 (BasicObjectVersionHandler) Considering object llgc-id:3143635 version at 2013-02-18T10:09:16.065Z.
DEBUG 01:10:41.622 (BasicObjectVersionHandler) Considering changed datastream version RELS-EXT.7
ERROR 01:17:21.102 (FedoraResourceImpl) error updating resource http://localhost:8888/rest/migrated-fedora3/llgc-id/31/43/63/5: 500 Request failed.
[WARNING] 
java.lang.reflect.InvocationTargetException
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:497)
    at org.codehaus.mojo.exec.ExecJavaMojo$1.run(ExecJavaMojo.java:293)
    at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.RuntimeException: org.fcrepo.client.FedoraException: error updating resource http://localhost:8888/rest/migrated-fedora3/llgc-id/31/43/63/5: 500 Request failed.
    at org.fcrepo.migration.handlers.BasicObjectVersionHandler.updateResourceProperties(BasicObjectVersionHandler.java:533)
    at org.fcrepo.migration.handlers.BasicObjectVersionHandler.updateObjectProperties(BasicObjectVersionHandler.java:237)
    at org.fcrepo.migration.handlers.BasicObjectVersionHandler.processObjectVersions(BasicObjectVersionHandler.java:168)
    at org.fcrepo.migration.handlers.VersionAbstractionFedoraObjectHandler.processObject(VersionAbstractionFedoraObjectHandler.java:95)
    at org.fcrepo.migration.handlers.ObjectAbstractionStreamingFedoraObjectHandler.completeObject(ObjectAbstractionStreamingFedoraObjectHandler.java:67)
    at org.fcrepo.migration.foxml11.Foxml11InputStreamFedoraObjectProcessor.processObject(Foxml11InputStreamFedoraObjectProcessor.java:129)
    at org.fcrepo.migration.Migrator.run(Migrator.java:101)
    at org.fcrepo.migration.Migrator.main(Migrator.java:37)
    ... 6 more
Caused by: org.fcrepo.client.FedoraException: error updating resource http://localhost:8888/rest/migrated-fedora3/llgc-id/31/43/63/5: 500 Request failed.
    at org.fcrepo.client.impl.FedoraResourceImpl.updateProperties(FedoraResourceImpl.java:192)
    at org.fcrepo.migration.handlers.BasicObjectVersionHandler.updateResourceProperties(BasicObjectVersionHandler.java:530)
    ... 13 more
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 22:24 min
[INFO] Finished at: 2015-05-20T01:17:21+01:00
[INFO] Final Memory: 35M/692M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.codehaus.mojo:exec-maven-plugin:1.4.0:java (default-cli) on project migration-utils: An exception occured while executing the Java class. null: InvocationTargetException: org.fcrepo.client.FedoraException: error updating resource http://localhost:8888/rest/migrated-fedora3/llgc-id/31/43/63/5: 500 Request failed. -> [Help 1]
[ERROR] 
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException

real    22m26.970s
user    0m20.998s
sys 0m4.066s
maclaptop04x6:f4-migration-utils gmr$

The output from Fedora is below:

INFO 00:55:55.801 (FedoraVersioning) Request to add version 'imported-version-80' for 'migrated-fedora3/llgc-id/31/43/63/5'
INFO 00:55:55.932 (FedoraLdp) PUT resource 'migrated-fedora3/llgc-id/31/43/63/5/METS'
INFO 00:55:56.209 (FedoraLdp) GET resource 'migrated-fedora3/llgc-id/31/43/63/5/METS/fcr:metadata'
INFO 00:55:56.252 (FedoraVersioning) Request to add version 'imported-version-81' for 'migrated-fedora3/llgc-id/31/43/63/5'
INFO 00:55:56.300 (FedoraLdp) HEAD for: migrated-fedora3/collection/scif_apn
INFO 00:55:56.362 (FedoraLdp) HEAD for: migrated-fedora3/collection/digitisation
INFO 00:55:56.385 (FedoraLdp) HEAD for: migrated-fedora3/collection/scif
INFO 00:55:56.410 (FedoraLdp) HEAD for: migrated-fedora3/model/scif-newspapers-title
INFO 00:55:56.443 (FedoraLdp) PATCH for 'migrated-fedora3/llgc-id/31/43/63/5'
INFO 00:55:57.305 (FedoraLdp) GET resource 'migrated-fedora3/llgc-id/31/43/63/5'
INFO 00:55:57.404 (FedoraVersioning) Request to add version 'imported-version-82' for 'migrated-fedora3/llgc-id/31/43/63/5'
INFO 00:55:57.449 (FedoraLdp) HEAD for: migrated-fedora3/collection/scif_apn
INFO 00:55:57.477 (FedoraLdp) HEAD for: migrated-fedora3/collection/digitisation
INFO 00:55:57.503 (FedoraLdp) HEAD for: migrated-fedora3/collection/scif
INFO 00:55:57.529 (FedoraLdp) HEAD for: migrated-fedora3/model/scif-newspapers-title
INFO 00:55:57.584 (FedoraLdp) PATCH for 'migrated-fedora3/llgc-id/31/43/63/5'
INFO 00:56:03.866 (FedoraLdp) GET resource 'migrated-fedora3/llgc-id/31/43/63/5'
INFO 00:56:03.925 (FedoraVersioning) Request to add version 'imported-version-83' for 'migrated-fedora3/llgc-id/31/43/63/5'
INFO 00:56:03.967 (FedoraLdp) HEAD for: migrated-fedora3/collection/scif_apn
INFO 00:56:03.993 (FedoraLdp) HEAD for: migrated-fedora3/collection/digitisation
INFO 00:56:04.020 (FedoraLdp) HEAD for: migrated-fedora3/collection/scif
INFO 00:56:04.048 (FedoraLdp) HEAD for: migrated-fedora3/model/scif-newspapers-title
INFO 00:56:04.074 (FedoraLdp) PATCH for 'migrated-fedora3/llgc-id/31/43/63/5'
INFO 00:57:00.774 (FedoraLdp) GET resource 'migrated-fedora3/llgc-id/31/43/63/5'
INFO 00:57:00.876 (FedoraVersioning) Request to add version 'imported-version-84' for 'migrated-fedora3/llgc-id/31/43/63/5'
INFO 00:57:01.195 (FedoraLdp) HEAD for: migrated-fedora3/collection/scif_apn
INFO 00:57:01.219 (FedoraLdp) HEAD for: migrated-fedora3/collection/digitisation
INFO 00:57:01.241 (FedoraLdp) HEAD for: migrated-fedora3/collection/scif
INFO 00:57:01.287 (FedoraLdp) HEAD for: migrated-fedora3/model/scif-newspapers-title
INFO 00:57:01.346 (FedoraLdp) PATCH for 'migrated-fedora3/llgc-id/31/43/63/5'
INFO 01:10:41.439 (FedoraLdp) GET resource 'migrated-fedora3/llgc-id/31/43/63/5'
INFO 01:10:41.568 (FedoraVersioning) Request to add version 'imported-version-85' for 'migrated-fedora3/llgc-id/31/43/63/5'
INFO 01:10:41.636 (FedoraLdp) HEAD for: migrated-fedora3/collection/scif_apn
INFO 01:10:41.656 (FedoraLdp) HEAD for: migrated-fedora3/collection/digitisation
INFO 01:10:41.676 (FedoraLdp) HEAD for: migrated-fedora3/collection/scif
INFO 01:10:41.696 (FedoraLdp) HEAD for: migrated-fedora3/model/scif-newspapers-title
INFO 01:10:41.725 (FedoraLdp) PATCH for 'migrated-fedora3/llgc-id/31/43/63/5'
[qtp1610565169-24] WARN org.eclipse.jetty.servlet.ServletHandler - 
javax.servlet.ServletException: org.glassfish.jersey.server.ContainerException: java.lang.OutOfMemoryError: GC overhead limit exceeded
    at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:393)
    at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:381)
    at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:344)
    at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:221)
    at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:769)
    at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585)
    at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
    at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:577)
    at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:223)
    at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1125)
    at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515)
    at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
    at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1059)
    at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
    at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:215)
    at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:110)
    at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)
    at org.eclipse.jetty.server.Server.handle(Server.java:497)
    at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:311)
    at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:248)
    at org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:540)
    at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:610)
    at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:539)
    at java.lang.Thread.run(Thread.java:745)
Caused by: org.glassfish.jersey.server.ContainerException: java.lang.OutOfMemoryError: GC overhead limit exceeded
    at org.glassfish.jersey.servlet.internal.ResponseWriter.rethrow(ResponseWriter.java:256)
    at org.glassfish.jersey.servlet.internal.ResponseWriter.failure(ResponseWriter.java:238)
    at org.glassfish.jersey.server.ServerRuntime$Responder.process(ServerRuntime.java:439)
    at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:277)
    at org.glassfish.jersey.internal.Errors$1.call(Errors.java:271)
    at org.glassfish.jersey.internal.Errors$1.call(Errors.java:267)
    at org.glassfish.jersey.internal.Errors.process(Errors.java:315)
    at org.glassfish.jersey.internal.Errors.process(Errors.java:297)
    at org.glassfish.jersey.internal.Errors.process(Errors.java:267)
    at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:297)
    at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:254)
    at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:1030)
    at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:373)
    ... 23 more
Caused by: java.lang.OutOfMemoryError: GC overhead limit exceeded
    at com.hp.hpl.jena.mem.HashCommon.keyIterator(HashCommon.java:283)
    at com.hp.hpl.jena.mem.HashedTripleBunch.iterator(HashedTripleBunch.java:106)
    at com.hp.hpl.jena.mem.NodeToTriplesMapMem.iterator(NodeToTriplesMapMem.java:125)
    at com.hp.hpl.jena.mem.GraphTripleStoreBase.find(GraphTripleStoreBase.java:167)
    at com.hp.hpl.jena.mem.GraphMem.graphBaseFind(GraphMem.java:137)
    at com.hp.hpl.jena.graph.impl.GraphBase.find(GraphBase.java:268)
    at com.hp.hpl.jena.graph.impl.GraphBase.graphBaseFind(GraphBase.java:290)
    at com.hp.hpl.jena.graph.impl.GraphBase.find(GraphBase.java:287)
    at com.hp.hpl.jena.sparql.engine.iterator.QueryIterTriplePattern$TripleMapper.<init>(QueryIterTriplePattern.java:80)
    at com.hp.hpl.jena.sparql.engine.iterator.QueryIterTriplePattern.nextStage(QueryIterTriplePattern.java:53)
    at com.hp.hpl.jena.sparql.engine.iterator.QueryIterRepeatApply.makeNextStage(QueryIterRepeatApply.java:110)
    at com.hp.hpl.jena.sparql.engine.iterator.QueryIterRepeatApply.hasNextBinding(QueryIterRepeatApply.java:67)
    at com.hp.hpl.jena.sparql.engine.iterator.QueryIteratorBase.hasNext(QueryIteratorBase.java:112)
    at com.hp.hpl.jena.sparql.engine.iterator.QueryIterRepeatApply.makeNextStage(QueryIterRepeatApply.java:103)
    at com.hp.hpl.jena.sparql.engine.iterator.QueryIterRepeatApply.hasNextBinding(QueryIterRepeatApply.java:67)
    at com.hp.hpl.jena.sparql.engine.iterator.QueryIteratorBase.hasNext(QueryIteratorBase.java:112)
    at com.hp.hpl.jena.sparql.engine.iterator.QueryIterBlockTriples.hasNextBinding(QueryIterBlockTriples.java:64)
    at com.hp.hpl.jena.sparql.engine.iterator.QueryIteratorBase.hasNext(QueryIteratorBase.java:112)
    at com.hp.hpl.jena.sparql.engine.iterator.QueryIteratorWrapper.hasNextBinding(QueryIteratorWrapper.java:40)
    at com.hp.hpl.jena.sparql.engine.iterator.QueryIteratorBase.hasNext(QueryIteratorBase.java:112)
    at com.hp.hpl.jena.sparql.engine.iterator.QueryIteratorWrapper.hasNextBinding(QueryIteratorWrapper.java:40)
    at com.hp.hpl.jena.sparql.engine.iterator.QueryIteratorBase.hasNext(QueryIteratorBase.java:112)
    at org.apache.jena.atlas.data.AbstractDataBag.addAll(AbstractDataBag.java:74)
    at com.hp.hpl.jena.sparql.modify.UpdateEngineWorker.visit(UpdateEngineWorker.java:367)
    at com.hp.hpl.jena.sparql.modify.request.UpdateDeleteWhere.visit(UpdateDeleteWhere.java:38)
    at com.hp.hpl.jena.sparql.modify.UpdateVisitorSink.send(UpdateVisitorSink.java:46)
    at com.hp.hpl.jena.sparql.modify.UpdateVisitorSink.send(UpdateVisitorSink.java:26)
    at org.apache.jena.atlas.iterator.Iter.sendToSink(Iter.java:667)
    at org.apache.jena.atlas.iterator.Iter.sendToSink(Iter.java:674)
    at com.hp.hpl.jena.sparql.modify.UpdateProcessorBase.execute(UpdateProcessorBase.java:61)
    at com.hp.hpl.jena.update.UpdateAction.execute$(UpdateAction.java:234)
    at com.hp.hpl.jena.update.UpdateAction.execute(UpdateAction.java:224)
[qtp1610565169-24] WARN org.eclipse.jetty.server.HttpChannel - /rest/migrated-fedora3/llgc-id/31/43/63/5
javax.servlet.ServletException: javax.servlet.ServletException: org.glassfish.jersey.server.ContainerException: java.lang.OutOfMemoryError: GC overhead limit exceeded
    at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:130)
    at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)
    at org.eclipse.jetty.server.Server.handle(Server.java:497)
    at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:311)
    at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:248)
    at org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:540)
    at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:610)
    at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:539)
    at java.lang.Thread.run(Thread.java:745)
Caused by: javax.servlet.ServletException: org.glassfish.jersey.server.ContainerException: java.lang.OutOfMemoryError: GC overhead limit exceeded
    at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:393)
    at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:381)
    at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:344)
    at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:221)
    at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:769)
    at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585)
    at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
    at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:577)
    at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:223)
    at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1125)
    at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515)
    at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
    at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1059)
    at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
    at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:215)
    at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:110)
    ... 8 more
Caused by: org.glassfish.jersey.server.ContainerException: java.lang.OutOfMemoryError: GC overhead limit exceeded
    at org.glassfish.jersey.servlet.internal.ResponseWriter.rethrow(ResponseWriter.java:256)
    at org.glassfish.jersey.servlet.internal.ResponseWriter.failure(ResponseWriter.java:238)
    at org.glassfish.jersey.server.ServerRuntime$Responder.process(ServerRuntime.java:439)
    at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:277)
    at org.glassfish.jersey.internal.Errors$1.call(Errors.java:271)
    at org.glassfish.jersey.internal.Errors$1.call(Errors.java:267)
    at org.glassfish.jersey.internal.Errors.process(Errors.java:315)
    at org.glassfish.jersey.internal.Errors.process(Errors.java:297)
    at org.glassfish.jersey.internal.Errors.process(Errors.java:267)
    at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:297)
    at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:254)
    at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:1030)
    at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:373)
    ... 23 more
Caused by: java.lang.OutOfMemoryError: GC overhead limit exceeded
    at com.hp.hpl.jena.mem.HashCommon.keyIterator(HashCommon.java:283)
    at com.hp.hpl.jena.mem.HashedTripleBunch.iterator(HashedTripleBunch.java:106)
    at com.hp.hpl.jena.mem.NodeToTriplesMapMem.iterator(NodeToTriplesMapMem.java:125)
    at com.hp.hpl.jena.mem.GraphTripleStoreBase.find(GraphTripleStoreBase.java:167)
    at com.hp.hpl.jena.mem.GraphMem.graphBaseFind(GraphMem.java:137)
    at com.hp.hpl.jena.graph.impl.GraphBase.find(GraphBase.java:268)
    at com.hp.hpl.jena.graph.impl.GraphBase.graphBaseFind(GraphBase.java:290)
    at com.hp.hpl.jena.graph.impl.GraphBase.find(GraphBase.java:287)
    at com.hp.hpl.jena.sparql.engine.iterator.QueryIterTriplePattern$TripleMapper.<init>(QueryIterTriplePattern.java:80)
    at com.hp.hpl.jena.sparql.engine.iterator.QueryIterTriplePattern.nextStage(QueryIterTriplePattern.java:53)
    at com.hp.hpl.jena.sparql.engine.iterator.QueryIterRepeatApply.makeNextStage(QueryIterRepeatApply.java:110)
    at com.hp.hpl.jena.sparql.engine.iterator.QueryIterRepeatApply.hasNextBinding(QueryIterRepeatApply.java:67)
    at com.hp.hpl.jena.sparql.engine.iterator.QueryIteratorBase.hasNext(QueryIteratorBase.java:112)
    at com.hp.hpl.jena.sparql.engine.iterator.QueryIterRepeatApply.makeNextStage(QueryIterRepeatApply.java:103)
    at com.hp.hpl.jena.sparql.engine.iterator.QueryIterRepeatApply.hasNextBinding(QueryIterRepeatApply.java:67)
    at com.hp.hpl.jena.sparql.engine.iterator.QueryIteratorBase.hasNext(QueryIteratorBase.java:112)
    at com.hp.hpl.jena.sparql.engine.iterator.QueryIterBlockTriples.hasNextBinding(QueryIterBlockTriples.java:64)
    at com.hp.hpl.jena.sparql.engine.iterator.QueryIteratorBase.hasNext(QueryIteratorBase.java:112)
    at com.hp.hpl.jena.sparql.engine.iterator.QueryIteratorWrapper.hasNextBinding(QueryIteratorWrapper.java:40)
    at com.hp.hpl.jena.sparql.engine.iterator.QueryIteratorBase.hasNext(QueryIteratorBase.java:112)
    at com.hp.hpl.jena.sparql.engine.iterator.QueryIteratorWrapper.hasNextBinding(QueryIteratorWrapper.java:40)
    at com.hp.hpl.jena.sparql.engine.iterator.QueryIteratorBase.hasNext(QueryIteratorBase.java:112)
    at org.apache.jena.atlas.data.AbstractDataBag.addAll(AbstractDataBag.java:74)
    at com.hp.hpl.jena.sparql.modify.UpdateEngineWorker.visit(UpdateEngineWorker.java:367)
    at com.hp.hpl.jena.sparql.modify.request.UpdateDeleteWhere.visit(UpdateDeleteWhere.java:38)
    at com.hp.hpl.jena.sparql.modify.UpdateVisitorSink.send(UpdateVisitorSink.java:46)
    at com.hp.hpl.jena.sparql.modify.UpdateVisitorSink.send(UpdateVisitorSink.java:26)
    at org.apache.jena.atlas.iterator.Iter.sendToSink(Iter.java:667)
    at org.apache.jena.atlas.iterator.Iter.sendToSink(Iter.java:674)
    at com.hp.hpl.jena.sparql.modify.UpdateProcessorBase.execute(UpdateProcessorBase.java:61)
    at com.hp.hpl.jena.update.UpdateAction.execute$(UpdateAction.java:234)
    at com.hp.hpl.jena.update.UpdateAction.execute(UpdateAction.java:224)
[qtp1610565169-24] WARN org.eclipse.jetty.server.HttpChannel - Could not send response error 500: javax.servlet.ServletException: javax.servlet.ServletException: org.glassfish.jersey.server.ContainerException: java.lang.OutOfMemoryError: GC overhead limit exceeded
mikedurbin commented 9 years ago

This is likely related to https://jira.duraspace.org/browse/FCREPO-1294. We've escalated the issue (discussed it as a high priority last week) and I think we have an idea how to fix it, but no one's had the time yet to work on it. Do you have any interest in working on the Fedora 4 code base? ;)

glenrobson commented 9 years ago

Thanks Mike, I can't promise Ill be able to sort the issue but I can have a look around and see if there is anything I can do. Is there a writeup of how you think it can be fixed? I read the issue above and the related tech meeting notes and it looks like its been identified as a blank node issue. Are there any pointers on where I should start looking?

mikedurbin commented 9 years ago

I was mostly kidding. I'm going to look first into whether it's the blank nodes that is causing the problem, by changing the way we migrate some information to avoid using them.

glenrobson commented 9 years ago

few...., wasn't sure what I was letting myself in for :-). Still happy to help with testing.

Surfrdan commented 4 months ago

Closing stale issues and this was related to Fedora4 by Mike's comments so no longer valid