search

ebi-ait / hca-ebi-wrangler-central

This repo is for tracking work related to wrangling datasets for the HCA, associated tasks and for maintaining related documentation.
https://ebi-ait.github.io/hca-ebi-wrangler-central/
Apache License 2.0
7 stars 2 forks source link

Delete Duplicate of the so called "Big Dataset" #888

Open ofanobilbao opened 2 years ago

ofanobilbao commented 2 years ago

Delete duplicate of "Big Dataset" with metadata invalid submission:

There is a duplicate copy of the "Big Dataset" project in Ingest. It contains a metadata invalid submission. I've tried to delete the submission in order to delete the duplicate. Every time I've tried I got an error message that said there were unknown issues that stopped the deletion

Screenshot 2022-09-01 at 10.30.22.png

Acceptance criteria for the task:

MightyAx commented 2 years ago

Running this delete on my local version of production: 2022-09-02 09:56:49.979 ERROR 50483 --- [io-8080-exec-10] o.h.i.c.web.GlobalStateExceptionHandler: Runtime exception encountered on DELETE request to resource http://localhost:8080/submissionEnvelopes/625e96cd0d00514fc697b3c4 

org.springframework.data.mongodb.LazyLoadingException: Unable to lazily resolve DBRef!;
nested exception is org.springframework.core.convert.ConversionFailedException:
Failed to convert from type [java.util.ArrayList<?>] to type [java.util.Set<?>] for value '[
 AbstractEntity(id=625e9a620d00514fc697dfaf, version=11, submissionDate=2022-04-19T11:17:54.901Z, updateDate=2022-04-19T11:39:36.093Z, user=anonymousUser, lastModifiedUser=anonymousUser, type=PROCESS, uuid=3e9d9b5d-3722-4fa6-a19b-57ab50a27432, events=[]),
  AbstractEntity(id=625e9a8b0d00514fc697e1d8, version=9, submissionDate=2022-04-19T11:18:35.590Z, updateDate=2022-04-19T11:40:50.155Z, user=anonymousUser, lastModifiedUser=anonymousUser, type=PROCESS, uuid=856f7386-608c-4850-9921-0c69206dd5de, events=[]),
  AbstractEntity(id=625e9a8b0d00514fc697e1d9, version=9, submissionDate=2022-04-19T11:18:35.650Z, updateDate=2022-04-19T11:40:50.270Z, user=anonymousUser, lastModifiedUser=anonymousUser, type=PROCESS, uuid=3955b60a-70da-4304-a864-1a926c01e5d4, events=[]),
  AbstractEntity(id=625e9a8b0d00514fc697e1da, version=9, submissionDate=2022-04-19T11:18:35.696Z, updateDate=2022-04-19T11:40:50.455Z, user=anonymousUser, lastModifiedUser=anonymousUser, type=PROCESS, uuid=6b6881b0-4418-4a18-8c7b-5fbfc687d960, events=[]),
  AbstractEntity(id=625e9a8b0d00514fc697e1db, version=9, submissionDate=2022-04-19T11:18:35.727Z, updateDate=2022-04-19T11:40:50.604Z, user=anonymousUser, lastModifiedUser=anonymousUser, type=PROCESS, uuid=cfc8c684-c2da-4298-a9ea-cfd6764e15fe, events=[]),
  AbstractEntity(id=625e9a8b0d00514fc697e1dc, version=9, submissionDate=2022-04-19T11:18:35.899Z, updateDate=2022-04-19T11:40:50.707Z, user=anonymousUser, lastModifiedUser=anonymousUser, type=PROCESS, uuid=5dede967-8e69-45d6-bb02-05e50c01f76a, events=[]),
  AbstractEntity(id=625e9a8c0d00514fc697e1eb, version=9, submissionDate=2022-04-19T11:18:36.864Z, updateDate=2022-04-19T11:40:52.156Z, user=anonymousUser, lastModifiedUser=anonymousUser, type=PROCESS, uuid=4ca799db-db3f-4adb-97d0-cfe421c96cb9, events=[]),
  AbstractEntity(id=625e9afc0d00514fc697e94a, version=7, submissionDate=2022-04-19T11:20:28.836Z, updateDate=2022-04-19T11:42:49.389Z, user=anonymousUser, lastModifiedUser=anonymousUser, type=PROCESS, uuid=fc2915f3-e40b-46cf-901b-9030e1916ee8, events=[]),
  AbstractEntity(id=625e9afc0d00514fc697e94b, version=7, submissionDate=2022-04-19T11:20:28.854Z, updateDate=2022-04-19T11:42:49.442Z, user=anonymousUser, lastModifiedUser=anonymousUser, type=PROCESS, uuid=28b04840-546b-4929-9ce2-375bf6b66d07, events=[]),
  AbstractEntity(id=625e9afd0d00514fc697e952, version=7, submissionDate=2022-04-19T11:20:29.273Z, updateDate=2022-04-19T11:42:49.811Z, user=anonymousUser, lastModifiedUser=anonymousUser, type=PROCESS, uuid=0582d273-1d6d-4870-b7d2-15298e12ec22, events=[]),
  AbstractEntity(id=625e9afe0d00514fc697e989, version=7, submissionDate=2022-04-19T11:20:30.036Z, updateDate=2022-04-19T11:42:53.087Z, user=anonymousUser, lastModifiedUser=anonymousUser, type=PROCESS, uuid=9bdb2ad5-f687-4d0f-9a89-40e93e46d5b6, events=[]),
  AbstractEntity(id=625e9afe0d00514fc697e98a, version=7, submissionDate=2022-04-19T11:20:30.047Z, updateDate=2022-04-19T11:42:53.140Z, user=anonymousUser, lastModifiedUser=anonymousUser, type=PROCESS, uuid=92f8fc94-b7fb-4be6-8450-ad86a7650aae, events=[]),
  AbstractEntity(id=625e9afe0d00514fc697e98b, version=7, submissionDate=2022-04-19T11:20:30.057Z, updateDate=2022-04-19T11:42:53.198Z, user=anonymousUser, lastModifiedUser=anonymousUser, type=PROCESS, uuid=a4c0c665-a1ba-404f-9840-b31dc851a21b, events=[]),
  AbstractEntity(id=625e9afe0d00514fc697e98c, version=7, submissionDate=2022-04-19T11:20:30.072Z, updateDate=2022-04-19T11:42:53.254Z, user=anonymousUser, lastModifiedUser=anonymousUser, type=PROCESS, uuid=20f5d94e-92ec-4971-98e6-702028a9b237, events=[]),
  AbstractEntity(id=625e9afe0d00514fc697e98d, version=7, submissionDate=2022-04-19T11:20:30.089Z, updateDate=2022-04-19T11:42:53.309Z, user=anonymousUser, lastModifiedUser=anonymousUser, type=PROCESS, uuid=819f3ac0-e214-42fa-b8d0-3aa188d990e1, events=[]),
  AbstractEntity(id=625e9afe0d00514fc697e98e, version=7, submissionDate=2022-04-19T11:20:30.103Z, updateDate=2022-04-19T11:42:53.394Z, user=anonymousUser, lastModifiedUser=anonymousUser, type=PROCESS, uuid=7d391c2e-6af4-44d2-a1d5-71593c12b0e2, events=[]),
  AbstractEntity(id=625e9afe0d00514fc697e98f, version=7, submissionDate=2022-04-19T11:20:30.118Z, updateDate=2022-04-19T11:42:53.466Z, user=anonymousUser, lastModifiedUser=anonymousUser, type=PROCESS, uuid=d2361c5d-d0c0-41f5-8d62-4b807e5b20ea, events=[]),
  AbstractEntity(id=625e9afe0d00514fc697e990, version=7, submissionDate=2022-04-19T11:20:30.133Z, updateDate=2022-04-19T11:42:53.521Z, user=anonymousUser, lastModifiedUser=anonymousUser, type=PROCESS, uuid=289e5021-aa18-413d-a318-e77229025062, events=[]),
  AbstractEntity(id=625e9afe0d00514fc697e991, version=7, submissionDate=2022-04-19T11:20:30.214Z, updateDate=2022-04-19T11:42:53.576Z, user=anonymousUser, lastModifiedUser=anonymousUser, type=PROCESS, uuid=6be10d70-b612-491c-b54e-2db969eb2e0d, events=[]),
  AbstractEntity(id=625e9afe0d00514fc697e992, version=7, submissionDate=2022-04-19T11:20:30.226Z, updateDate=2022-04-19T11:42:53.626Z, user=anonymousUser, lastModifiedUser=anonymousUser, type=PROCESS, uuid=541ebf57-815e-40fe-b996-443bfc036a68, events=[])
]';
nested exception is java.lang.reflect.InvocationTargetException
    at org.springframework.data.mongodb.core.convert.DefaultDbRefResolver$LazyLoadingInterceptor.resolve(DefaultDbRefResolver.java:485) ~[spring-data-mongodb-2.1.9.RELEASE.jar:2.1.9.RELEASE]
    at org.springframework.data.mongodb.core.convert.DefaultDbRefResolver$LazyLoadingInterceptor.ensureResolved(DefaultDbRefResolver.java:417) ~[spring-data-mongodb-2.1.9.RELEASE.jar:2.1.9.RELEASE]
    at org.springframework.data.mongodb.core.convert.DefaultDbRefResolver$LazyLoadingInterceptor.intercept(DefaultDbRefResolver.java:346) ~[spring-data-mongodb-2.1.9.RELEASE.jar:2.1.9.RELEASE]
    at org.springframework.data.mongodb.core.convert.DefaultDbRefResolver$LazyLoadingInterceptor.invoke(DefaultDbRefResolver.java:307) ~[spring-data-mongodb-2.1.9.RELEASE.jar:2.1.9.RELEASE]
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) ~[spring-aop-5.1.8.RELEASE.jar:5.1.8.RELEASE]
    at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:212) ~[spring-aop-5.1.8.RELEASE.jar:5.1.8.RELEASE]
    at com.sun.proxy.$Proxy236.remove(Unknown Source) ~[na:na]
    at org.humancellatlas.ingest.core.service.strategy.impl.ProcessCrudStrategy.lambda$removeLinksToDocument$4(ProcessCrudStrategy.java:70) ~[main/:na]
    at java.base/java.util.Iterator.forEachRemaining(Iterator.java:133) ~[na:na]
    at java.base/java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1801) ~[na:na]
    at java.base/java.util.stream.ReferencePipeline$Head.forEach(ReferencePipeline.java:658) ~[na:na]
    at org.humancellatlas.ingest.core.service.strategy.impl.ProcessCrudStrategy.removeLinksToDocument(ProcessCrudStrategy.java:69) ~[main/:na]
    at org.humancellatlas.ingest.core.service.strategy.impl.ProcessCrudStrategy.removeLinksToDocument(ProcessCrudStrategy.java:19) ~[main/:na]
    at org.humancellatlas.ingest.core.service.MetadataCrudService.removeLinksToDocument(MetadataCrudService.java:68) ~[main/:na]
    at java.base/java.util.Iterator.forEachRemaining(Iterator.java:133) ~[na:na]
    at java.base/java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1801) ~[na:na]
    at java.base/java.util.stream.ReferencePipeline$Head.forEach(ReferencePipeline.java:658) ~[na:na]
    at org.humancellatlas.ingest.submission.SubmissionEnvelopeService.cleanupLinksToSubmissionMetadata(SubmissionEnvelopeService.java:269) ~[main/:na]
    at org.humancellatlas.ingest.submission.SubmissionEnvelopeService.lambda$deleteSubmission$6(SubmissionEnvelopeService.java:242) ~[main/:na]
    at org.springframework.retry.support.RetryTemplate.doExecute(RetryTemplate.java:329) ~[spring-retry-1.3.1.jar:na]
    at org.springframework.retry.support.RetryTemplate.execute(RetryTemplate.java:209) ~[spring-retry-1.3.1.jar:na]
    at org.humancellatlas.ingest.submission.SubmissionEnvelopeService.deleteSubmission(SubmissionEnvelopeService.java:241) ~[main/:na]
    at org.humancellatlas.ingest.submission.web.SubmissionController.forceDeleteSubmission(SubmissionController.java:372) ~[main/:na]
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:na]
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:na]
    at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:na]
    at java.base/java.lang.reflect.Method.invoke(Method.java:566) ~[na:na]
    at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:343) ~[spring-aop-5.1.8.RELEASE.jar:5.1.8.RELEASE]
    at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:198) ~[spring-aop-5.1.8.RELEASE.jar:5.1.8.RELEASE]
    at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:752) ~[spring-aop-5.1.8.RELEASE.jar:5.1.8.RELEASE]
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) ~[spring-aop-5.1.8.RELEASE.jar:5.1.8.RELEASE]
    at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:93) ~[spring-aop-5.1.8.RELEASE.jar:5.1.8.RELEASE]
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) ~[spring-aop-5.1.8.RELEASE.jar:5.1.8.RELEASE]
    at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:688) ~[spring-aop-5.1.8.RELEASE.jar:5.1.8.RELEASE]
    at org.humancellatlas.ingest.submission.web.SubmissionController$$EnhancerBySpringCGLIB$$c359614b.forceDeleteSubmission(<generated>) ~[main/:na]
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:na]
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:na]
    at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:na]
    at java.base/java.lang.reflect.Method.invoke(Method.java:566) ~[na:na]
    at org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:190) ~[spring-web-5.1.8.RELEASE.jar:5.1.8.RELEASE]
    at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:138) ~[spring-web-5.1.8.RELEASE.jar:5.1.8.RELEASE]
    at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:104) ~[spring-webmvc-5.1.8.RELEASE.jar:5.1.8.RELEASE]
    at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:892) ~[spring-webmvc-5.1.8.RELEASE.jar:5.1.8.RELEASE]
    at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:797) ~[spring-webmvc-5.1.8.RELEASE.jar:5.1.8.RELEASE]
    at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:87) ~[spring-webmvc-5.1.8.RELEASE.jar:5.1.8.RELEASE]
    at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:1039) ~[spring-webmvc-5.1.8.RELEASE.jar:5.1.8.RELEASE]
    at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:942) ~[spring-webmvc-5.1.8.RELEASE.jar:5.1.8.RELEASE]
    at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:1005) ~[spring-webmvc-5.1.8.RELEASE.jar:5.1.8.RELEASE]
    at org.springframework.web.servlet.FrameworkServlet.doDelete(FrameworkServlet.java:930) ~[spring-webmvc-5.1.8.RELEASE.jar:5.1.8.RELEASE]
ofanobilbao commented 2 years ago

Adding to epic as it's one of the required things to do for the clean-up of ingest

ofanobilbao commented 2 years ago

Yesterday I had the same error with the Project with UUID: 54661b25-f2a9-4474-85fe-c482380db2a4. I was going to record in this ticket and when I checked it now, the submission has disappeared and I could delete the duplicate

ofanobilbao commented 2 years ago

Might or might not be related to https://app.zenhub.com/workspaces/operations-5fa2d8f2df78bb000f7fb2b5/issues/ebi-ait/hca-ebi-wrangler-central/916

idazucchi commented 1 year ago

the submission is too big to delete it as is. I am deleting the biomaterials and the processes through the api then I'll try again to delete the submission