idr0011-ledesmafernandez-dad4 S-BIAD866

[will-moore]

idr0011-ledesmafernandez-dad4

[pwalczysko]

Import failed, see details below

``` 2023-02-27 17:25:29,331 11258844 [2-thread-1] INFO ormats.importer.cli.LoggingImportMonitor - IMPORT_STARTED Logfile: 1121916 2023-02-27 17:25:47,761 11277274 [l.Client-1] ERROR ome.formats.importer.cli.ErrorHandler - INTERNAL_EXCEPTION: /data/ngff/idr0011/Plate2-TS-Blue-B.ome.zarr/A/1/0/0/0/0/0/0/0 java.lang.RuntimeException: Failure response on import! Category: ::omero::grid::ImportRequest Name: import-request-failure Parameters: {stacktrace=java.lang.RuntimeException: omero.ValidationException serverStackTrace = "ome.conditions.ValidationException: could not insert: [ome.model.screen.Well]; SQL [insert into well (alpha, blue, "column", creation_id, external_id, group_id, owner_id, permissions, update_id, externalDescription, externalIdentifier, green, plate, red, "row", status, type, version, id) values (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)]; constraint [well_col_row]; nested exception is org.hibernate.exception.ConstraintViolationException: could not insert: [ome.model.screen.Well] at org.springframework.orm.hibernate3.SessionFactoryUtils.convertHibernateAccessException(SessionFactoryUtils.java:660) at org.springframework.orm.hibernate3.HibernateAccessor.convertHibernateAccessException(HibernateAccessor.java:416) at org.springframework.orm.hibernate3.HibernateInterceptor.invoke(HibernateInterceptor.java:125) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179) at org.springframework.transaction.interceptor.TransactionInterceptor$1.proceedWithInvocation(TransactionInterceptor.java:99) at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:282) at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:96) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179) at ome.tools.hibernate.ProxyCleanupFilter$Interceptor.invoke(ProxyCleanupFilter.java:249) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179) at ome.services.util.ServiceHandler.invoke(ServiceHandler.java:121) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179) at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:213) at com.sun.proxy.$Proxy100.doWork(Unknown Source) at ome.services.util.Executor$Impl.execute(Executor.java:447) at ome.services.util.Executor$Impl.execute(Executor.java:392) at ome.services.throttling.Adapter.run(Adapter.java:50) at ome.services.throttling.InThreadThrottlingStrategy.runnableCall(InThreadThrottlingStrategy.java:89) at ome.services.blitz.impl.AbstractAmdServant.runnableCall(AbstractAmdServant.java:154) at ome.services.blitz.impl.MetadataStoreI.saveToDB_async(MetadataStoreI.java:208) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:566) at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:333) at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:190) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157) at omero.cmd.CallContext.invoke(CallContext.java:85) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179) at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:213) at com.sun.proxy.$Proxy139.saveToDB_async(Unknown Source) at omero.api._MetadataStoreTie.saveToDB_async(_MetadataStoreTie.java:84) at omero.api._MetadataStoreDisp.___saveToDB(_MetadataStoreDisp.java:258) at omero.api._MetadataStoreDisp.__dispatch(_MetadataStoreDisp.java:416) at IceInternal.Incoming.invoke(Incoming.java:221) at Ice.ConnectionI.invokeAll(ConnectionI.java:2536) at Ice.ConnectionI.dispatch(ConnectionI.java:1145) at Ice.ConnectionI.message(ConnectionI.java:1056) at IceInternal.ThreadPool.run(ThreadPool.java:395) at IceInternal.ThreadPool.access$300(ThreadPool.java:12) at IceInternal.ThreadPool$EventHandlerThread.run(ThreadPool.java:832) at java.base/java.lang.Thread.run(Thread.java:829) " serverExceptionClass = "ome.conditions.ValidationException" message = "could not insert: [ome.model.screen.Well]; SQL [insert into well (alpha, blue, "column", creation_id, external_id, group_id, owner_id, permissions, update_id, externalDescription, externalIdentifier, green, plate, red, "row", status, type, version, id) values (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)]; constraint [well_col_row]; nested exception is org.hibernate.exception.ConstraintViolationException: could not insert: [ome.model.screen.Well]" at ome.formats.OMEROMetadataStoreClient.saveToDB(OMEROMetadataStoreClient.java:1958) at ome.services.blitz.repo.ManagedImportRequestI.importMetadata(ManagedImportRequestI.java:653) at ome.services.blitz.repo.ManagedImportRequestI.step(ManagedImportRequestI.java:523) at omero.cmd.HandleI.steps(HandleI.java:448) at omero.cmd.HandleI$RunSteps.innerWork(HandleI.java:509) at omero.cmd.HandleI$2.doWork(HandleI.java:383) at omero.cmd.HandleI$2.doWork(HandleI.java:380) at jdk.internal.reflect.GeneratedMethodAccessor302.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:566) at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:333) at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:190) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157) at ome.services.util.Executor$Impl$Interceptor.invoke(Executor.java:568) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179) at ome.security.basic.EventHandler.invoke(EventHandler.java:154) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179) at org.springframework.orm.hibernate3.HibernateInterceptor.invoke(HibernateInterceptor.java:119) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179) at org.springframework.transaction.interceptor.TransactionInterceptor$1.proceedWithInvocation(TransactionInterceptor.java:99) at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:282) at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:96) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179) at ome.tools.hibernate.ProxyCleanupFilter$Interceptor.invoke(ProxyCleanupFilter.java:249) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179) at ome.services.util.ServiceHandler.invoke(ServiceHandler.java:121) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179) at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:213) at com.sun.proxy.$Proxy100.doWork(Unknown Source) at ome.services.util.Executor$Impl.execute(Executor.java:447) at omero.cmd.HandleI.run(HandleI.java:379) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) at ome.services.util.Executor$Impl$1.call(Executor.java:488) at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at java.base/java.lang.Thread.run(Thread.java:829) Caused by: omero.ValidationException serverStackTrace = "ome.conditions.ValidationException: could not insert: [ome.model.screen.Well]; SQL [insert into well (alpha, blue, "column", creation_id, external_id, group_id, owner_id, permissions, update_id, externalDescription, externalIdentifier, green, plate, red, "row", status, type, version, id) values (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)]; constraint [well_col_row]; nested exception is org.hibernate.exception.ConstraintViolationException: could not insert: [ome.model.screen.Well] at org.springframework.orm.hibernate3.SessionFactoryUtils.convertHibernateAccessException(SessionFactoryUtils.java:660) at org.springframework.orm.hibernate3.HibernateAccessor.convertHibernateAccessException(HibernateAccessor.java:416) at org.springframework.orm.hibernate3.HibernateInterceptor.invoke(HibernateInterceptor.java:125) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179) at org.springframework.transaction.interceptor.TransactionInterceptor$1.proceedWithInvocation(TransactionInterceptor.java:99) at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:282) at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:96) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179) at ome.tools.hibernate.ProxyCleanupFilter$Interceptor.invoke(ProxyCleanupFilter.java:249) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179) at ome.services.util.ServiceHandler.invoke(ServiceHandler.java:121) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179) at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:213) at com.sun.proxy.$Proxy100.doWork(Unknown Source) at ome.services.util.Executor$Impl.execute(Executor.java:447) at ome.services.util.Executor$Impl.execute(Executor.java:392) at ome.services.throttling.Adapter.run(Adapter.java:50) at ome.services.throttling.InThreadThrottlingStrategy.runnableCall(InThreadThrottlingStrategy.java:89) at ome.services.blitz.impl.AbstractAmdServant.runnableCall(AbstractAmdServant.java:154) at ome.services.blitz.impl.MetadataStoreI.saveToDB_async(MetadataStoreI.java:208) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:566) at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:333) at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:190) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157) at omero.cmd.CallContext.invoke(CallContext.java:85) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179) at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:213) at com.sun.proxy.$Proxy139.saveToDB_async(Unknown Source) at omero.api._MetadataStoreTie.saveToDB_async(_MetadataStoreTie.java:84) at omero.api._MetadataStoreDisp.___saveToDB(_MetadataStoreDisp.java:258) at omero.api._MetadataStoreDisp.__dispatch(_MetadataStoreDisp.java:416) at IceInternal.Incoming.invoke(Incoming.java:221) at Ice.ConnectionI.invokeAll(ConnectionI.java:2536) at Ice.ConnectionI.dispatch(ConnectionI.java:1145) at Ice.ConnectionI.message(ConnectionI.java:1056) at IceInternal.ThreadPool.run(ThreadPool.java:395) at IceInternal.ThreadPool.access$300(ThreadPool.java:12) at IceInternal.ThreadPool$EventHandlerThread.run(ThreadPool.java:832) at java.base/java.lang.Thread.run(Thread.java:829) " serverExceptionClass = "ome.conditions.ValidationException" message = "could not insert: [ome.model.screen.Well]; SQL [insert into well (alpha, blue, "column", creation_id, external_id, group_id, owner_id, permissions, update_id, externalDescription, externalIdentifier, green, plate, red, "row", status, type, version, id) values (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)]; constraint [well_col_row]; nested exception is org.hibernate.exception.ConstraintViolationException: could not insert: [ome.model.screen.Well]" at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:490) at java.base/java.lang.Class.newInstance(Class.java:584) at IceInternal.BasicStream.createUserException(BasicStream.java:2785) at IceInternal.BasicStream.access$300(BasicStream.java:14) at IceInternal.BasicStream$EncapsDecoder11.throwException(BasicStream.java:3620) at IceInternal.BasicStream.throwException(BasicStream.java:2291) at IceInternal.OutgoingAsync.throwUserException(OutgoingAsync.java:399) at omero.api.MetadataStorePrxHelper.end_saveToDB(MetadataStorePrxHelper.java:715) at omero.api.MetadataStorePrxHelper.saveToDB(MetadataStorePrxHelper.java:593) at omero.api.MetadataStorePrxHelper.saveToDB(MetadataStorePrxHelper.java:580) at ome.formats.OMEROMetadataStoreClient.saveToDB(OMEROMetadataStoreClient.java:1942) ... 36 more , message=omero.ValidationException serverStackTrace = "ome.conditions.ValidationException: could not insert: [ome.model.screen.Well]; SQL [insert into well (alpha, blue, "column", creation_id, external_id, group_id, owner_id, permissions, update_id, externalDescription, externalIdentifier, green, plate, red, "row", status, type, version, id) values (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)]; constraint [well_col_row]; nested exception is org.hibernate.exception.ConstraintViolationException: could not insert: [ome.model.screen.Well] at org.springframework.orm.hibernate3.SessionFactoryUtils.convertHibernateAccessException(SessionFactoryUtils.java:660) at org.springframework.orm.hibernate3.HibernateAccessor.convertHibernateAccessException(HibernateAccessor.java:416) at org.springframework.orm.hibernate3.HibernateInterceptor.invoke(HibernateInterceptor.java:125) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179) at org.springframework.transaction.interceptor.TransactionInterceptor$1.proceedWithInvocation(TransactionInterceptor.java:99) at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:282) at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:96) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179) at ome.tools.hibernate.ProxyCleanupFilter$Interceptor.invoke(ProxyCleanupFilter.java:249) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179) at ome.services.util.ServiceHandler.invoke(ServiceHandler.java:121) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179) at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:213) at com.sun.proxy.$Proxy100.doWork(Unknown Source) at ome.services.util.Executor$Impl.execute(Executor.java:447) at ome.services.util.Executor$Impl.execute(Executor.java:392) at ome.services.throttling.Adapter.run(Adapter.java:50) at ome.services.throttling.InThreadThrottlingStrategy.runnableCall(InThreadThrottlingStrategy.java:89) at ome.services.blitz.impl.AbstractAmdServant.runnableCall(AbstractAmdServant.java:154) at ome.services.blitz.impl.MetadataStoreI.saveToDB_async(MetadataStoreI.java:208) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:566) at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:333) at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:190) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157) at omero.cmd.CallContext.invoke(CallContext.java:85) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179) at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:213) at com.sun.proxy.$Proxy139.saveToDB_async(Unknown Source) at omero.api._MetadataStoreTie.saveToDB_async(_MetadataStoreTie.java:84) at omero.api._MetadataStoreDisp.___saveToDB(_MetadataStoreDisp.java:258) at omero.api._MetadataStoreDisp.__dispatch(_MetadataStoreDisp.java:416) at IceInternal.Incoming.invoke(Incoming.java:221) at Ice.ConnectionI.invokeAll(ConnectionI.java:2536) at Ice.ConnectionI.dispatch(ConnectionI.java:1145) at Ice.ConnectionI.message(ConnectionI.java:1056) at IceInternal.ThreadPool.run(ThreadPool.java:395) at IceInternal.ThreadPool.access$300(ThreadPool.java:12) at IceInternal.ThreadPool$EventHandlerThread.run(ThreadPool.java:832) at java.base/java.lang.Thread.run(Thread.java:829) " serverExceptionClass = "ome.conditions.ValidationException" message = "could not insert: [ome.model.screen.Well]; SQL [insert into well (alpha, blue, "column", creation_id, external_id, group_id, owner_id, permissions, update_id, externalDescription, externalIdentifier, green, plate, red, "row", status, type, version, id) values (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)]; constraint [well_col_row]; nested exception is org.hibernate.exception.ConstraintViolationException: could not insert: [ome.model.screen.Well]"} at ome.formats.importer.ImportLibrary$ImportCallback.onFinished(ImportLibrary.java:807) at omero.cmd.CmdCallbackI.finished(CmdCallbackI.java:334) at omero.cmd._CmdCallbackDisp.___finished(_CmdCallbackDisp.java:118) at omero.cmd._CmdCallbackDisp.__dispatch(_CmdCallbackDisp.java:145) at IceInternal.Incoming.invoke(Incoming.java:221) at Ice.ConnectionI.invokeAll(ConnectionI.java:2536) at Ice.ConnectionI.dispatch(ConnectionI.java:1145) at Ice.ConnectionI.message(ConnectionI.java:1056) at IceInternal.ThreadPool.run(ThreadPool.java:395) at IceInternal.ThreadPool.access$300(ThreadPool.java:12) at IceInternal.ThreadPool$EventHandlerThread.run(ThreadPool.java:832) at java.base/java.lang.Thread.run(Thread.java:829) java.lang.RuntimeException: Failure response on import! Category: ::omero::grid::ImportRequest Name: import-request-failure Parameters: {stacktrace=java.lang.RuntimeException: omero.ValidationException serverStackTrace = "ome.conditions.ValidationException: could not insert: [ome.model.screen.Well]; SQL [insert into well (alpha, blue, "column", creation_id, external_id, group_id, owner_id, permissions, update_id, externalDescription, externalIdentifier, green, plate, red, "row", status, type, version, id) values (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)]; constraint [well_col_row]; nested exception is org.hibernate.exception.ConstraintViolationException: could not insert: [ome.model.screen.Well] at org.springframework.orm.hibernate3.SessionFactoryUtils.convertHibernateAccessException(SessionFactoryUtils.java:660) at org.springframework.orm.hibernate3.HibernateAccessor.convertHibernateAccessException(HibernateAccessor.java:416) at org.springframework.orm.hibernate3.HibernateInterceptor.invoke(HibernateInterceptor.java:125) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179) at org.springframework.transaction.interceptor.TransactionInterceptor$1.proceedWithInvocation(TransactionInterceptor.java:99) at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:282) at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:96) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179) at ome.tools.hibernate.ProxyCleanupFilter$Interceptor.invoke(ProxyCleanupFilter.java:249) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179) at ome.services.util.ServiceHandler.invoke(ServiceHandler.java:121) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179) at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:213) at com.sun.proxy.$Proxy100.doWork(Unknown Source) at ome.services.util.Executor$Impl.execute(Executor.java:447) at ome.services.util.Executor$Impl.execute(Executor.java:392) at ome.services.throttling.Adapter.run(Adapter.java:50) at ome.services.throttling.InThreadThrottlingStrategy.runnableCall(InThreadThrottlingStrategy.java:89) at ome.services.blitz.impl.AbstractAmdServant.runnableCall(AbstractAmdServant.java:154) at ome.services.blitz.impl.MetadataStoreI.saveToDB_async(MetadataStoreI.java:208) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:566) at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:333) at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:190) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157) at omero.cmd.CallContext.invoke(CallContext.java:85) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179) at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:213) at com.sun.proxy.$Proxy139.saveToDB_async(Unknown Source) at omero.api._MetadataStoreTie.saveToDB_async(_MetadataStoreTie.java:84) at omero.api._MetadataStoreDisp.___saveToDB(_MetadataStoreDisp.java:258) at omero.api._MetadataStoreDisp.__dispatch(_MetadataStoreDisp.java:416) at IceInternal.Incoming.invoke(Incoming.java:221) at Ice.ConnectionI.invokeAll(ConnectionI.java:2536) at Ice.ConnectionI.dispatch(ConnectionI.java:1145) at Ice.ConnectionI.message(ConnectionI.java:1056) at IceInternal.ThreadPool.run(ThreadPool.java:395) at IceInternal.ThreadPool.access$300(ThreadPool.java:12) at IceInternal.ThreadPool$EventHandlerThread.run(ThreadPool.java:832) at java.base/java.lang.Thread.run(Thread.java:829) " serverExceptionClass = "ome.conditions.ValidationException" message = "could not insert: [ome.model.screen.Well]; SQL [insert into well (alpha, blue, "column", creation_id, external_id, group_id, owner_id, permissions, update_id, externalDescription, externalIdentifier, green, plate, red, "row", status, type, version, id) values (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)]; constraint [well_col_row]; nested exception is org.hibernate.exception.ConstraintViolationException: could not insert: [ome.model.screen.Well]" at ome.formats.OMEROMetadataStoreClient.saveToDB(OMEROMetadataStoreClient.java:1958) at ome.services.blitz.repo.ManagedImportRequestI.importMetadata(ManagedImportRequestI.java:653) at ome.services.blitz.repo.ManagedImportRequestI.step(ManagedImportRequestI.java:523) at omero.cmd.HandleI.steps(HandleI.java:448) at omero.cmd.HandleI$RunSteps.innerWork(HandleI.java:509) at omero.cmd.HandleI$2.doWork(HandleI.java:383) at omero.cmd.HandleI$2.doWork(HandleI.java:380) at jdk.internal.reflect.GeneratedMethodAccessor302.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:566) at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:333) at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:190) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157) at ome.services.util.Executor$Impl$Interceptor.invoke(Executor.java:568) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179) at ome.security.basic.EventHandler.invoke(EventHandler.java:154) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179) at org.springframework.orm.hibernate3.HibernateInterceptor.invoke(HibernateInterceptor.java:119) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179) at org.springframework.transaction.interceptor.TransactionInterceptor$1.proceedWithInvocation(TransactionInterceptor.java:99) at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:282) at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:96) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179) at ome.tools.hibernate.ProxyCleanupFilter$Interceptor.invoke(ProxyCleanupFilter.java:249) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179) at ome.services.util.ServiceHandler.invoke(ServiceHandler.java:121) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179) at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:213) at com.sun.proxy.$Proxy100.doWork(Unknown Source) at ome.services.util.Executor$Impl.execute(Executor.java:447) at omero.cmd.HandleI.run(HandleI.java:379) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) at ome.services.util.Executor$Impl$1.call(Executor.java:488) at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at java.base/java.lang.Thread.run(Thread.java:829) Caused by: omero.ValidationException serverStackTrace = "ome.conditions.ValidationException: could not insert: [ome.model.screen.Well]; SQL [insert into well (alpha, blue, "column", creation_id, external_id, group_id, owner_id, permissions, update_id, externalDescription, externalIdentifier, green, plate, red, "row", status, type, version, id) values (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)]; constraint [well_col_row]; nested exception is org.hibernate.exception.ConstraintViolationException: could not insert: [ome.model.screen.Well] at org.springframework.orm.hibernate3.SessionFactoryUtils.convertHibernateAccessException(SessionFactoryUtils.java:660) at org.springframework.orm.hibernate3.HibernateAccessor.convertHibernateAccessException(HibernateAccessor.java:416) at org.springframework.orm.hibernate3.HibernateInterceptor.invoke(HibernateInterceptor.java:125) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179) at org.springframework.transaction.interceptor.TransactionInterceptor$1.proceedWithInvocation(TransactionInterceptor.java:99) at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:282) at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:96) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179) at ome.tools.hibernate.ProxyCleanupFilter$Interceptor.invoke(ProxyCleanupFilter.java:249) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179) at ome.services.util.ServiceHandler.invoke(ServiceHandler.java:121) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179) at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:213) at com.sun.proxy.$Proxy100.doWork(Unknown Source) at ome.services.util.Executor$Impl.execute(Executor.java:447) at ome.services.util.Executor$Impl.execute(Executor.java:392) at ome.services.throttling.Adapter.run(Adapter.java:50) at ome.services.throttling.InThreadThrottlingStrategy.runnableCall(InThreadThrottlingStrategy.java:89) at ome.services.blitz.impl.AbstractAmdServant.runnableCall(AbstractAmdServant.java:154) at ome.services.blitz.impl.MetadataStoreI.saveToDB_async(MetadataStoreI.java:208) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:566) at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:333) at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:190) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157) at omero.cmd.CallContext.invoke(CallContext.java:85) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179) at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:213) at com.sun.proxy.$Proxy139.saveToDB_async(Unknown Source) at omero.api._MetadataStoreTie.saveToDB_async(_MetadataStoreTie.java:84) at omero.api._MetadataStoreDisp.___saveToDB(_MetadataStoreDisp.java:258) at omero.api._MetadataStoreDisp.__dispatch(_MetadataStoreDisp.java:416) at IceInternal.Incoming.invoke(Incoming.java:221) at Ice.ConnectionI.invokeAll(ConnectionI.java:2536) at Ice.ConnectionI.dispatch(ConnectionI.java:1145) at Ice.ConnectionI.message(ConnectionI.java:1056) at IceInternal.ThreadPool.run(ThreadPool.java:395) at IceInternal.ThreadPool.access$300(ThreadPool.java:12) at IceInternal.ThreadPool$EventHandlerThread.run(ThreadPool.java:832) at java.base/java.lang.Thread.run(Thread.java:829) " serverExceptionClass = "ome.conditions.ValidationException" message = "could not insert: [ome.model.screen.Well]; SQL [insert into well (alpha, blue, "column", creation_id, external_id, group_id, owner_id, permissions, update_id, externalDescription, externalIdentifier, green, plate, red, "row", status, type, version, id) values (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)]; constraint [well_col_row]; nested exception is org.hibernate.exception.ConstraintViolationException: could not insert: [ome.model.screen.Well]" at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:490) at java.base/java.lang.Class.newInstance(Class.java:584) at IceInternal.BasicStream.createUserException(BasicStream.java:2785) at IceInternal.BasicStream.access$300(BasicStream.java:14) at IceInternal.BasicStream$EncapsDecoder11.throwException(BasicStream.java:3620) at IceInternal.BasicStream.throwException(BasicStream.java:2291) at IceInternal.OutgoingAsync.throwUserException(OutgoingAsync.java:399) at omero.api.MetadataStorePrxHelper.end_saveToDB(MetadataStorePrxHelper.java:715) at omero.api.MetadataStorePrxHelper.saveToDB(MetadataStorePrxHelper.java:593) at omero.api.MetadataStorePrxHelper.saveToDB(MetadataStorePrxHelper.java:580) at ome.formats.OMEROMetadataStoreClient.saveToDB(OMEROMetadataStoreClient.java:1942) ... 36 more , message=omero.ValidationException serverStackTrace = "ome.conditions.ValidationException: could not insert: [ome.model.screen.Well]; SQL [insert into well (alpha, blue, "column", creation_id, external_id, group_id, owner_id, permissions, update_id, externalDescription, externalIdentifier, green, plate, red, "row", status, type, version, id) values (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)]; constraint [well_col_row]; nested exception is org.hibernate.exception.ConstraintViolationException: could not insert: [ome.model.screen.Well] at org.springframework.orm.hibernate3.SessionFactoryUtils.convertHibernateAccessException(SessionFactoryUtils.java:660) at org.springframework.orm.hibernate3.HibernateAccessor.convertHibernateAccessException(HibernateAccessor.java:416) at org.springframework.orm.hibernate3.HibernateInterceptor.invoke(HibernateInterceptor.java:125) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179) at org.springframework.transaction.interceptor.TransactionInterceptor$1.proceedWithInvocation(TransactionInterceptor.java:99) at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:282) at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:96) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179) at ome.tools.hibernate.ProxyCleanupFilter$Interceptor.invoke(ProxyCleanupFilter.java:249) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179) at ome.services.util.ServiceHandler.invoke(ServiceHandler.java:121) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179) at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:213) at com.sun.proxy.$Proxy100.doWork(Unknown Source) at ome.services.util.Executor$Impl.execute(Executor.java:447) at ome.services.util.Executor$Impl.execute(Executor.java:392) at ome.services.throttling.Adapter.run(Adapter.java:50) at ome.services.throttling.InThreadThrottlingStrategy.runnableCall(InThreadThrottlingStrategy.java:89) at ome.services.blitz.impl.AbstractAmdServant.runnableCall(AbstractAmdServant.java:154) at ome.services.blitz.impl.MetadataStoreI.saveToDB_async(MetadataStoreI.java:208) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:566) at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:333) at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:190) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157) at omero.cmd.CallContext.invoke(CallContext.java:85) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179) at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:213) at com.sun.proxy.$Proxy139.saveToDB_async(Unknown Source) at omero.api._MetadataStoreTie.saveToDB_async(_MetadataStoreTie.java:84) at omero.api._MetadataStoreDisp.___saveToDB(_MetadataStoreDisp.java:258) at omero.api._MetadataStoreDisp.__dispatch(_MetadataStoreDisp.java:416) at IceInternal.Incoming.invoke(Incoming.java:221) at Ice.ConnectionI.invokeAll(ConnectionI.java:2536) at Ice.ConnectionI.dispatch(ConnectionI.java:1145) at Ice.ConnectionI.message(ConnectionI.java:1056) at IceInternal.ThreadPool.run(ThreadPool.java:395) at IceInternal.ThreadPool.access$300(ThreadPool.java:12) at IceInternal.ThreadPool$EventHandlerThread.run(ThreadPool.java:832) at java.base/java.lang.Thread.run(Thread.java:829) " serverExceptionClass = "ome.conditions.ValidationException" message = "could not insert: [ome.model.screen.Well]; SQL [insert into well (alpha, blue, "column", creation_id, external_id, group_id, owner_id, permissions, update_id, externalDescription, externalIdentifier, green, plate, red, "row", status, type, version, id) values (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)]; constraint [well_col_row]; nested exception is org.hibernate.exception.ConstraintViolationException: could not insert: [ome.model.screen.Well]"} at ome.formats.importer.ImportLibrary$ImportCallback.onFinished(ImportLibrary.java:807) at omero.cmd.CmdCallbackI.finished(CmdCallbackI.java:334) at omero.cmd._CmdCallbackDisp.___finished(_CmdCallbackDisp.java:118) at omero.cmd._CmdCallbackDisp.__dispatch(_CmdCallbackDisp.java:145) at IceInternal.Incoming.invoke(Incoming.java:221) at Ice.ConnectionI.invokeAll(ConnectionI.java:2536) at Ice.ConnectionI.dispatch(ConnectionI.java:1145) at Ice.ConnectionI.message(ConnectionI.java:1056) at IceInternal.ThreadPool.run(ThreadPool.java:395) at IceInternal.ThreadPool.access$300(ThreadPool.java:12) at IceInternal.ThreadPool$EventHandlerThread.run(ThreadPool.java:832) at java.base/java.lang.Thread.run(Thread.java:829) 2023-02-27 17:25:47,768 11277281 [2-thread-1] ERROR ome.formats.importer.ImportLibrary - Error on import java.lang.Exception: Import failure at ome.formats.importer.ImportLibrary.importImage(ImportLibrary.java:701) at ome.formats.importer.ImportLibrary$1.call(ImportLibrary.java:354) at ome.formats.importer.ImportLibrary$1.call(ImportLibrary.java:328) at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at java.base/java.lang.Thread.run(Thread.java:829) 2023-02-27 17:25:47,768 11277281 [2-thread-1] INFO ome.formats.importer.ImportLibrary - Exiting on error ==> Summary 9597 files uploaded, 0 filesets created, 0 images imported, 1 error in 3:07:36.884 ```

[pwalczysko]

Updated the error with full stacktrace https://github.com/IDR/idr-metadata/issues/642#issuecomment-1447820599

[pwalczysko]

ToDo

• take ome-xml alone and import
• will bfconvert show an error on the file ?

[pwalczysko]

cc @dgault

[pwalczysko]

The import of METADATA.ome.xml file which was copied to local machine from /data/ngff/idr0011/Plate2-TS-Blue-B.ome.zarr/OME/METADATA.ome.xml was successful, see user-3 https://merge-ci.openmicroscopy.org/web/webclient/?show=image-258515

[pwalczysko]

Imported the idr0011 plate on pilot-idrtesting - all looks fine, thumbs and full images are generated, no errors. The OMEZarrReader 0.3.1 was used, see https://github.com/IDR/deployment/pull/380

Time to reimport this plate was 2h 15mins

[dominikl]

I'll start conversion to NGFF of the full dataset.

[dominikl]

Converted files will be in pilot-zarr1-dev:/data/idr0011

[will-moore]

Looks like this is done:

(base) [wmoore@pilot-zarr1-dev ~]$ ls /data/idr0011 | wc
    182     182    4555

Matches the 182 filesets expected (see https://github.com/IDR/idr-utils/pull/56)

[dominikl]

👍 It is, thanks Will.

[will-moore]

Created bucket:

$ aws --endpoint-url https://uk1s3.embassy.ebi.ac.uk s3 mb s3://idr0011
make_bucket: idr0011
$ aws --endpoint-url https://uk1s3.embassy.ebi.ac.uk s3api put-bucket-policy --bucket idr0011 --policy file://policy.json
$ aws --endpoint-url https://uk1s3.embassy.ebi.ac.uk s3api put-bucket-cors --bucket idr0011 --cors-configuration file://cors.json

$ /home/wmoore/mc cp -r idr0011/ uk1s3/idr0011/zarr
...
/data/idr0011/Target-2-B.ome.zarr/OME/METADATA.ome.xml:     469.25 GiB / 469.25 GiB ━━━━━━━━━━

# took several hours to upload, but looks like all complete
$ /home/wmoore/mc ls uk1s3/idr0011/zarr | wc
    182     910   10743
$ ls idr0011 | wc
    182     182    4555

e.g. https://hms-dbmi.github.io/vizarr/?source=https://uk1s3.embassy.ebi.ac.uk/idr0011/zarr/Target-2-B.ome.zarr

Validator: https://ome.github.io/ome-ngff-validator/?source=https://uk1s3.embassy.ebi.ac.uk/idr0011/zarr/Target-2-B.ome.zarr

[will-moore]

Going to try import of a plate or two to validate on idr0125-pilot...

ssh -A -o 'ProxyCommand ssh idr-pilot.openmicroscopy.org -W %h:%p' idr0125-omeroreadwrite -L 1080:localhost:80

# list data on s3...
aws s3 ls --no-sign-request --endpoint-url https://uk1s3.embassy.ebi.ac.uk s3://idr0011/zarr/ 

mkdir idr0011
cd idr0011
mkdir Target-1-A.ome.zarr && cd Target-1-A.ome.zarr
aws s3 sync --no-sign-request --exclude '*' --include "*/.z*" --include "*.xml" --endpoint-url https://uk1s3.embassy.ebi.ac.uk s3://idr0011/zarr/Target-1-A.ome.zarr .

cd ../
mkdir Plate1-Blue-A.ome.zarr && cd Plate1-Blue-A.ome.zarr
aws s3 sync --no-sign-request --exclude '*' --include "*/.z*" --include "*.xml" --endpoint-url https://uk1s3.embassy.ebi.ac.uk s3://idr0011/zarr/Plate1-Blue-A.ome.zarr .

cd ../../
sudo mv -f idr0011 /ngff/
sudo chown -R omero-server idr001

Import as usual...

cd /ngff/idr0011
for dir in *; do
  omero import --transfer=ln_s --depth=100 --name=${dir/.ome.zarr/} --skip=all $dir --file /tmp/$dir.log  --errs /tmp/$dir.err;
done

logs...

2023-06-26 12:41:15,068 236225     [l.Client-1] INFO   ormats.importer.cli.LoggingImportMonitor - METADATA_PROCESSED Step: 4 of 5  Logfile: 50486066
2023-06-26 12:41:15,101 236258     [l.Client-0] INFO   ormats.importer.cli.LoggingImportMonitor - OBJECTS_RETURNED Step: 5 of 5  Logfile: 50486066
2023-06-26 12:41:15,801 236958     [l.Client-1] INFO   ormats.importer.cli.LoggingImportMonitor - IMPORT_DONE Imported file: /ngff/idr0011/Plate1-Blue-A.ome.zarr/OME/METADATA.ome.xml
Other imported objects:
Fileset:5287127

==> Summary
346 files uploaded, 1 fileset created, 48 images imported, 0 errors in 0:03:42.206

2023-06-26 12:42:32,512 74446      [l.Client-1] INFO   ormats.importer.cli.LoggingImportMonitor - OBJECTS_RETURNED Step: 5 of 5  Logfile: 50486523
2023-06-26 12:42:32,830 74764      [l.Client-0] INFO   ormats.importer.cli.LoggingImportMonitor - IMPORT_DONE Imported file: /ngff/idr0011/Target-1-A.ome.zarr/OME/METADATA.ome.xml
Other imported objects:
Fileset:5287128

==> Summary
150 files uploaded, 1 fileset created, 20 images imported, 0 errors in 0:01:06.269

Mount s3 buckets and update symlinks...

sudo mkdir /idr0011 && sudo /opt/goofys --endpoint https://uk1s3.embassy.ebi.ac.uk/ -o allow_other idr0011 /idr0011

Can't see the Plates in webclient... No Plates etc. See what is linked to the Fileset(s) created above...

$ omero delete Fileset:5287128 --dry-run --report
Using session for demo@localhost:4064. Idle timeout: 10 min. Current group: Public
omero.cmd.Delete2 Fileset:5287128 Dry run performed
ok
Steps: 4
Elapsed time: 2.03 secs.
Flags: []
Deleted objects
  Detector:155556-155558
  DetectorSettings:51647-51706
  Instrument:146154
  Objective:147902
  ObjectiveSettings:149099-149118
  CommentAnnotation:33516736
  FilesetAnnotationLink:5286926
  Channel:44137844-44137903
  Image:14834772-14834791
  LogicalChannel:16075189-16075248
  OriginalFile:50486523-50486673
  Pixels:14834772-14834791
  PlaneInfo:45114279-45115538
  Fileset:5287128
  FilesetEntry:35937074-35937223
  FilesetJobLink:21495711-21495714
  IndexingJob:21505878
  JobOriginalFileLink:5304542
  MetadataImportJob:21505876
  PixelDataJob:21505877
  UploadJob:21505875

$ omero delete Fileset:5287127 --dry-run --report
Using session for demo@localhost:4064. Idle timeout: 10 min. Current group: Public
omero.cmd.Delete2 Fileset:5287127 Dry run performed
ok
Steps: 4
Elapsed time: 5.836 secs.
Flags: []
Deleted objects
  Detector:155553-155555
  DetectorSettings:51503-51646
  Instrument:146153
  Objective:147901
  ObjectiveSettings:149051-149098
  CommentAnnotation:33516735
  FilesetAnnotationLink:5286925
  Channel:44137700-44137843
  Image:14834724-14834771
  LogicalChannel:16075045-16075188
  OriginalFile:50486066-50486412
  Pixels:14834724-14834771
  PlaneInfo:45111255-45114278
  Fileset:5287127
  FilesetEntry:35936662-35936700,35936767-35937073
  FilesetJobLink:21495707-21495710
  IndexingJob:21505874
  JobOriginalFileLink:5304541
  MetadataImportJob:21505872
  PixelDataJob:21505873
  UploadJob:21505871

No Plates or Wells etc?! Looks like they got imported as Images (orphaned).

[will-moore]

Realised that I'd failed to copy the top level .zattrs for the plates above. Updated them and tried again...

2023-06-26 15:01:10,877 186163     [l.Client-0] INFO   ormats.importer.cli.LoggingImportMonitor - IMPORT_DONE Imported file: /ngff/idr0011/Plate1-Blue-A.ome.zarr/OME/METADATA.ome.xml
Other imported objects:
Fileset:5287129

==> Summary
347 files uploaded, 1 fileset, 1 plate created, 48 images imported, 0 errors in 0:02:51.928

2023-06-26 15:02:32,005 78658      [l.Client-1] INFO   ormats.importer.cli.LoggingImportMonitor - OBJECTS_RETURNED Step: 5 of 5  Logfile: 50487383
2023-06-26 15:02:32,511 79164      [l.Client-0] INFO   ormats.importer.cli.LoggingImportMonitor - IMPORT_DONE Imported file: /ngff/idr0011/Target-1-A.ome.zarr/OME/METADATA.ome.xml
Other imported objects:
Fileset:5287130

==> Summary
151 files uploaded, 1 fileset, 1 plate created, 20 images imported, 0 errors in 0:01:10.869

Wanted to find Plate linked to Fileset but...

$ omero delete Fileset:5287129 --dry-run --report
Using session for demo@localhost:4064. Idle timeout: 10 min. Current group: Public
omero.cmd.Delete2 Fileset:5287129 failed: 'graph-fail'
failed: may not delete Image[14834826] while WellSample[9235200] remains
Steps: 4
Elapsed time: 3.318 secs.
Flags: [FAILURE, CANCELLED]

$ omero delete Fileset:5287130 --dry-run --report
Using session for demo@localhost:4064. Idle timeout: 10 min. Current group: Public
omero.cmd.Delete2 Fileset:5287130 failed: 'graph-fail'
failed: may not delete Image[14834858] while WellSample[9235232] remains
Steps: 4
Elapsed time: 1.728 secs.
Flags: [FAILURE, CANCELLED]

Put the 2 Plates into a screen in webclient, then...

$ python idr-utils/scripts/managed_repo_symlinks.py Screen:3253 /idr0011/zarr/ --report

Fileset: 5287129 /data/OMERO/ManagedRepository/demo_2/Blitz-0-Ice.ThreadPool.Server-7/2023-06/26/14-58-19.152/
Render Image 14834806
fileset_dirs {}
fs_contents ['Plate1-Blue-A.ome.zarr']
Link from /data/OMERO/ManagedRepository/demo_2/Blitz-0-Ice.ThreadPool.Server-7/2023-06/26/14-58-19.152/Plate1-Blue-A.ome.zarr to /idr0011/zarr/Plate1-Blue-A.ome.zarr

Fileset: 5287130 /data/OMERO/ManagedRepository/demo_2/Blitz-0-Ice.ThreadPool.Server-2/2023-06/26/15-01-21.806/
Render Image 14834851
fileset_dirs {}
fs_contents ['Target-1-A.ome.zarr']
Link from /data/OMERO/ManagedRepository/demo_2/Blitz-0-Ice.ThreadPool.Server-2/2023-06/26/15-01-21.806/Target-1-A.ome.zarr to /idr0011/zarr/Target-1-A.ome.zarr

Both plates look identical to IDR:

[will-moore]

Started zip creation...

ssh pilot-zarr1-dev
cd /data/idr0011
screen -S idr0011_zip
for i in */; do sudo zip -r "${i%/}.zip" "$i"; done

[will-moore]

Last zip created this morning at Jun 28 03:34. Took > 30 hours to zip 182 zarrs.

[will-moore]

wget https://ak-delivery04-mul.dhe.ibm.com/sar/CMA/OSA/08q6g/0/ibm-aspera-cli-3.9.6.1467.159c5b1-linux-64-release.sh
chmod +x ibm-aspera-cli-3.9.6.1467.159c5b1-linux-64-release.sh 
bash ibm-aspera-cli-3.9.6.1467.159c5b1-linux-64-release.sh 
cd /.aspera/cli/bin
$ ./ascp -P33001 -i ../etc/asperaweb_id_dsa.openssh -d /data/idr0011/idr0011/ bsaspera_w@hx-fasp-1.ebi.ac.uk:5f/136e8d-**********

...
Completed: 315655334K bytes transferred in 10586 seconds
 (244269K bits/sec), in 182 files, 1 directory.

[will-moore]

All 182 Filesets are viewable at https://uk1s3.embassy.ebi.ac.uk/bia-integrator-data/pages/S-BIAD866.html

Testing on idr0138-pilot with the first 10...

idr0011/Plate11-Green-B.ome.zarr,S-BIAD866/013015ae-64f8-494c-85ee-b4e364960b75,21832
idr0011/Plate2-Yellow-A.ome.zarr,S-BIAD866/029bd12c-9ee5-4278-a177-a14f52466abc,21757
idr0011/Plate16-Blue-B.ome.zarr,S-BIAD866/0980f7a9-2ab6-4116-b9a9-59fcbc5ffc63,21823
idr0011/Plate15-Yellow-A.ome.zarr,S-BIAD866/0e5e1147-449f-4a89-96c6-3a3c85f8c1a7,21851
idr0011/Plate9-Red-A.ome.zarr,S-BIAD866/0ebb098d-7b05-4d54-b9f0-128833580fba,21764
idr0011/Plate5-TS-Yellow-B-repeated.ome.zarr,S-BIAD866/10cd251a-f2ff-4623-99dc-a8aec9555c4f,21910
idr0011/Target-2-A.ome.zarr,S-BIAD866/1217032d-8a87-4bc7-8d22-665791732ec6,21935
idr0011/Plate6-Yellow-A.ome.zarr,S-BIAD866/121878f9-674a-45e1-a051-d883d19fbef3,21858
idr0011/Plate5-TS-Yellow-A.ome.zarr,S-BIAD866/129cf0d8-6f16-4817-bc6c-904c9731dcd7,21908
idr0011/Plate3-Yellow-A.ome.zarr,S-BIAD866/15c1d12d-2d14-4c7a-a90b-1864e59e741b,21790

Found prefix demo_2/2016-07/28 // 18-03-56.973 for fileset 21832
Checking for prefix_dir /data/OMERO/ManagedRepository/demo_2/2016-07/28/18-03-56.973
Creating dir at /data/OMERO/ManagedRepository/demo_2/2016-07/28/18-03-56.973_mkngff
Creating symlink /data/OMERO/ManagedRepository/demo_2/2016-07/28/18-03-56.973_mkngff/013015ae-64f8-494c-85ee-b4e364960b75.zarr -> /bia-integrator-data/S-BIAD866/013015ae-64f8-494c-85ee-b4e364960b75/013015ae-64f8-494c-85ee-b4e364960b75.zarr
Using session for demo@localhost:4064. Idle timeout: 10 min. Current group: Public
Found prefix demo_2/2016-07/28 // 15-41-13.359 for fileset 21757
Checking for prefix_dir /data/OMERO/ManagedRepository/demo_2/2016-07/28/15-41-13.359
Creating dir at /data/OMERO/ManagedRepository/demo_2/2016-07/28/15-41-13.359_mkngff
Creating symlink /data/OMERO/ManagedRepository/demo_2/2016-07/28/15-41-13.359_mkngff/029bd12c-9ee5-4278-a177-a14f52466abc.zarr -> /bia-integrator-data/S-BIAD866/029bd12c-9ee5-4278-a177-a14f52466abc/029bd12c-9ee5-4278-a177-a14f52466abc.zarr
...

[will-moore]

...
 mkngff_fileset 
----------------
        5811635
(1 row)
COMMIT
UPDATE 22
BEGIN
 mkngff_fileset 
----------------
        5811636
(1 row)
COMMIT
UPDATE 19
BEGIN
 mkngff_fileset 
----------------
        5811637
(1 row)
COMMIT
UPDATE 48
BEGIN
 mkngff_fileset 
----------------
        5811638
(1 row)
COMMIT
UPDATE 27
BEGIN
 mkngff_fileset 
----------------
        5811639
(1 row)
COMMIT
UPDATE 48
BEGIN
 mkngff_fileset 
----------------
        5811640
(1 row)
COMMIT

Find last Fileset created...

=> select id from image where fileset = 5811640;
   id    
---------
2852632
...

http://localhost:1040/webclient/?show=image-2852632

Didn't take long to view images...

[will-moore]

On idr-testing:omeroreadwrite...

Had to manually tweak Fileset names where the Plate names in IDR don't match the fileset names on disk:

idr0011/Plate1-Blue-A_TS-Stinger.ome.zarr,S-BIAD866/16014bb5-1926-49f6-80e0-b18cd6e3f3cd
idr0011/Plate1-Red-A-TS-Stinger.ome.zarr,S-BIAD866/1f03d4ec-2783-4d8e-94a1-ee4862ea3d22
idr0011/Plate1-Yellow-B-TS-Stinger.ome.zarr,S-BIAD866/4215f37e-483d-4471-ad57-c5c052a5bda3
idr0011/Plate1-Green-A-TS-Stinger.ome.zarr,S-BIAD866/42db5162-4855-4167-921d-4f99e7c12fea 
idr0011/Plate1-Blue-B-TS-Stinger.ome.zarr,S-BIAD866/82231f2b-aa52-4069-9ced-a8d1f1608422
idr0011/Plate1-Red-B-TS-Stinger.ome.zarr,S-BIAD866/88ca4912-eaaf-4f35-8777-8b649155007e
idr0011/Plate1-Yellow-A-TS-Stinger.ome.zarr,S-BIAD866/ced2d9c5-8eb9-4c48-938d-d768df0611c1
idr0011/Plate1-Green-B-TS-Stinger.ome.zarr,S-BIAD866/dc7c9a6c-db59-4652-a372-78a309594036
idr0011/TS-Stinger-Target-1&2.ome.zarr,S-BIAD866/f58816a3-311c-4b60-aa92-fbc85a619a79

Commited idr0011.csv to https://github.com/IDR/idr-utils/pull/56/files

Started mkngff... - took about 3.5 hours for 182 Filesets - about 1 minute each.

[will-moore]

Ran sql 6:30 am.

Viewing image from first plate to setId: http://localhost:1080/webclient/?show=image-2849940

$ grep -A 2 "saved memo" /opt/omero/server/OMERO.server/var/log/Blitz-0.log | grep -A 2 "13-56-03.328_mkngff"
2023-09-12 05:34:53,887 DEBUG [                   loci.formats.Memoizer] (l.Server-4) saved memo file: /data/OMERO/BioFormatsCache/data/OMERO/ManagedRepository/demo_2/2016-07/28/13-56-03.328_mkngff/7f95aba3-cfbf-4ae8-a106-edaa36f5b07f.zarr/OME/.METADATA.ome.xml.bfmemo (6989006 bytes)
2023-09-12 05:34:53,887 DEBUG [                   loci.formats.Memoizer] (l.Server-4) start[1694496832274] time[61613] tag[loci.formats.Memoizer.setId]
2023-09-12 05:34:53,887 INFO  [                ome.io.nio.PixelsService] (l.Server-4) Creating BfPixelBuffer: /data/OMERO/ManagedRepository/demo_2/2016-07/28/13-56-03.328_mkngff/7f95aba3-cfbf-4ae8-a106-edaa36f5b07f.zarr/OME/METADATA.ome.xml Series: 0

61613ms is 1 minute

[will-moore]

Four plates found to have missing files, although they all look OK where uploaded above. eg. https://ome.github.io/ome-ngff-validator/?source=https://uk1s3.embassy.ebi.ac.uk/idr0011/zarr/Plate3-Yellow-B.ome.zarr

• Plate3-Yellow-B
• Plate11-Blue-A
• Plate13-Red-B
• Plate15-Yellow-A

We could try to download them from there, but probably easier to simply recreate them...

on zarr1-dev-pilot /data/idr0011

screen -S idr0011_bf2raw
conda activate bioformats2raw2
cd /data/idr0011
for i in Plate3-Yellow-B Plate11-Blue-A Plate13-Red-B Plate15-Yellow-A; do
~/bioformats2raw-0.6.0-24/bin/bioformats2raw --memo-directory ../memo  /uod/idr/metadata/idr0011-ledesmafernandez-dad4/screens/$i.screen $i.ome.zarr; done

EDIT: uploading...

./ascp -P33001 -i ../etc/asperaweb_id_dsa.openssh -d /data/idr0011/idr0011 bsaspera_w@hx-fasp-1.ebi.ac.uk:/5f/xxxxxx
Plate11-Blue-A.ome.zarr.zip                                                          100% 1703MB  152Mb/s    02:29
Plate13-Red-B.ome.zarr.zip                                                          100% 1668MB  154Mb/s    04:40
Plate15-Yellow-A.ome.zarr.zip                                                       100% 1662MB  229Mb/s    06:45
Plate3-Yellow-B.ome.zarr.zip                                                        100% 1646MB  280Mb/s    09:07
Completed: 6841932K bytes transferred in 547 seconds
 (102354K bits/sec), in 4 files, 1 directory.

[will-moore]

With check_pixels.py run on ALL images across 4 Screens: no Errors found except the 4 known Plates above: https://github.com/IDR/idr-utils/pull/55#issuecomment-1827971457

[will-moore]

checking for updates... https://uk1s3.embassy.ebi.ac.uk/bia-integrator-data/pages/S-BIAD866.html

No change... idr0011/Plate11-Blue-A https://ome.github.io/ome-ngff-validator/?source=https://uk1s3.embassy.ebi.ac.uk/bia-integrator-data/S-BIAD866/bd01548b-da85-4a66-8422-7b16995fabba/bd01548b-da85-4a66-8422-7b16995fabba.zarr/

[will-moore]

We have the 4 plates available on uk1s3 and they are all valid eg. https://ome.github.io/ome-ngff-validator/?source=https://uk1s3.embassy.ebi.ac.uk/idr0011/zarr/Plate3-Yellow-B.ome.zarr

Let's use those for mkngff so as not to be held up by EBI re-processing them. Plan is to run mkngff on those plates hosted on uk1s3 and apply the sql on idr-next. When the data becomes available on BioStudies, we can update symlinks and clientpaths accordingly.

First, let's create sql and test on idr0125-pilot...

Use login to idr.openmicroscopy.org for mkngff sql (using original Fileset IDs), but we'll need to use different IDs for testing?

idr0011.csv based on https://github.com/IDR/idr-utils/pull/56/commits/125c4e559d5020dacbf359bc2bf7694dd86fb708

idr0011/Plate15-Yellow-A.ome.zarr,Plate15-Yellow-A.ome.zarr,21851
idr0011/Plate13-Red-B.ome.zarr,Plate13-Red-B.ome.zarr,21842
idr0011/Plate3-Yellow-B.ome.zarr,Plate3-Yellow-B.ome.zarr,21789
idr0011/Plate11-Blue-A.ome.zarr,Plate11-Blue-A.ome.zarr,21834

for r in $(cat $IDRID.csv); do
  platename=$(echo $r | cut -d',' -f2)
  fsid=$(echo $r | cut -d',' -f3 | tr -d '[:space:]')
  omero mkngff sql $fsid --clientpath="https://uk1s3.embassy.ebi.ac.uk/idr0011/zarr/$platename" "/idr0011/zarr/$platename" > "$IDRID/$fsid.sql"
done

$ for i in $(ls idr0011); do grep -c ".zattrs" idr0011/$i; done
98
98
98
98

Previously, sql were missing some files:

(base) Williams-MacBook-Pro:mkngff_upgrade_scripts wmoore$ for i in 21789 21834 21842 21851; do grep -c ".zattrs" ngff_filesets/idr0011/$i.sql; done
96
81
81
81

Fixed in https://github.com/IDR/mkngff_upgrade_scripts/commit/1f2949c2937a6959ee0e370181ff48de9139c9bc

[will-moore]

Since idr0125-pilot has not had mkngff run on those 4 Plates, we can run sql with original Fileset IDs as omero-server...

for i in $(ls); do sed -i 's/SECRETUUID/9630ba1e-ed3a-42e3-9296-.../g' $i; done

for r in $(cat $IDRID.csv); do
  platename=$(echo $r | cut -d',' -f2)
  fsid=$(echo $r | cut -d',' -f3 | tr -d '[:space:]')
  psql -U omero -d idr -h $DBHOST -f "$IDRID/$fsid.sql"
  omero mkngff symlink /data/OMERO/ManagedRepository $fsid "/idr0011/zarr/$platename" --bfoptions
done

(venv3) (base) bash-4.2$ echo $IDRID
idr0015
(venv3) (base) bash-4.2$ export IDRID=idr0011
(venv3) (base) bash-4.2$
(venv3) (base) bash-4.2$
(venv3) (base) bash-4.2$
(venv3) (base) bash-4.2$ for r in $(cat $IDRID.csv); do   platename=$(echo $r | cut -d',' -f2);   fsid=$(echo $r | cut -d',' -f3 | tr -d '[:space:]');   psql -U omero -d idr -h $DBHOST -f "$IDRID/$fsid.sql";   omero mkngff symlink /data/OMERO/ManagedRepository $fsid "/idr0011/zarr/$platename" --bfoptions; done
UPDATE 48
BEGIN
 mkngff_fileset
----------------
        5289232
(1 row)

COMMIT
Using session for demo@localhost:4064. Idle timeout: 10 min. Current group: Public
Checking for prefix_dir /data/OMERO/ManagedRepository/demo_2/2016-07/28/18-37-49.702
Creating dir at /data/OMERO/ManagedRepository/demo_2/2016-07/28/18-37-49.702_mkngff
Creating symlink /data/OMERO/ManagedRepository/demo_2/2016-07/28/18-37-49.702_mkngff/Plate15-Yellow-A.ome.zarr -> /idr0011/zarr/Plate15-Yellow-A.ome.zarr
Checking for prefix_dir /data/OMERO/ManagedRepository/demo_2/2016-07/28/18-37-49.702
write bfoptions to: /data/OMERO/ManagedRepository/demo_2/2016-07/28/18-37-49.702_mkngff/Plate15-Yellow-A.ome.zarr.bfoptions
UPDATE 48
BEGIN
 mkngff_fileset 
----------------
        5289233
(1 row)

COMMIT
Using session for demo@localhost:4064. Idle timeout: 10 min. Current group: Public
Checking for prefix_dir /data/OMERO/ManagedRepository/demo_2/2016-07/28/18-22-12.907
Creating dir at /data/OMERO/ManagedRepository/demo_2/2016-07/28/18-22-12.907_mkngff
Creating symlink /data/OMERO/ManagedRepository/demo_2/2016-07/28/18-22-12.907_mkngff/Plate13-Red-B.ome.zarr -> /idr0011/zarr/Plate13-Red-B.ome.zarr
Checking for prefix_dir /data/OMERO/ManagedRepository/demo_2/2016-07/28/18-22-12.907
write bfoptions to: /data/OMERO/ManagedRepository/demo_2/2016-07/28/18-22-12.907_mkngff/Plate13-Red-B.ome.zarr.bfoptions
UPDATE 48
BEGIN
 mkngff_fileset 
----------------
        5289234
(1 row)

COMMIT
Using session for demo@localhost:4064. Idle timeout: 10 min. Current group: Public
Checking for prefix_dir /data/OMERO/ManagedRepository/demo_2/2016-07/28/16-42-19.788
Creating dir at /data/OMERO/ManagedRepository/demo_2/2016-07/28/16-42-19.788_mkngff
Creating symlink /data/OMERO/ManagedRepository/demo_2/2016-07/28/16-42-19.788_mkngff/Plate3-Yellow-B.ome.zarr -> /idr0011/zarr/Plate3-Yellow-B.ome.zarr
Checking for prefix_dir /data/OMERO/ManagedRepository/demo_2/2016-07/28/16-42-19.788
write bfoptions to: /data/OMERO/ManagedRepository/demo_2/2016-07/28/16-42-19.788_mkngff/Plate3-Yellow-B.ome.zarr.bfoptions
UPDATE 48
BEGIN
 mkngff_fileset 
----------------
        5289235
(1 row)

COMMIT
Using session for demo@localhost:4064. Idle timeout: 10 min. Current group: Public
Checking for prefix_dir /data/OMERO/ManagedRepository/demo_2/2016-07/28/18-07-38.354
Creating dir at /data/OMERO/ManagedRepository/demo_2/2016-07/28/18-07-38.354_mkngff
Creating symlink /data/OMERO/ManagedRepository/demo_2/2016-07/28/18-07-38.354_mkngff/Plate11-Blue-A.ome.zarr -> /idr0011/zarr/Plate11-Blue-A.ome.zarr
Checking for prefix_dir /data/OMERO/ManagedRepository/demo_2/2016-07/28/18-07-38.354
write bfoptions to: /data/OMERO/ManagedRepository/demo_2/2016-07/28/18-07-38.354_mkngff/Plate11-Blue-A.ome.zarr.bfoptions

[will-moore]

Check all images from 4 plates:

for i in 5534 5551 5542 5489; do
  python check_pixels.py Plate:$i --max-planes=sizeC >> /tmp/check_pix_20240104_idr0011.log;
done

(base) bash-4.2$ grep "Check Image" /tmp/check_pix_20240104_idr0011.log | wc
    192    1536   11576
(base) bash-4.2$ grep "Error" /tmp/check_pix_20240104_idr0011.log | wc
      0       0       0

[will-moore]

On idr-testing, test newly available on BIA...

idr0011.csv Fileset IDs from idr-testing...

idr0011/Plate15-Yellow-A.ome.zarr,S-BIAD866/684211eb-77bc-4693-a904-851c79f19b28,6313184
idr0011/Plate13-Red-B.ome.zarr,S-BIAD866/a3781c5d-c4e2-4ad6-9d57-357069721845,6313214
idr0011/Plate3-Yellow-B.ome.zarr,S-BIAD866/311935f7-f3d5-4405-9456-d8c49f736f11,6313274
idr0011/Plate11-Blue-A.ome.zarr,S-BIAD866/e138116a-2215-4581-928e-73c32d667018,6313304
idr0011/Target-TS,S-BIAD866/ef7df8ac-0c98-47a6-944e-6e86f18f174a,6313354

export IDRID=idr0011
for r in $(cat $IDRID.csv); do
  biapath=$(echo $r | cut -d',' -f2)
  uuid=$(echo $biapath | cut -d'/' -f2)
  fsid=$(echo $r | cut -d',' -f3 | tr -d '[:space:]')
  omero mkngff sql $fsid --fs_suffix=None --clientpath="https://uk1s3.embassy.ebi.ac.uk/bia-integrator-data/$biapath/$uuid.zarr" "/bia-integrator-data/$biapath/$uuid.zarr" > "$IDRID/$fsid.sql"
done

for i in $(ls); do sed -i 's/SECRETUUID/f464e059-16b5-4013-b9a2-417e5976371c/g' $i; done

for r in $(cat $IDRID.csv); do
  biapath=$(echo $r | cut -d',' -f2)
  uuid=$(echo $biapath | cut -d'/' -f2)
  fsid=$(echo $r | cut -d',' -f3 | tr -d '[:space:]')
  psql -U omero -d idr -h $DBHOST -f "$IDRID/$fsid.sql"
  omero mkngff symlink /data/OMERO/ManagedRepository $fsid "/bia-integrator-data/$biapath/$uuid.zarr" --fs_suffix=None --bfoptions
done

[will-moore]

Manually updated idr0011 sql scripts for 5 plates to work with original Fileset IDs: https://github.com/IDR/mkngff_upgrade_scripts/commit/e48a80b6065271b7a0b105ff067b0ecefb349d06