search

idr-contrib / community

Community oriented repository and issue tracker for using the IDR
https://idr.openmicroscopy.org
2 stars 0 forks source link

instance is read-only #27

Open sunyi000 opened 4 years ago

sunyi000 commented 4 years ago

Hi IDR,

I;m trying to download images from IDR but for the below image IDs, I have a 'instance is read-only' error.

Images IDs are 3086615 3086838 3086854 3086717 3086597 3086938 3086592 3086932 These are the one I have so far but there may be more...

Appreciate any help and pointers. Thanks

The full stack trace is as below

omero.ApiUsageException: exception ::omero::ApiUsageException
{
    serverStackTrace = ome.conditions.ApiUsageException: This instance is read-only
        at ome.security.basic.EventHandler.invoke(EventHandler.java:136)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
        at org.springframework.orm.hibernate3.HibernateInterceptor.invoke(HibernateInterceptor.java:119)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
        at org.springframework.transaction.interceptor.TransactionInterceptor$1.proceedWithInvocation(TransactionInterceptor.java:99)
        at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:282)
        at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:96)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
        at ome.tools.hibernate.ProxyCleanupFilter$Interceptor.invoke(ProxyCleanupFilter.java:249)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
        at ome.services.util.ServiceHandler.invoke(ServiceHandler.java:121)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
        at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:213)
        at com.sun.proxy.$Proxy80.doWork(Unknown Source)
        at ome.services.util.Executor$Impl.execute(Executor.java:447)
        at ome.services.util.Executor$Impl.execute(Executor.java:392)
        at ome.services.RenderingBean._resetDefaults(RenderingBean.java:2193)
        at ome.services.RenderingBean.internalReset(RenderingBean.java:713)
        at ome.services.RenderingBean.resetDefaultSettings(RenderingBean.java:677)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:333)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:190)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)
        at ome.services.util.ServiceHandler.invoke(ServiceHandler.java:121)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
        at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:213)
        at com.sun.proxy.$Proxy98.resetDefaultSettings(Unknown Source)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:333)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:190)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)
        at ome.security.basic.BasicSecurityWiring.invoke(BasicSecurityWiring.java:93)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
        at ome.services.blitz.fire.AopContextInitializer.invoke(AopContextInitializer.java:43)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
        at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:213)
        at com.sun.proxy.$Proxy98.resetDefaultSettings(Unknown Source)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at ome.services.blitz.util.IceMethodInvoker.invoke(IceMethodInvoker.java:172)
        at ome.services.throttling.Callback.run(Callback.java:56)
        at ome.services.throttling.InThreadThrottlingStrategy.callInvokerOnRawArgs(InThreadThrottlingStrategy.java:56)
        at ome.services.blitz.impl.AbstractAmdServant.callInvokerOnRawArgs(AbstractAmdServant.java:140)
        at ome.services.blitz.impl.RenderingEngineI.resetDefaultSettings_async(RenderingEngineI.java:380)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:333)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:190)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)
        at omero.cmd.CallContext.invoke(CallContext.java:85)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
        at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:213)
        at com.sun.proxy.$Proxy100.resetDefaultSettings_async(Unknown Source)
        at omero.api._RenderingEngineTie.resetDefaultSettings_async(_RenderingEngineTie.java:316)
        at omero.api._RenderingEngineDisp.___resetDefaultSettings(_RenderingEngineDisp.java:2014)
        at omero.api._RenderingEngineDisp.__dispatch(_RenderingEngineDisp.java:2434)
        at IceInternal.Incoming.invoke(Incoming.java:221)
        at Ice.ConnectionI.invokeAll(ConnectionI.java:2536)
        at Ice.ConnectionI.dispatch(ConnectionI.java:1145)
        at Ice.ConnectionI.message(ConnectionI.java:1056)
        at IceInternal.ThreadPool.run(ThreadPool.java:395)
        at IceInternal.ThreadPool.access$300(ThreadPool.java:12)
        at IceInternal.ThreadPool$EventHandlerThread.run(ThreadPool.java:832)
        at java.lang.Thread.run(Thread.java:748)

    serverExceptionClass = ome.conditions.ApiUsageException
    message = This instance is read-only
manics commented 4 years ago

Hi Could you please show us the code or command you're using to download? Thanks!

sunyi000 commented 4 years ago

im using the idr_download_by_ids.py script in the idr_download galaxy tool and run it like this

cat testid.txt | python idr_download_by_ids.py -c 'Hoechst 33258' -f 0 -z 0 --rectangle 0 0 696 520 --skip-failed

testid.txt has a list of ids and one per line...

for example,

3086615 3086529

3086615 generates the exception.. whereas 3086529 is working, i expectedto get one image downloaded, but I can't get any of them.

Thanks for looking into it.

jburel commented 4 years ago

@sunyi000 @beatrizserrano Will you be free to discuss next Monday at 10am UK during the IDR call? It looks like you have exposed some issues that we need to resolve in order to achieve what you need. I will send an email to @beatrizserrano with video link if you are happy to join

beatrizserrano commented 4 years ago

Sure @jburel! We will join 😃

jburel commented 4 years ago

Closing the issue, call scheduled next week

jburel commented 4 years ago

RE-opening so it is visible when we working on narrowing down the problem

sunyi000 commented 4 years ago

just posting other exceptions that I came across during the download

  1. image id 3103314 1 3103103 3103253 3103205 3103361 3103300 3103334 3103701 3103766 3103817 3103596 3103683 3103560 3103728 3103591 3091422

from browser, z section is 2x1, others are 1x1

omero.ApiUsageException: exception ::omero::ApiUsageException
{
    serverStackTrace = ome.conditions.ApiUsageException: Invalid C index: 1/1
        at loci.formats.FormatTools.getIndex(FormatTools.java:480)
        at loci.formats.FormatTools.getIndex(FormatTools.java:407)
        at loci.formats.ChannelSeparator.getIndex(ChannelSeparator.java:264)
        at loci.formats.ReaderWrapper.getIndex(ReaderWrapper.java:456)
        at ome.io.bioformats.BfPixelsWrapper.getTile(BfPixelsWrapper.java:350)
        at ome.io.bioformats.BfPixelBuffer.getTileDirect(BfPixelBuffer.java:518)
        at ome.services.RawPixelsBean.getTile(RawPixelsBean.java:933)
        at sun.reflect.GeneratedMethodAccessor919.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:333)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:190)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)
        at ome.security.basic.EventHandler.invoke(EventHandler.java:154)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
        at ome.tools.hibernate.SessionHandler.doStateful(SessionHandler.java:216)
        at ome.tools.hibernate.SessionHandler.invoke(SessionHandler.java:200)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
        at org.springframework.transaction.interceptor.TransactionInterceptor$1.proceedWithInvocation(TransactionInterceptor.java:99)
        at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:282)
        at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:96)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
        at ome.tools.hibernate.ProxyCleanupFilter$Interceptor.invoke(ProxyCleanupFilter.java:249)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
        at ome.services.util.ServiceHandler.invoke(ServiceHandler.java:121)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
        at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:213)
        at com.sun.proxy.$Proxy101.getTile(Unknown Source)
        at sun.reflect.GeneratedMethodAccessor919.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:333)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:190)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)
        at ome.security.basic.BasicSecurityWiring.invoke(BasicSecurityWiring.java:93)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
        at ome.services.blitz.fire.AopContextInitializer.invoke(AopContextInitializer.java:43)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
        at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:213)
        at com.sun.proxy.$Proxy101.getTile(Unknown Source)
        at sun.reflect.GeneratedMethodAccessor999.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at ome.services.blitz.util.IceMethodInvoker.invoke(IceMethodInvoker.java:172)
        at ome.services.throttling.Callback.run(Callback.java:56)
        at ome.services.throttling.InThreadThrottlingStrategy.callInvokerOnRawArgs(InThreadThrottlingStrategy.java:56)
        at ome.services.blitz.impl.AbstractAmdServant.callInvokerOnRawArgs(AbstractAmdServant.java:140)
        at ome.services.blitz.impl.RawPixelsStoreI.getTile_async(RawPixelsStoreI.java:279)
        at sun.reflect.GeneratedMethodAccessor998.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:333)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:190)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)
        at omero.cmd.CallContext.invoke(CallContext.java:85)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
        at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:213)
        at com.sun.proxy.$Proxy102.getTile_async(Unknown Source)
        at omero.api._RawPixelsStoreTie.getTile_async(_RawPixelsStoreTie.java:216)
        at omero.api._RawPixelsStoreDisp.___getTile(_RawPixelsStoreDisp.java:970)
        at omero.api._RawPixelsStoreDisp.__dispatch(_RawPixelsStoreDisp.java:1695)
        at IceInternal.Incoming.invoke(Incoming.java:221)
        at Ice.ConnectionI.invokeAll(ConnectionI.java:2536)
        at Ice.ConnectionI.dispatch(ConnectionI.java:1145)
        at Ice.ConnectionI.message(ConnectionI.java:1056)
        at IceInternal.ThreadPool.run(ThreadPool.java:395)
        at IceInternal.ThreadPool.access$300(ThreadPool.java:12)
        at IceInternal.ThreadPool$EventHandlerThread.run(ThreadPool.java:832)
        at java.lang.Thread.run(Thread.java:748)

    serverExceptionClass = ome.conditions.ApiUsageException
    message = Invalid C index: 1/1
}
  1. Image id: 566631 566327
WARNING:omero.gateway:ResourceError on <class 'omero.gateway.OmeroGatewaySafeCallWrapper'> to <ddb3d27d-c9f9-496b-86c8-7aabd5018209omero.api.RawPixelsStore> getTile((0, 1, 0, 0, 0, 1392, 1040), {})
Traceback (most recent call last):
  File "/home/ysun/miniconda2/envs/idr/lib/python3.8/site-packages/omero/gateway/__init__.py", line 4746, in __call__
    return self.f(*args, **kwargs)
  File "/home/ysun/miniconda2/envs/idr/lib/python3.8/site-packages/omero_api_RawPixelsStore_ice.py", line 998, in getTile
    return _M_omero.api.RawPixelsStore._op_getTile.invoke(self, ((z, c, t, x, y, w, h), _ctx))
omero.ResourceError: exception ::omero::ResourceError
{
    serverStackTrace = ome.conditions.ResourceError: /data/OMERO/Pixels/Dir-566/566631 (No such file or directory)
        at ome.services.RawPixelsBean.handleException(RawPixelsBean.java:833)
        at ome.services.RawPixelsBean.getTile(RawPixelsBean.java:935)
        at sun.reflect.GeneratedMethodAccessor911.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:333)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:190)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)
        at ome.security.basic.EventHandler.invoke(EventHandler.java:154)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
        at ome.tools.hibernate.SessionHandler.doStateful(SessionHandler.java:216)
        at ome.tools.hibernate.SessionHandler.invoke(SessionHandler.java:200)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
        at org.springframework.transaction.interceptor.TransactionInterceptor$1.proceedWithInvocation(TransactionInterceptor.java:99)
        at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:282)
        at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:96)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
        at ome.tools.hibernate.ProxyCleanupFilter$Interceptor.invoke(ProxyCleanupFilter.java:249)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
        at ome.services.util.ServiceHandler.invoke(ServiceHandler.java:121)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
        at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:213)
        at com.sun.proxy.$Proxy101.getTile(Unknown Source)
        at sun.reflect.GeneratedMethodAccessor911.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:333)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:190)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)
        at ome.security.basic.BasicSecurityWiring.invoke(BasicSecurityWiring.java:93)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
        at ome.services.blitz.fire.AopContextInitializer.invoke(AopContextInitializer.java:43)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
        at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:213)
        at com.sun.proxy.$Proxy101.getTile(Unknown Source)
        at sun.reflect.GeneratedMethodAccessor996.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at ome.services.blitz.util.IceMethodInvoker.invoke(IceMethodInvoker.java:172)
        at ome.services.throttling.Callback.run(Callback.java:56)
        at ome.services.throttling.InThreadThrottlingStrategy.callInvokerOnRawArgs(InThreadThrottlingStrategy.java:56)
        at ome.services.blitz.impl.AbstractAmdServant.callInvokerOnRawArgs(AbstractAmdServant.java:140)
        at ome.services.blitz.impl.RawPixelsStoreI.getTile_async(RawPixelsStoreI.java:279)
        at sun.reflect.GeneratedMethodAccessor995.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:333)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:190)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)
        at omero.cmd.CallContext.invoke(CallContext.java:85)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
        at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:213)
        at com.sun.proxy.$Proxy102.getTile_async(Unknown Source)
        at omero.api._RawPixelsStoreTie.getTile_async(_RawPixelsStoreTie.java:216)
        at omero.api._RawPixelsStoreDisp.___getTile(_RawPixelsStoreDisp.java:970)
        at omero.api._RawPixelsStoreDisp.__dispatch(_RawPixelsStoreDisp.java:1695)
        at IceInternal.Incoming.invoke(Incoming.java:221)
        at Ice.ConnectionI.invokeAll(ConnectionI.java:2536)
        at Ice.ConnectionI.dispatch(ConnectionI.java:1145)
        at Ice.ConnectionI.message(ConnectionI.java:1056)
        at IceInternal.ThreadPool.run(ThreadPool.java:395)
        at IceInternal.ThreadPool.access$300(ThreadPool.java:12)
        at IceInternal.ThreadPool$EventHandlerThread.run(ThreadPool.java:832)
        at java.lang.Thread.run(Thread.java:748)

    serverExceptionClass = ome.conditions.ResourceError
    message = /data/OMERO/Pixels/Dir-566/566631 (No such file or directory)

  1. db connection issue For the most days of last week, code hangs at conn.connect() with no exception in stdout, but this is working now so not sure if this is related to embassy cloud outage or something else.

  2. Java heap space error.. this only happened to me once..I don;t see it now

I will report again if I have more issues.. Thanks very much

beatrizserrano commented 4 years ago

(2), (3), (4) are new to me, but I've seen (1) before, that's one of the cases in which not much can be done, the channel is missing...

sunyi000 commented 4 years ago

mitoCheck.txt

also, for all the ids in the file attached the same "invalid C index 1/1 exception" occurred, so no images can be downloaded for mitoCheck dataset at the moment

parameters used are: channel: GFP rectangle: 336 256 672 512

tried changing frame value as well , same error happened.

jburel commented 4 years ago

Thanks Checking the first batch of Id i.e. 3086615.. 3086932, Images are coming from http://idr.openmicroscopy.org/webclient/?show=plate-5894. This seems to be the only "broken" plate in the study

jburel commented 4 years ago

See https://github.com/idr-contrib/community/issues/10. The plate might have been corrupted during the transfer. At least the error you noticed has an explanation

jburel commented 4 years ago

https://github.com/idr-contrib/community/issues/27#issuecomment-609749575 I am using the mitocheck file provide Running the following:

cat ~/Downloads/mitoCheck.txt | python idr_download_by_ids.py -c 'GFP' -f 0 -z 0 --rectangle 336 256 672 512 --skip-failed

so far around 1500 images have been downloaded

sunyi000 commented 4 years ago

Thanks, I will try again.. may be something to do with the library change, I'm running the one using libtiff.. The original matplotlib will download all channels , even if just one channel is specified.

we tried open the image with Fiji, and it shows 4 channels.( 3 has images, 1 is blank)

I will see what else I can find

jburel commented 4 years ago

I have made few adjustments to the code. I will open a PR against your branch I installed libtiff from pypi in my conda. The version I found on conda was not compatible with Python 3.

sunyi000 commented 4 years ago

I have made few adjustments to the code. I will open a PR against your branch I installed libtiff from pypi in my conda. The version I found on conda was not compatible with Python 3.

yes I did the same, install libtiff from pypi in python-omero conda env. it has name conflict with the C version of libtiff. Bjoern created a recipe of pylibtiff a week ago, but not sure if it is already available in conda

jburel commented 4 years ago

I cannot see you in the GH UI to open a PR against your repo, the branch I have been looking at things https://github.com/jburel/tools-iuc/tree/idr-download

sunyi000 commented 4 years ago

thanks, my branch is here https://github.com/sunyi000/tools-iuc/tree/idr-download/tools/idr_download

jburel commented 4 years ago

I have worked on top of your branch. The problem is that I cannot open a PR against it via the UI the user sunyi000 does not show up in the list and when I enter the name nothing gets updated

sunyi000 commented 4 years ago

Thanks very much I have merged the changes, and it started working for mitoCheck now.. I will report back if I found more

sunyi000 commented 4 years ago

@jburel I'm trying to download these images in the attached file. bioactiveCompound.txt

exceptions occurred after the 5144th image is download. Is this the file handler issue we talked about before ? Thanks for any advice..

jburel commented 4 years ago

What are the exceptions?

sunyi000 commented 4 years ago

What are the exceptions?

@jburel sorry I replied in wrong place.. got too many PRs open at the same time... https://github.com/galaxyproject/tools-iuc/pull/2938#issuecomment-611422987

It's already reported as @beatrizserrano mentioned...so am just wondering if it's possible to skip this type of exception in the py code and continue with the rest..

Thanks

beatrizserrano commented 4 years ago

@jburel, is there news on this? Are you migration the IDR data back to the Embassy Cloud?

jburel commented 4 years ago

The upgrade of the Embassy Cloud is now done with a bit of delay @manics is looking at "re-activating" IDR, we should give an update this week hopefully

beatrizserrano commented 4 years ago

Great, thanks! Keep us updated :)

sbesson commented 4 years ago

@beatrizserrano @sunyi000 you might have seen the announcement on image.sc but the IDR instance is now fully migrated back to EBI Embassy.

beatrizserrano commented 4 years ago

great, thanks for the info @sbesson!