search

mcib3d / tapas-ext

Tapas for external image processing libraries
1 stars 1 forks source link

CLIJ support #1

Closed haesleinhuepf closed 4 years ago

haesleinhuepf commented 4 years ago

Hi Thomas @mcib3d ,

I just saw this repository popping up. ❤️ Actually, bridging CLIJ to TAPAS was on my todo list for some time and I never managed to dig into this. Thus, here comes an offer: If you need support or want to outsource some tasks, feel free to let me know. :-)

Cheers, Robert

mcib3d commented 4 years ago

Hi Robert @haesleinhuepf ,

Thanks for your message and sorry I did not contact you earlier. Actually I wanted to validate the proof of concept so I can convince you ;-) . Thanks for supporting the idea, I will be happy if we can work together on this. As a first help, can I ask you to write one module to see how it goes and get feedback from a external developer.

Best,

Thomas

haesleinhuepf commented 4 years ago

Hey Thomas @mcib3d ,

awesome! I'm happy to provide a module. Starting from your work, it might make sense to turn a label image resulting from connected components into a parametric image, e.g. showing the size of detected objects in grey values. I almost managed to make it a module. https://github.com/haesleinhuepf/tapas-ext/blob/master/src/main/java/mcib3d/tapas/ext/CLIJ/plugins/ParametricImageProcess.java

Unfortunately, I cannot test it. I followed the installation instructions, but I don't have any Omero server around. I also copied the tapas-clij.txt to the Fiji dir and renamed it to tapas.txt. I adapted the example processing file.

I come so far: image

image

And receive this error shown below. Unfortunately, the error line doesn't match with the source code on github. I guess in that line, a NullPointerException cannot happen: https://github.com/mcib3d/tapas-core/blob/1e89e0d7ffcc20ea23c874d105741712072fcd17/src/main/java/mcib3d/tapas/core/TapasBatchProcess.java#L544

Furthermore, I'm new to TAPAS. Am I using it in the right way? Does the CLIJ_test_local_file.txt look ok?

Thanks!

Cheers, Robert

Exception in thread "AWT-EventQueue-0" java.lang.NullPointerException
    at mcib3d.tapas.core.TapasBatchProcess.readProcessings(TapasBatchProcess.java:544)
    at mcib3d.tapas.core.TapasBatchProcess.readProcessing(TapasBatchProcess.java:567)
    at mcib3d.tapas.core.TapasBatchProcess.init(TapasBatchProcess.java:251)
    at mcib3d.tapas.plugins.TAPAS_FILES.processing(TAPAS_FILES.java:110)
    at mcib3d.tapas.plugins.TAPAS_FILES.lambda$new$4(TAPAS_FILES.java:65)
    at javax.swing.AbstractButton.fireActionPerformed(AbstractButton.java:2022)
    at javax.swing.AbstractButton$Handler.actionPerformed(AbstractButton.java:2348)
    at javax.swing.DefaultButtonModel.fireActionPerformed(DefaultButtonModel.java:402)
    at javax.swing.DefaultButtonModel.setPressed(DefaultButtonModel.java:259)
    at javax.swing.plaf.basic.BasicButtonListener.mouseReleased(BasicButtonListener.java:252)
    at java.awt.Component.processMouseEvent(Component.java:6539)
    at javax.swing.JComponent.processMouseEvent(JComponent.java:3324)
    at java.awt.Component.processEvent(Component.java:6304)
    at java.awt.Container.processEvent(Container.java:2239)
    at java.awt.Component.dispatchEventImpl(Component.java:4889)
    at java.awt.Container.dispatchEventImpl(Container.java:2297)
    at java.awt.Component.dispatchEvent(Component.java:4711)
    at java.awt.LightweightDispatcher.retargetMouseEvent(Container.java:4904)
    at java.awt.LightweightDispatcher.processMouseEvent(Container.java:4535)
    at java.awt.LightweightDispatcher.dispatchEvent(Container.java:4476)
    at java.awt.Container.dispatchEventImpl(Container.java:2283)
    at java.awt.Window.dispatchEventImpl(Window.java:2746)
    at java.awt.Component.dispatchEvent(Component.java:4711)
    at java.awt.EventQueue.dispatchEventImpl(EventQueue.java:760)
    at java.awt.EventQueue.access$500(EventQueue.java:97)
    at java.awt.EventQueue$3.run(EventQueue.java:709)
    at java.awt.EventQueue$3.run(EventQueue.java:703)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:74)
    at java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:84)
    at java.awt.EventQueue$4.run(EventQueue.java:733)
    at java.awt.EventQueue$4.run(EventQueue.java:731)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:74)
    at java.awt.EventQueue.dispatchEvent(EventQueue.java:730)
    at java.awt.EventDispatchThread.pumpOneEventForFilters(EventDispatchThread.java:205)
    at java.awt.EventDispatchThread.pumpEventsForFilter(EventDispatchThread.java:116)
    at java.awt.EventDispatchThread.pumpEventsForHierarchy(EventDispatchThread.java:105)
    at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:101)
    at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:93)
    at java.awt.EventDispatchThread.run(EventDispatchThread.java:82)
Exception in thread "AWT-EventQueue-0" java.lang.NullPointerException
    at java.io.File.<init>(File.java:277)
    at mcib3d.tapas.plugins.TAPAS_FILES.browseRoot(TAPAS_FILES.java:211)
    at mcib3d.tapas.plugins.TAPAS_FILES.lambda$new$0(TAPAS_FILES.java:59)
    at javax.swing.AbstractButton.fireActionPerformed(AbstractButton.java:2022)
    at javax.swing.AbstractButton$Handler.actionPerformed(AbstractButton.java:2348)
    at javax.swing.DefaultButtonModel.fireActionPerformed(DefaultButtonModel.java:402)
    at javax.swing.DefaultButtonModel.setPressed(DefaultButtonModel.java:259)
    at javax.swing.plaf.basic.BasicButtonListener.mouseReleased(BasicButtonListener.java:252)
    at java.awt.Component.processMouseEvent(Component.java:6539)
    at javax.swing.JComponent.processMouseEvent(JComponent.java:3324)
    at java.awt.Component.processEvent(Component.java:6304)
    at java.awt.Container.processEvent(Container.java:2239)
    at java.awt.Component.dispatchEventImpl(Component.java:4889)
    at java.awt.Container.dispatchEventImpl(Container.java:2297)
    at java.awt.Component.dispatchEvent(Component.java:4711)
    at java.awt.LightweightDispatcher.retargetMouseEvent(Container.java:4904)
    at java.awt.LightweightDispatcher.processMouseEvent(Container.java:4535)
    at java.awt.LightweightDispatcher.dispatchEvent(Container.java:4476)
    at java.awt.Container.dispatchEventImpl(Container.java:2283)
    at java.awt.Window.dispatchEventImpl(Window.java:2746)
    at java.awt.Component.dispatchEvent(Component.java:4711)
    at java.awt.EventQueue.dispatchEventImpl(EventQueue.java:760)
    at java.awt.EventQueue.access$500(EventQueue.java:97)
    at java.awt.EventQueue$3.run(EventQueue.java:709)
    at java.awt.EventQueue$3.run(EventQueue.java:703)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:74)
    at java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:84)
    at java.awt.EventQueue$4.run(EventQueue.java:733)
    at java.awt.EventQueue$4.run(EventQueue.java:731)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:74)
    at java.awt.EventQueue.dispatchEvent(EventQueue.java:730)
    at java.awt.EventDispatchThread.pumpOneEventForFilters(EventDispatchThread.java:205)
    at java.awt.EventDispatchThread.pumpEventsForFilter(EventDispatchThread.java:116)
    at java.awt.EventDispatchThread.pumpEventsForHierarchy(EventDispatchThread.java:105)
    at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:101)
    at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:93)
    at java.awt.EventDispatchThread.run(EventDispatchThread.java:82)
Exception in thread "AWT-EventQueue-0" java.lang.NullPointerException
    at mcib3d.tapas.core.TapasBatchProcess.readProcessings(TapasBatchProcess.java:544)
    at mcib3d.tapas.core.TapasBatchProcess.readProcessing(TapasBatchProcess.java:567)
    at mcib3d.tapas.core.TapasBatchProcess.init(TapasBatchProcess.java:251)
    at mcib3d.tapas.plugins.TAPAS_FILES.processing(TAPAS_FILES.java:110)
    at mcib3d.tapas.plugins.TAPAS_FILES.lambda$new$4(TAPAS_FILES.java:65)
    at javax.swing.AbstractButton.fireActionPerformed(AbstractButton.java:2022)
    at javax.swing.AbstractButton$Handler.actionPerformed(AbstractButton.java:2348)
    at javax.swing.DefaultButtonModel.fireActionPerformed(DefaultButtonModel.java:402)
    at javax.swing.DefaultButtonModel.setPressed(DefaultButtonModel.java:259)
    at javax.swing.plaf.basic.BasicButtonListener.mouseReleased(BasicButtonListener.java:252)
    at java.awt.Component.processMouseEvent(Component.java:6539)
    at javax.swing.JComponent.processMouseEvent(JComponent.java:3324)
    at java.awt.Component.processEvent(Component.java:6304)
    at java.awt.Container.processEvent(Container.java:2239)
    at java.awt.Component.dispatchEventImpl(Component.java:4889)
    at java.awt.Container.dispatchEventImpl(Container.java:2297)
    at java.awt.Component.dispatchEvent(Component.java:4711)
    at java.awt.LightweightDispatcher.retargetMouseEvent(Container.java:4904)
    at java.awt.LightweightDispatcher.processMouseEvent(Container.java:4535)
    at java.awt.LightweightDispatcher.dispatchEvent(Container.java:4476)
    at java.awt.Container.dispatchEventImpl(Container.java:2283)
    at java.awt.Window.dispatchEventImpl(Window.java:2746)
    at java.awt.Component.dispatchEvent(Component.java:4711)
    at java.awt.EventQueue.dispatchEventImpl(EventQueue.java:760)
    at java.awt.EventQueue.access$500(EventQueue.java:97)
    at java.awt.EventQueue$3.run(EventQueue.java:709)
    at java.awt.EventQueue$3.run(EventQueue.java:703)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:74)
    at java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:84)
    at java.awt.EventQueue$4.run(EventQueue.java:733)
    at java.awt.EventQueue$4.run(EventQueue.java:731)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:74)
    at java.awt.EventQueue.dispatchEvent(EventQueue.java:730)
    at java.awt.EventDispatchThread.pumpOneEventForFilters(EventDispatchThread.java:205)
    at java.awt.EventDispatchThread.pumpEventsForFilter(EventDispatchThread.java:116)
    at java.awt.EventDispatchThread.pumpEventsForHierarchy(EventDispatchThread.java:105)
    at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:101)
    at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:93)
    at java.awt.EventDispatchThread.run(EventDispatchThread.java:82)
mcib3d commented 4 years ago

Hi Robert @haesleinhuepf ,

I suspect there is a pb with the load Module, I think for windows we need like C:\ or C:// , not a big fan of windows anyway ;) . Would you minf trying with process:CLIJ_input and CLIJ_output instead ?

Load and save modules should be used for temporary file only, like opening the raw image, save it, then threshold and label it, then use this saved image as parameter for an intensity measurement process. Hope this makes sense.

Thomas

haesleinhuepf commented 4 years ago

Good hint, but I'm not sure if this is related to loading files. Furthermore, I cannot change the paths in these text fields, they are read-only: image

Can you trace down, what happens in this line 544 mentioned in the error message?

I changed my prcess text file to this:

// need to specify the processor
processor:mcib3d.tapas.ext.CLIJ.core.TapasProcessorCLIJ
// first process should be input 
// to read image from OMERO 
// or from file 
process:CLIJ_input
channel:1

// 2D/3D filters with CLIJ
process:CLIJ_filters
radxy:2
radz:1
filter:median

// Auto Thresholding with CLIJ
process:CLIJ_autoThreshold
method:otsu

// Labelling diamond from CLIJ
process:CLIJ_label

// Draw parametric image
process:CLIJ_parametricImage
measurement:PIXEL_COUNT

process:CLIJ_output
name:?name?-clijtest

The log windows says

Checking tapas file C:\programs\FIJI-W~1\Fiji.app\tapas.txt
Found root project : C:\structure\data\
Selected project : C:\structure\data\tapas
Selected dataset : C:\structure\data\tapas\dataset1
Reading tapas file C:\programs\FIJI-W~1\Fiji.app\tapas.txt
Found plugin CLIJ_filters mcib3d.tapas.ext.CLIJ.plugins.FilterProcess
Found plugin CLIJ_autoThreshold mcib3d.tapas.ext.CLIJ.plugins.AutoThresholdProcess
Found plugin CLIJ_label mcib3d.tapas.ext.CLIJ.plugins.LabellingProcess
Found plugin CLIJ_load mcib3d.tapas.ext.CLIJ.plugins.LoadProcess
Found plugin CLIJ_save mcib3d.tapas.ext.CLIJ.plugins.SaveProcess
Found plugin CLIJ_input mcib3d.tapas.ext.CLIJ.plugins.InputProcess
Found plugin CLIJ_output mcib3d.tapas.ext.CLIJ.plugins.OutputProcess
Found plugin CLIJ_parametricImage mcib3d.tapas.ext.CLIJ.plugins.ParametricImageProcess

And the error is 

Exception in thread "AWT-EventQueue-0" java.lang.NullPointerException
    at mcib3d.tapas.core.TapasBatchProcess.readProcessings(TapasBatchProcess.java:544)
    at mcib3d.tapas.core.TapasBatchProcess.readProcessing(TapasBatchProcess.java:567)
    at mcib3d.tapas.core.TapasBatchProcess.init(TapasBatchProcess.java:251)
    at mcib3d.tapas.plugins.TAPAS_FILES.processing(TAPAS_FILES.java:110)
    at mcib3d.tapas.plugins.TAPAS_FILES.lambda$new$4(TAPAS_FILES.java:65)
    at javax.swing.AbstractButton.fireActionPerformed(AbstractButton.java:2022)
    at javax.swing.AbstractButton$Handler.actionPerformed(AbstractButton.java:2348)
    at javax.swing.DefaultButtonModel.fireActionPerformed(DefaultButtonModel.java:402)
    at javax.swing.DefaultButtonModel.setPressed(DefaultButtonModel.java:259)
    at javax.swing.plaf.basic.BasicButtonListener.mouseReleased(BasicButtonListener.java:252)
    at java.awt.Component.processMouseEvent(Component.java:6539)
    at javax.swing.JComponent.processMouseEvent(JComponent.java:3324)
    at java.awt.Component.processEvent(Component.java:6304)
    at java.awt.Container.processEvent(Container.java:2239)
    at java.awt.Component.dispatchEventImpl(Component.java:4889)
    at java.awt.Container.dispatchEventImpl(Container.java:2297)
    at java.awt.Component.dispatchEvent(Component.java:4711)
    at java.awt.LightweightDispatcher.retargetMouseEvent(Container.java:4904)
    at java.awt.LightweightDispatcher.processMouseEvent(Container.java:4535)
    at java.awt.LightweightDispatcher.dispatchEvent(Container.java:4476)
    at java.awt.Container.dispatchEventImpl(Container.java:2283)
    at java.awt.Window.dispatchEventImpl(Window.java:2746)
    at java.awt.Component.dispatchEvent(Component.java:4711)
    at java.awt.EventQueue.dispatchEventImpl(EventQueue.java:760)
    at java.awt.EventQueue.access$500(EventQueue.java:97)
    at java.awt.EventQueue$3.run(EventQueue.java:709)
    at java.awt.EventQueue$3.run(EventQueue.java:703)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:74)
    at java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:84)
    at java.awt.EventQueue$4.run(EventQueue.java:733)
    at java.awt.EventQueue$4.run(EventQueue.java:731)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:74)
    at java.awt.EventQueue.dispatchEvent(EventQueue.java:730)
    at java.awt.EventDispatchThread.pumpOneEventForFilters(EventDispatchThread.java:205)
    at java.awt.EventDispatchThread.pumpEventsForFilter(EventDispatchThread.java:116)
    at java.awt.EventDispatchThread.pumpEventsForHierarchy(EventDispatchThread.java:105)
    at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:101)
    at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:93)
    at java.awt.EventDispatchThread.run(EventDispatchThread.java:82)
mcib3d commented 4 years ago

Hi Robert,

I do not know what is the pb, maybe wrong version. It should be fixed if you use version 0.6.2 of tapas-core.

I tested your new module parametric, the result image is all 0, maybe because my objects are big (>65 535 voxels).

One measurement that would definitively benefit from GPU processing is Feret :-) .

Thomas

haesleinhuepf commented 4 years ago

Hi Thomas,

it's becoming more complicated. I'm afraid there is a version clash. If I combine tapas-core 0.6.2 from jitpack.io with tapas-plugins 0.5 and my tapas-ext, I receivd this error:

(Fiji Is Just) ImageJ 2.0.0-rc-72/1.52r; Java 1.8.0_172 [64-bit]; Windows 10 10.0; 254MB of 36438MB (<1%)

java.lang.NullPointerException at mcib3d.tapas.plugins.TAPAS_FILES.(TAPAS_FILES.java:43) at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at java.lang.Class.newInstance(Class.java:442) at ij.IJ.runUserPlugIn(IJ.java:229) at ij.IJ.runPlugIn(IJ.java:194) at ij.Executer.runCommand(Executer.java:148) at ij.Executer.run(Executer.java:66) at java.lang.Thread.run(Thread.java:748)

A more general question: If I mvn install tapas-core and tapas-plugins, I think the forms are not packed corectly in the JAR file. How do you build your jar files? Is there special maven command for packing the jars correctly?

Thanks!

Cheers, Robert

haesleinhuepf commented 4 years ago

I tested your new module parametric, the result image is all 0, maybe because my objects are big (>65 535 voxels).

I was able to fix this issue. GenerateParametricImage is internally broken, but there is a replacement for the moment: https://github.com/haesleinhuepf/tapas-ext/commit/f6acd7c7d1dfa55e94dbbea583ae19471581e349#diff-ec147cec1be67f7a7ba39f6dceebe280R62-R63

mcib3d commented 4 years ago

Hi Robert @haesleinhuepf,

Now everything package is up to version 0.6.2, sorry about discrepancy in versions, this is why I needed a beta tester, thanks.

For packaging I use mvn package, but I think some of my pom lacked the packaging:jar option.

Finally I tested with the updated version for parametric, it works :)

Best,

Thomas

haesleinhuepf commented 4 years ago

Hey Thomas @mcib3d ,

Now everything package is up to version 0.6.2, sorry about discrepancy in versions, this is why I needed a beta tester, thanks.

I remember how long it took me until I managed the maven-jungle. I think I got quite some grey hair from that time. If you need any hints or tricks, let me know. For example, having a parent-pom for your projects might help you. Not sure if I would do it for 3 repositories, but now that clij has more than 10, it totally pays off.

Finally I tested with the updated version for parametric, it works :)

Great! I think I will program some more and send you a Pull-Request. :-)

Thanks for your support. These bridge-building projects are sooo useful! And as soon as I get some Omero infrastructure running in my institute, I may come back to you and TAPAS. It looks like the thing I am looking for streamlining workflows.

Cheers, Robert

haesleinhuepf commented 4 years ago

One measurement that would definitively benefit from GPU processing is Feret :-) .

Ah and one more thing: Determining Feret on a parallel device such as GPUs is hard. Not sure if it's efficiently possible at all. I could imaging two surrogat parameters: 1) The maximum distance of any labelled pixel from the labels center statisticsOfBackgroundAndLabelledPixels meausres that. 2) If the labelled objects are touching, you could determine the standard deviation of the distance to neighbors between touching neighbors. Implementing this would be fun, similar to the workflow here. Let me know if we should try this on a specific dataset ;-)

Cheers, Robert