NTNDArray decompression improvements

[MarkRivers]

@mrkraimer has written a new NTNDCodec.java which moves the decompression code into a separate class. This is cleaner and more re-usable so it should be used in a future release. It will require some work, and I want to release R1-3 soon so this will wait for R1-4.

[mrkraimer]

There are problem so do not merge branch ntndcodec

[MarkRivers]

I am working today to implement 2 new decompressors, LZ4 and BitshuffleLZ4. I can either do this in the master branch, or in the ntndcodec branch. If I do it in the master branch there will be work to do to move those new functions into the new NTNDCodec class.

What are the problems with the NTNDCodec branch? Should I work on fixing those, or just work on the master branch?

[mrkraimer]

I just merged my changes. blosc should now work. I upgraded to feroda 29 and now the code for jpeg can not be found.

Go ahead an make your changes on branch ntndcodec. Let me know when You have pushed your changes.

[mrkraimer]

Mark, I see that you have made the changes on the master branch. When you are done please let me know and I will make similar changes to ntndcodec branch. I assume that I also have to get the latest changes in ADCore. Are there any other components of areaDetector that I need to update?

[MarkRivers]

I plan to do the ntndcodec branch too. Probably today or tomorrow. You need the master branch of ADCore and ADSupport.

Mark

Sent from my iPhone

On Dec 9, 2018, at 2:46 PM, Marty Kraimer notifications@github.com<mailto:notifications@github.com> wrote:

Mark, I see that you have made the changes on the master branch. When you are done please let me know and I will make similar changes to ntndcodec branch. I assume that I also have to get the latest changes in ADCore. Are there any other components of areaDetector that I need to update?

- You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://github.com/areaDetector/ADViewers/issues/4#issuecomment-445565542, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AF1QzvtDWHFx0bRtsSngxRuKPfkOWsSuks5u3WiJgaJpZM4Y-zWZ.

[MarkRivers]

I have merged my changes. The changes are:

• Added support for lz4 and bitshuffle/lz4.
• Improved the logic. The large "if else" block for each datatype was previously only for Blosc. Now it works on each of the compressors. Decompression is done before this block for each compression type.

I have a Java question. I tried changing this statement:

        if (scalarType==ScalarType.pvByte) {            

to

    switch(scalarType) {
        case ScalarType.pvByte:

        case ScalarType.pvUByte:

etc.

But the Java compiler complains that these are not enums I am comparing to?

Should we be distributing a .jar file or just the .java files? I just discovered that if I delete all of the .class files in the ImageJ/plugins/EPICS_areaDetector directory, then I only need to do Compile and Run on EPICS_NTNDA_Viewer.java. It automatically finds and compiles all of the other .java files required (myUtil.java, decompressJPEGDll.java, etc.). This is convenient, and means perhaps we don't really need a .jar file, since the user only has to compile a single file.

[MarkRivers]

I merged the ntndcodec branch into master and deleted it on Github.

I tested it on both Linux and Windows.

[mrkraimer]

On 12/9/18 7:15 PM, Mark Rivers wrote:

I have merged my changes. The changes are:

• Added support for lz4 and bitshuffle/lz4.
• Improved the logic. The large "if else" block for each datatype was previously only for Blosc. Now it works on each of the compressors. Decompression is done before this block for each compression type.

OK i got latest and tried it. All works except JPEG. See below

I have a Java question. I tried changing this statement:

|if (scalarType==ScalarType.pvByte) { |

to

|switch(scalarType) { case ScalarType.pvByte: case ScalarType.pvUByte: etc. |

But the Java compiler complains that these are not enums I am comparing to?

The following works:

switch(scalarType) {

case pvByte:

etc

Should we be distributing a .jar file or just the .java files? I just discovered that if I delete all of the .class files in the ImageJ/plugins/EPICS_areaDetector directory, then I only need to do Compile and Run on EPICS_NTNDA_Viewer.java. It automatically finds and compiles all of the other .java files required (myUtil.java, decompressJPEGDll.java, etc.). This is convenient, and means perhaps we don't really need a .jar file, since the user only has to compile a single file.

I have always done this. So there is no need to distribute a jar file for EPICS_NTNDA_Viewer.java

BUT I do have a problem.

I upgraded my development computer to fedora core 29.

Since then the JPEG plugin fails with

java.lang.UnsatisfiedLinkError: Unable to load library 'decompressJPEG': Native library (linux-x86-64/libdecompressJPEG.so) not found in resource path ([file:/home/install/ImageJ/ImageJ/plugins/, file:/home/install/ImageJ/ImageJ/plugins/Analyze/, file:/home/install/ImageJ/ImageJ/plugins/Scripts/, file:/home/install/ImageJ/ImageJ/plugins/Epics/, file:/home/install/ImageJ/ImageJ/plugins/Epics/caj-1.1.10.jar, file:/home/install/ImageJ/ImageJ/plugins/Epics/jblosc-1.0.1.dev.jar, file:/home/install/ImageJ/ImageJ/plugins/Epics/epics-ntypes-0.3.1.jar, file:/home/install/ImageJ/ImageJ/plugins/Epics/jca-2.3.6.jar, file:/home/install/ImageJ/ImageJ/plugins/Epics/epics-pvdata-6.1.1.jar, file:/home/install/ImageJ/ImageJ/plugins/Epics/jna-5.1.0.jar, file:/home/install/ImageJ/ImageJ/plugins/Epics/epics-pvdatabase-4.3.1.jar, file:/home/install/ImageJ/ImageJ/plugins/Epics/epics-pvaclient-4.3.2.jar, file:/home/install/ImageJ/ImageJ/plugins/Epics/epics-pvaccess-5.1.1.jar, file:/home/install/ImageJ/ImageJ/plugins/Epics/NetCDF_.jar, file:/home/install/ImageJ/ImageJ/plugins/Examples/, file:/home/install/ImageJ/ImageJ/plugins/Stacks/, file:/home/install/ImageJ/ImageJ/plugins/Filters/, file:/home/install/ImageJ/ImageJ/plugins/3D/, file:/home/install/ImageJ/ImageJ/plugins/3D/Interactive_3D_Surface_Plot.jar, file:/home/install/ImageJ/ImageJ/plugins/3D/Volume_Viewer.jar, file:/home/install/ImageJ/ImageJ/plugins/Graphics/, file:/home/install/ImageJ/ImageJ/plugins/Color/, file:/home/install/ImageJ/ImageJ/plugins/Input-Output/, file:/home/install/ImageJ/ImageJ/plugins/Input-Output/Exif_Reader.jar, file:/home/install/ImageJ/ImageJ/plugins/Input-Output/nifti_io.jar, file:/home/install/ImageJ/ImageJ/plugins/Input-Output/Deltavision_Opener.jar, file:/home/install/ImageJ/ImageJ/plugins/Input-Output/LSM_Reader.jar, file:/home/install/ImageJ/ImageJ/plugins/Tools/, file:/home/install/ImageJ/ImageJ/plugins/Tools/Bezier_Tool.jar, file:/home/install/ImageJ/ImageJ/plugins/jars/, file:/home/install/ImageJ/ImageJ/plugins/jars/BeanShell.jar, file:/home/install/ImageJ/ImageJ/plugins/jars/Auto_Threshold.jar])     at com.sun.jna.NativeLibrary.loadLibrary(NativeLibrary.java:303)     at com.sun.jna.NativeLibrary.getInstance(NativeLibrary.java:427)     at com.sun.jna.Native.register(Native.java:1736)     at com.sun.jna.Native.register(Native.java:1457)     at decompressJPEGDll.(decompressJPEGDll.java:8)     at NTNDCodec.decompress(NTNDCodec.java:119)     at EPICS_NTNDA_Viewer.updateImage(EPICS_NTNDA_Viewer.java:405)     at EPICS_NTNDA_Viewer.handleEvents(EPICS_NTNDA_Viewer.java:239)     at EPICS_NTNDA_Viewer.run(EPICS_NTNDA_Viewer.java:278)     at ij.IJ.runUserPlugIn(IJ.java:222)     at ij.IJ.runUserPlugIn(IJ.java:235)     at ij.IJ.runPlugIn(IJ.java:186)     at ij.Executer.runCommand(Executer.java:137)     at ij.Executer.run(Executer.java:66)     at java.lang.Thread.run(Thread.java:748)

Both of new plugins you added, i. e. LZ4 and BSLZ4 do work. Strange. Any ideas?

Also I still think that issue #6 about byte order is a problem.

Note that the code says:

decompressInBuffer.order(ByteOrder.nativeOrder()); decompressOutBuffer.order(ByteOrder.nativeOrder());

This means that it says both in and out have the same byte order. If server that compressed has different byte order than client it will not work.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/areaDetector/ADViewers/issues/4#issuecomment-445591634, or mute the thread https://github.com/notifications/unsubscribe-auth/AF1Q1oIZiRNfWbQpKNdN9kX7wDJB-V15ks5u3aeUgaJpZM4Y-zWZ.

[MarkRivers]

I think I see your JPEG problem.

I just ran "ldd" on ADSupport/lib/linux-x86_64/libdecompressJPEG.so. I see the following:

corvette:ADSupport/lib/linux-x86_64>ldd libdecompressJPEG.so
        linux-vdso.so.1 =>  (0x00007ffd659fe000)
        libjpeg.so => /lib64/libjpeg.so (0x00007f951d12c000)
        libpthread.so.0 => /lib64/libpthread.so.0 (0x00007f951cf0f000)
        librt.so.1 => /lib64/librt.so.1 (0x00007f951cd07000)
        libdl.so.2 => /lib64/libdl.so.2 (0x00007f951cb03000)
        libstdc++.so.6 => /lib64/libstdc++.so.6 (0x00007f951c7fb000)
        libm.so.6 => /lib64/libm.so.6 (0x00007f951c4f9000)
        libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00007f951c2e3000)
        libc.so.6 => /lib64/libc.so.6 (0x00007f951bf15000)
        /lib64/ld-linux-x86-64.so.2 (0x000055e2747e3000)

Note that it depends on /lib64/libjpeg.so. That is not correct, because it means you need to have the jpeg devel package installed. It should be using the version in ADSupport, not the system version. I will look into it.

[MarkRivers]

I found that on my system it was finding the system version of libjpeg.so because in that terminal I had not defined LD_LIBRARY_PATH to include ADSupport/lib/linux-x86_64. Once I did that then ldd reported this:

corvette:ADSupport/lib/linux-x86_64>ldd libdecompressJPEG.so
        linux-vdso.so.1 =>  (0x00007fffb593a000)
        libjpeg.so => /home/epics/devel/areaDetector/ADSupport/lib/linux-x86_64/libjpeg.so (0x00007f969c3e0000)
        libpthread.so.0 => /lib64/libpthread.so.0 (0x00007f969c1a7000)
        librt.so.1 => /lib64/librt.so.1 (0x00007f969bf9f000)
        libdl.so.2 => /lib64/libdl.so.2 (0x00007f969bd9b000)
        libstdc++.so.6 => /lib64/libstdc++.so.6 (0x00007f969ba93000)
        libm.so.6 => /lib64/libm.so.6 (0x00007f969b791000)
        libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00007f969b57b000)
        libc.so.6 => /lib64/libc.so.6 (0x00007f969b1ad000)
        /lib64/ld-linux-x86-64.so.2 (0x00005649f0042000)

What do you see when you run ldd on libdecompressJPEG.so?

[MarkRivers]

Regarding the byte order issue, I think we really need to test to be sure.

Both the Blosc compressor and the bitshuffle compressor are passed the number of bytes per element. Each of them puts a header at the start of the compressed block. It is possible that that header contains both the number of bytes per element, and the endianess of the machine that did the compression. The decompressor would then have the information needed to do the byte swapping to the endianess of the decompressing machine. I don't know if either decompressor actually does this, but in principle they might.

For straight lz4 compression the information on bytes per element is not provided to the compressor or decompressor, so I don't see how it could do byte swapping.

The issue is not too pressing, because most of the machines we are using for these large arrays are currently Intel little-endian. But eventually we should get it right, but that means finding big-endian machines to test with.

[mrkraimer]

On 12/10/18 8:21 AM, Mark Rivers wrote:

I found that on my system it was finding the system version of libjpeg.so because in that terminal I had not defined LD_LIBRARY_PATH to include ADSupport/lib/linux-x86_64. Once I did that then ldd reported this:

|corvette:ADSupport/lib/linux-x86_64>ldd libdecompressJPEG.so linux-vdso.so.1 => (0x00007fffb593a000) libjpeg.so => /home/epics/devel/areaDetector/ADSupport/lib/linux-x86_64/libjpeg.so (0x00007f969c3e0000) libpthread.so.0 => /lib64/libpthread.so.0 (0x00007f969c1a7000) librt.so.1 => /lib64/librt.so.1 (0x00007f969bf9f000) libdl.so.2 => /lib64/libdl.so.2 (0x00007f969bd9b000) libstdc++.so.6 => /lib64/libstdc++.so.6 (0x00007f969ba93000) libm.so.6 => /lib64/libm.so.6 (0x00007f969b791000) libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00007f969b57b000) libc.so.6 => /lib64/libc.so.6 (0x00007f969b1ad000) /lib64/ld-linux-x86-64.so.2 (0x00005649f0042000) |

What do you see when you run ldd on libdecompressJPEG.so?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/areaDetector/ADViewers/issues/4#issuecomment-445812676, or mute the thread https://github.com/notifications/unsubscribe-auth/AF1Q1vI4I159ZUh08GGMEW64xanuY765ks5u3l_GgaJpZM4Y-zWZ.

mrk> ldd libdecompressJPEG.so     linux-vdso.so.1 (0x00007ffd1f9fe000)     libjpeg.so => /home/epicsv4/areaDetector/ADSupport/lib/linux-x86_64/libjpeg.so (0x00007fe6a1485000)     libpthread.so.0 => /lib64/libpthread.so.0 (0x00007fe6a1440000)     librt.so.1 => /lib64/librt.so.1 (0x00007fe6a1436000)     libdl.so.2 => /lib64/libdl.so.2 (0x00007fe6a1430000)     libstdc++.so.6 => /lib64/libstdc++.so.6 (0x00007fe6a1298000)     libm.so.6 => /lib64/libm.so.6 (0x00007fe6a1114000)     libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00007fe6a10f7000)     libc.so.6 => /lib64/libc.so.6 (0x00007fe6a0f31000)     /lib64/ld-linux-x86-64.so.2 (0x00007fe6a14d5000) mrk> pwd /home/epicsv4/areaDetector/ADSupport/lib/linux-x86_64 mrk>

[mrkraimer]

On 12/10/18 8:29 AM, Mark Rivers wrote:

Regarding the byte order issue, I think we really need to test to be sure.

Both the Blosc compressor and the bitshuffle compressor are passed the number of bytes per element. Each of them puts a header at the start of the compressed block. It is possible that that header contains both the number of bytes per element, and the endianess of the machine that did the compression. The decompressor would then have the information needed to do the byte swapping to the endianess of the decompressing machine. I don't know if either decompressor actually does this, but in principle they might.

Note that blosc.h says:

  typesize is the number of bytes for the atomic type in binary   src buffer.  This is mainly useful for the shuffle filters.   For implementation reasons, only a 1 < typesize < 256 will allow the   shuffle filter to work.  When typesize is not in this range, shuffle   will be silently disabled.

Thus it does sound like it refers to the element type.

For straight lz4 compression the information on bytes per element is not provided to the compressor or decompressor, so I don't see how it could do byte swapping.

The issue is not too pressing, because most of the machines we are using for these large arrays are currently Intel little-endian. But eventually we should get it right, but that means finding big-endian machines to test with.

Let me spend some time investigating this. I will let you know what I find. I think I can test by "lying" to byteBuffer. But the final test must be where we can find a big-endian machine.

Marty

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/areaDetector/ADViewers/issues/4#issuecomment-445815278, or mute the thread https://github.com/notifications/unsubscribe-auth/AF1Q1huPL2EnQlodWa_eatz41mDK8xJEks5u3mHJgaJpZM4Y-zWZ.

[MarkRivers]

For the JPEG problem try temporarily coping libdecompressJPEG.so into ImageJ/plugins/EPICS_areaDetector.

When I was having a problem I found that when I did that I got a better diagnostic error message about the real problem.

Be sure to check both the ImageJ log window and your Linux shell Window that launched ImageJ for error messages.

[mrkraimer]

On 12/10/18 8:21 AM, Mark Rivers wrote:

I found that on my system it was finding the system version of libjpeg.so because in that terminal I had not defined LD_LIBRARY_PATH to include ADSupport/lib/linux-x86_64. Once I did that then ldd reported this:

|corvette:ADSupport/lib/linux-x86_64>ldd libdecompressJPEG.so linux-vdso.so.1 => (0x00007fffb593a000) libjpeg.so => /home/epics/devel/areaDetector/ADSupport/lib/linux-x86_64/libjpeg.so (0x00007f969c3e0000) libpthread.so.0 => /lib64/libpthread.so.0 (0x00007f969c1a7000) librt.so.1 => /lib64/librt.so.1 (0x00007f969bf9f000) libdl.so.2 => /lib64/libdl.so.2 (0x00007f969bd9b000) libstdc++.so.6 => /lib64/libstdc++.so.6 (0x00007f969ba93000) libm.so.6 => /lib64/libm.so.6 (0x00007f969b791000) libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00007f969b57b000) libc.so.6 => /lib64/libc.so.6 (0x00007f969b1ad000) /lib64/ld-linux-x86-64.so.2 (0x00005649f0042000) |

What do you see when you run ldd on libdecompressJPEG.so?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/areaDetector/ADViewers/issues/4#issuecomment-445812676, or mute the thread https://github.com/notifications/unsubscribe-auth/AF1Q1vI4I159ZUh08GGMEW64xanuY765ks5u3l_GgaJpZM4Y-zWZ.

    I also had not set my LD_LIBRARY_PATH.

It is now

LD_LIBRARY_PATH=/home/install/qwt/qwt-6.1.3/lib:/home/epicsv4/areaDetector/ADSupport/lib/linux-x86_64

Now JPEG works

[MarkRivers]

If you had not set LD_LIBRARY_PATH then how did ImageJ find the libraries for bitshuffle and Blosc? Perhaps those come with the new Fedora, and so it was using the system versions?

[mrkraimer]

On 12/10/18 10:48 AM, Mark Rivers wrote:

If you had not set LD_LIBRARY_PATH then how did ImageJ find the libraries for bitshuffle and Blosc? Perhaps those come with the new Fedora, and so it was using the system versions?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/areaDetector/ADViewers/issues/4#issuecomment-445863026, or mute the thread https://github.com/notifications/unsubscribe-auth/AF1Q1gqeSpSuOnowVHIFj4a6Bcp7prx7ks5u3oI2gaJpZM4Y-zWZ.

I looked harder.

For lz4 and bslz4 I see errors like on attached screen.

Thus viewer never tried these.

I did a

pull make disclean make

in all of

ADCore ADSupport ADSimDetector

What am I doing wrong?

[MarkRivers]

Hi Marty, There was no attachment, so I don't know what the problem was. For Github messages you need to log in, and use the Issue to reply. You can then drag or paste images into the message.

[MarkRivers]

I did a commit in NTNDCodec.java at 21:23 yesterday. Did you get that? It was causing bitshuffle and lz4 not to work. The problem was with the variable numElements, which was causing confusion. It was being used to mean both "the number of elements in imagedata", and "the size of imagedata in bytes", which are not the same for non-byte data. I removed the variable completely since it's simpler.

[mrkraimer]

On 12/10/18 6:56 PM, Mark Rivers wrote:

I did a commit in NTNDCodec.java at 21:23 yesterday. Did you get that? It was causing bitshuffle and lz4 not to work. The problem was with the variable numElements, which was causing confusion. It was being used to mean both "the number of elements in imagedata", and "the size of imagedata in bytes", which are not the same for non-byte data. I removed the variable completely since it's simpler.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/areaDetector/ADViewers/issues/4#issuecomment-446020455, or mute the thread https://github.com/notifications/unsubscribe-auth/AF1Q1qDibnvarn5xFjbuilvZm83a5iZ8ks5u3vSigaJpZM4Y-zWZ.

That was the problem!! All works now. But an interesting observation. I first tried without setting export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/home/epicsv4/areaDetector/ADSupport/lib/linux-x86_64

And all worked except JPEG. Thus it was finding code for the two new supports you added but not JPEG. When I  defined the LD_LIBRARY_PATH JPEG also worked.

[mrkraimer]

Sorry I spoke too soon. I still see the error on

[MarkRivers]

Hi Marty,

That error means that WITH_BITSHUFFLE=YES is not set in your areaDetector/configure/CONFIG_SITE.local. That was added to EXAMPLE_CONFIG_SITE.local the other day.

[mrkraimer]

On 12/11/18 7:22 AM, Mark Rivers wrote:

Hi Marty,

That error means that WITH_BITSHUFFLE=YES is not set in your areaDetector/configure/CONFIG_SITE.local. That was added to EXAMPLE_CONFIG_SITE.local the other day.

— You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub https://github.com/areaDetector/ADViewers/issues/4#issuecomment-446183833, or mute the thread https://github.com/notifications/unsubscribe-auth/AF1Q1tHq4uS0nltP4W-7CTAwL9cI_w5aks5u36N-gaJpZM4Y-zWZ.

OK now everything works.

And if I do not set LD_LIBRARY_PATH all of JPEG, LZ4, and BSLZ4 fail because the code can not be found

[MarkRivers]

And if I do not set LD_LIBRARY_PATH all of JPEG, LZ4, and BSLZ4 fail because the code can not be found

OK, good that makes sense.

[mrkraimer]

On 12/11/18 11:52 AM, Mark Rivers wrote:

And if I do not set LD_LIBRARY_PATH all of JPEG, LZ4, and BSLZ4
fail because the code can not be found

OK, good that makes sense.

— You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub https://github.com/areaDetector/ADViewers/issues/4#issuecomment-446276139, or mute the thread https://github.com/notifications/unsubscribe-auth/AF1Q1gCS85xVNymYBPuY_QU5jum0i5mjks5u3-LogaJpZM4Y-zWZ.

I have been looking further. What you did for codecs in ADSupport is to take the sources from github and then build via the epics build system. Is this  correct? In ADViewers you used com.sun.jna to "wrap" the code from ADSupport for all codecs except for  blosc. But why not do the same for blosc?

I created a new file:

public class decompressBLOSC {

    static {         Native.register("blosc" + getArchPlatform());     }

    public static String getArchPlatform() {         String archDataModel = System.getProperty("sun.arch.data.model");         if (archDataModel.equals("64")) {             archDataModel = "";         }         return archDataModel;     }

    public static native int blosc_decompress(Buffer src, Buffer dest,long destSize);

}

Then I changed

git diff ImageJ/EPICS_areaDetector/NTNDCodec.java diff --git a/ImageJ/EPICS_areaDetector/NTNDCodec.java b/ImageJ/EPICS_areaDetector/NTNDCodec.java index 14f646f..cedc04b 100755 --- a/ImageJ/EPICS_areaDetector/NTNDCodec.java +++ b/ImageJ/EPICS_areaDetector/NTNDCodec.java @@ -105,15 +105,8 @@ public class NTNDCodec          decompressInBuffer.position(0);          decompressOutBuffer.position(0);

-        if (codecName.equals("blosc")) { -            if (jBlosc == null) { -                jBlosc = new JBlosc(); -            } -            int status = jBlosc.decompress(decompressInBuffer, decompressOutBuffer, uncompressedSize); -            if (status != uncompressedSize) { -                message = "jBlosc.decompress returned status="+status; -                return false; -            } +        if (codecName.equals("blosc")) {

• decompressBLOSC.blosc_decompress(decompressInBuffer, decompressOutBuffer,uncompressedSize);^M          } else if (codecName.equals("jpeg")) {              if (scalarType==ScalarType.pvUByte) {                  decompressJPEGDll.decompressJPEG( mrk>

And it does work.

If you like this idea I can create a new branch and make this change plus a couple of other minor changes and let you see it.

What do You think?

Marty

[MarkRivers]

JBlosc is just a Java wrapper around the Blosc C library, using com.sun.jna. Since JBlosc exists, why not use it instead of writing our own?

[mrkraimer]

On 12/11/18 2:59 PM, Mark Rivers wrote:

JBlosc is just a Java wrapper around the Blosc C library, using com.sun.jna. Since JBlosc exists, why not use it instead of writing our own?

— You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub https://github.com/areaDetector/ADViewers/issues/4#issuecomment-446341222, or mute the thread https://github.com/notifications/unsubscribe-auth/AF1Q1tGZQPVH1d6qt9u1Q0TQK6YxBSuzks5u4A6ngaJpZM4Y-zWZ.

To make sure that what appears in ADSupport is in sync with what is in ADViewers.

[MarkRivers]

To make sure that what appears in ADSupport is in sync with what is in ADViewers.

That is already guaranteed. Jblosc is using the blosc C library from ADSupport. Jblosc does not have its own copy of the C library.

[MarkRivers]

That is already guaranteed. Jblosc is using the blosc C library from ADSupport. Jblosc does not have its own copy of the C library.

Or are you worried about something different?

[mrkraimer]

On 12/11/18 6:20 PM, Mark Rivers wrote:

That is already guaranteed. Jblosc is using the blosc C library
from ADSupport. Jblosc does not have its own copy of the C library.

Or are you worried about something different?

Consistency and ease of understanding and maintaining the code.

If Blosc uses the same technique as JPEG, LZ4, and BSLZ4 then you can just state : "The compression code uses com.sun.jna to wrap code in ADSupport.lib."

These is also no need to describe Jblosc or including jblosc-1.0.1.dev.jar If you upgrade to a later version of blosc in ADSupport you do not have to remember to also upgrade ADViewers. It is also a model to follow if other code in ADSupport needs to be used from Java.

I just want to also say that I learned a lot while working on this issue. I was not familiar with com.sun.jna. Learning about it was alone worth the time. I also learned more about the internals of areaDetector, especially ADSupport.

[MarkRivers]

I have taken your advance and removed jblosc. I added decompressBloscDll.java to replace it.

I renamed myUtil.java to ByteBufferUtil.java, and removed its dependency on Jblosc.

I changed NTNDCodec.java to use switch statements rather than long if else if blocks. I think it makes it easier to read.

Thanks for the help and advice.

[mrkraimer]

You are welcome. Also please delete branch ntndcodec

[MarkRivers]

I have deleted the ntndcodec branch. Closing this issue.