search

areaDetector / ADCore

The home of the core components of the EPICS areaDetector software. It includes base classes for drivers and code for all of the standard plugins.
https://areadetector.github.io/areaDetector
Other
20 stars 69 forks source link

HDF5 plugin feature where datasets with type=constant is broken #36

Open ulrikpedersen opened 10 years ago

ulrikpedersen commented 10 years ago

@MarkRivers comments on issue #31 that attempting to define a constant dataset, like this is broken:

 <dataset name="axis1" source="constant" value="1"></dataset>

The above XML is valid according to the schema, but does not actually work when it comes to creating the dataset in the plugin code.

A constant attribute has a type field of type="attrSourceTypeEnum". The equivalent has not been defined for the datasetType.

This constant dataset feature is a requirement from Arthur and Nicholas. When I grep through my example XML files, I can't find a single instance of constant datasets; so I am afraid this has never been tested at my end. @aglowacki : do you have any examples of this feature, working or not at your end?

We don't really need this feature here at Diamond. Are anyone interested in adding it or can we leave it out? As I understand it, Arthur and Nicholas have already been using this version of the plugin without reporting an issue with this, so maybe it is not necessary after all...?

aglowacki commented 10 years ago

We wanted the constant dataset feature for adding meta data to the hdf5 file such as experimenter, sector, ext. Since we can send the xml definition over a waveform we could dynamically add more meta data to the hdf5 file.

The current TXM xml definition at sector 32 (where we first deployed the hdf5 plugin) does not yet set this kind of constant meta data. It only does constants data for attributes. I think we deployed and tested there and didn't notice the constant dataset not working since they don't have any.

So currently we don't use the constant dataset but I think it can be useful in the future.

MarkRivers commented 10 years ago

OK, we will plan on supporting this in a future release. It won’t be supported in R2-1.

From: aglowacki [mailto:notifications@github.com] Sent: Monday, October 13, 2014 10:44 AM To: areaDetector/ADCore Cc: Mark Rivers Subject: Re: [ADCore] HDF5 plugin feature where datasets with type=constant is broken (#36)

We wanted the constant dataset feature for adding meta data to the hdf5 file such as experimenter, sector, ext. Since we can send the xml definition over a waveform we could dynamically add more meta data to the hdf5 file.

The current TXM xml definition at sector 32 (where we first deployed the hdf5 plugin) does not yet set this kind of constant meta data. It only does constants data for attributes. I think we deployed and tested there and didn't notice the constant dataset not working since they don't have any.

So currently we don't use the constant dataset but I think it can be useful in the future.

— Reply to this email directly or view it on GitHubhttps://github.com/areaDetector/ADCore/issues/36#issuecomment-58911044.

ulrikpedersen commented 10 years ago

Agreed. We don't support constant-value datasets in R2-1.

I'm assigning this ticket to Arthur as he is most likely the one that will need this feature later on (and can close it if the requirement goes away).

MarkRivers commented 9 years ago

Hi Arthur and Ulrik,

I am working on the new demonstration file ADCore/iocs/simDetectorNoIOC/simDetectorNoIOCApp/src/simDetectorNoIOC.cpp. This is an example application that instantiates a simDetector outside the context of an IOC.

This application builds fine on Windows with Visual Studio, and on Linux with gcc 4.6.3. However, with gcc 4.3.0 and 4.2.1 I get the following errors:

/usr/bin/g++ -c -D_POSIX_C_SOURCE=199506L -D_POSIX_THREADS -D_XOPEN_SOURCE=500 -DX86 -DUNIX -D_BSD_SOURCE -Dlinux -D_REENTRANT -O3 -Wall -m32 -MMD -I. -I../O.Common -I. -I.. -I../../../include/os/Linux -I../../../include -I/corvette/usr/local/epics/base-3.14.12.4/include/os/Linux -I/corvette/usr/local/epics/base-3.14.12.4/include -I/corvette/home/epics/devel/asyn-4-24/include -I/corvette/home/epics/devel/areaDetector-2-1/ADBinaries/include -I/corvette/home/epics/devel/areaDetector-2-1/ADCore/include -I/corvette/home/epics/devel/calc-3-4-1/include -I/corvette/home/epics/devel/busy-1-6/include -I/corvette/home/epics/devel/sscan-2-10/include -I/corvette/home/epics/devel/autosave-5-4-2/include/os/Linux -I/corvette/home/epics/devel/autosave-5-4-2/include -I/usr/local/hdf5/hdf5-1.8.7/hdf5/include -I/usr/include/libxml2 ../simDetectorNoIOC.cpp /usr/lib/gcc/i386-redhat-linux/4.3.0/../../../../include/c++/4.3.0/bits/ostream_insert.h: In function 'std::basic_ostream<_CharT, _Traits>& std::__ostream_insert(std::basic_ostream<_CharT, _Traits>&, const _CharT*, std::streamsize) [with _CharT = char, _Traits = std::char_traits]': /usr/lib/gcc/i386-redhat-linux/4.3.0/../../../../include/c++/4.3.0/bits/basic_string.h:2425: instantiated from 'std::basic_ostream<_CharT, _Traits>& std::operator<<(std::basic_ostream<_CharT, _Traits>&, const std::basic_string<_CharT, _Traits, _Alloc>&) [with _CharT = char, _Traits = std::char_traits, _Alloc = std::allocator]' /corvette/home/epics/devel/areaDetector-2-1/ADCore/include/NDFileHDF5Layout.h:155: instantiated from here /usr/lib/gcc/i386-redhat-linux/4.3.0/../../../../include/c++/4.3.0/bits/ostream_insert.h:83: error: invalid use of incomplete type 'struct std::basic_ostream<char, std::char_traits >' /usr/lib/gcc/i386-redhat-linux/4.3.0/../../../../include/c++/4.3.0/iosfwd:61: error: declaration of 'struct std::basic_ostream<char, std::char_traits >' /usr/lib/gcc/i386-redhat-linux/4.3.0/../../../../include/c++/4.3.0/bits/ostream_insert.h:85: error: invalid use of incomplete type 'struct std::basic_ostream<char, std::char_traits >' /usr/lib/gcc/i386-redhat-linux/4.3.0/../../../../include/c++/4.3.0/iosfwd:61: error: declaration of 'struct std::basic_ostream<char, std::char_traits >' /usr/lib/gcc/i386-redhat-linux/4.3.0/../../../../include/c++/4.3.0/bits/ostream_insert.h:85: error: invalid use of incomplete type 'struct std::basic_ostream<char, std::char_traits >' /usr/lib/gcc/i386-redhat-linux/4.3.0/../../../../include/c++/4.3.0/iosfwd:61: error: declaration of 'struct std::basic_ostream<char, std::char_traits >' /usr/lib/gcc/i386-redhat-linux/4.3.0/../../../../include/c++/4.3.0/bits/ostream_insert.h:90: error: invalid use of incomplete type 'struct std::basic_ostream<char, std::char_traits >' /usr/lib/gcc/i386-redhat-linux/4.3.0/../../../../include/c++/4.3.0/iosfwd:61: error: declaration of 'struct std::basic_ostream<char, std::char_traits >' /usr/lib/gcc/i386-redhat-linux/4.3.0/../../../../include/c++/4.3.0/bits/ostream_insert.h:95: error: invalid use of incomplete type 'struct std::basic_ostream<char, std::char_traits >' /usr/lib/gcc/i386-redhat-linux/4.3.0/../../../../include/c++/4.3.0/iosfwd:61: error: declaration of 'struct std::basic_ostream<char, std::char_traits >' /usr/lib/gcc/i386-redhat-linux/4.3.0/../../../../include/c++/4.3.0/bits/basic_string.h:2425: instantiated from 'std::basic_ostream<_CharT, _Traits>& std::operator<<(std::basic_ostream<_CharT, _Traits>&, const std::basic_string<_CharT, _Traits, _Alloc>&) [with _CharT = char, _Traits = std::char_traits, _Alloc = std::allocator]' /corvette/home/epics/devel/areaDetector-2-1/ADCore/include/NDFileHDF5Layout.h:155: instantiated from here

Note that the simDetectorIOC builds fine, as do other real detector IOCs on these older compilers. So the compilers can handle the HDF5 code OK, it must just be the order of include files or something?

Any ideas?

Thanks, Mark

MarkRivers commented 9 years ago

I figured out the problem. There was an error in ADCore/ADApp/pluginSrc/NDFileHDF5Layout.h. That file references the ostream class, but did not include . I added the include and that fixed the problem.

I’m not sure why it only gave errors on older compilers, but it was definitely an error.

Mark

From: Mark Rivers Sent: Wednesday, December 10, 2014 5:02 PM To: 'areaDetector/ADCore'; areaDetector/ADCore Cc: 'Glowacki, Arthur T.'; 'ulrik.pedersen@diamond.ac.uk' Subject: Problem with HDF5 includes

Hi Arthur and Ulrik,

I am working on the new demonstration file ADCore/iocs/simDetectorNoIOC/simDetectorNoIOCApp/src/simDetectorNoIOC.cpp. This is an example application that instantiates a simDetector outside the context of an IOC.

This application builds fine on Windows with Visual Studio, and on Linux with gcc 4.6.3. However, with gcc 4.3.0 and 4.2.1 I get the following errors:

/usr/bin/g++ -c -D_POSIX_C_SOURCE=199506L -D_POSIX_THREADS -D_XOPEN_SOURCE=500 -DX86 -DUNIX -D_BSD_SOURCE -Dlinux -D_REENTRANT -O3 -Wall -m32 -MMD -I. -I../O.Common -I. -I.. -I../../../include/os/Linux -I../../../include -I/corvette/usr/local/epics/base-3.14.12.4/include/os/Linux -I/corvette/usr/local/epics/base-3.14.12.4/include -I/corvette/home/epics/devel/asyn-4-24/include -I/corvette/home/epics/devel/areaDetector-2-1/ADBinaries/include -I/corvette/home/epics/devel/areaDetector-2-1/ADCore/include -I/corvette/home/epics/devel/calc-3-4-1/include -I/corvette/home/epics/devel/busy-1-6/include -I/corvette/home/epics/devel/sscan-2-10/include -I/corvette/home/epics/devel/autosave-5-4-2/include/os/Linux -I/corvette/home/epics/devel/autosave-5-4-2/include -I/usr/local/hdf5/hdf5-1.8.7/hdf5/include -I/usr/include/libxml2 ../simDetectorNoIOC.cpp /usr/lib/gcc/i386-redhat-linux/4.3.0/../../../../include/c++/4.3.0/bits/ostream_insert.h: In function 'std::basic_ostream<_CharT, _Traits>& std::__ostream_insert(std::basic_ostream<_CharT, _Traits>&, const _CharT*, std::streamsize) [with _CharT = char, _Traits = std::char_traits]': /usr/lib/gcc/i386-redhat-linux/4.3.0/../../../../include/c++/4.3.0/bits/basic_string.h:2425: instantiated from 'std::basic_ostream<_CharT, _Traits>& std::operator<<(std::basic_ostream<_CharT, _Traits>&, const std::basic_string<_CharT, _Traits, _Alloc>&) [with _CharT = char, _Traits = std::char_traits, _Alloc = std::allocator]' /corvette/home/epics/devel/areaDetector-2-1/ADCore/include/NDFileHDF5Layout.h:155: instantiated from here /usr/lib/gcc/i386-redhat-linux/4.3.0/../../../../include/c++/4.3.0/bits/ostream_insert.h:83: error: invalid use of incomplete type 'struct std::basic_ostream<char, std::char_traits >' /usr/lib/gcc/i386-redhat-linux/4.3.0/../../../../include/c++/4.3.0/iosfwd:61: error: declaration of 'struct std::basic_ostream<char, std::char_traits >' /usr/lib/gcc/i386-redhat-linux/4.3.0/../../../../include/c++/4.3.0/bits/ostream_insert.h:85: error: invalid use of incomplete type 'struct std::basic_ostream<char, std::char_traits >' /usr/lib/gcc/i386-redhat-linux/4.3.0/../../../../include/c++/4.3.0/iosfwd:61: error: declaration of 'struct std::basic_ostream<char, std::char_traits >' /usr/lib/gcc/i386-redhat-linux/4.3.0/../../../../include/c++/4.3.0/bits/ostream_insert.h:85: error: invalid use of incomplete type 'struct std::basic_ostream<char, std::char_traits >' /usr/lib/gcc/i386-redhat-linux/4.3.0/../../../../include/c++/4.3.0/iosfwd:61: error: declaration of 'struct std::basic_ostream<char, std::char_traits >' /usr/lib/gcc/i386-redhat-linux/4.3.0/../../../../include/c++/4.3.0/bits/ostream_insert.h:90: error: invalid use of incomplete type 'struct std::basic_ostream<char, std::char_traits >' /usr/lib/gcc/i386-redhat-linux/4.3.0/../../../../include/c++/4.3.0/iosfwd:61: error: declaration of 'struct std::basic_ostream<char, std::char_traits >' /usr/lib/gcc/i386-redhat-linux/4.3.0/../../../../include/c++/4.3.0/bits/ostream_insert.h:95: error: invalid use of incomplete type 'struct std::basic_ostream<char, std::char_traits >' /usr/lib/gcc/i386-redhat-linux/4.3.0/../../../../include/c++/4.3.0/iosfwd:61: error: declaration of 'struct std::basic_ostream<char, std::char_traits >' /usr/lib/gcc/i386-redhat-linux/4.3.0/../../../../include/c++/4.3.0/bits/basic_string.h:2425: instantiated from 'std::basic_ostream<_CharT, _Traits>& std::operator<<(std::basic_ostream<_CharT, _Traits>&, const std::basic_string<_CharT, _Traits, _Alloc>&) [with _CharT = char, _Traits = std::char_traits, _Alloc = std::allocator]' /corvette/home/epics/devel/areaDetector-2-1/ADCore/include/NDFileHDF5Layout.h:155: instantiated from here

Note that the simDetectorIOC builds fine, as do other real detector IOCs on these older compilers. So the compilers can handle the HDF5 code OK, it must just be the order of include files or something?

Any ideas?

Thanks, Mark

MarkRivers commented 9 years ago

Hi Mark,

Thanks for that. I have found this kind of problem before: some versions of gcc and g++ appear to be better at finding “standard” includes (i.e. basic headers from gcc or from STL). Most recently it happened to me on another project when I used the travis-ci.org continuous integration service, which builds using a more modern gcc on Ubuntu.

The behaviour seems to differ between Linux distributions, so it may be some configuration when you build gcc. I have never got to the bottom of it, but I think it’s best always to include the appropriate header anyway.

Cheers, Ulrik

From: Mark Rivers [mailto:rivers@cars.uchicago.edu] Sent: 10 December 2014 23:31 To: Mark Rivers; 'areaDetector/ADCore'; 'areaDetector/ADCore' Cc: 'Glowacki, Arthur T.'; Pedersen, Ulrik (DLSLtd,RAL,TEC) Subject: RE: Problem with HDF5 includes

I figured out the problem. There was an error in ADCore/ADApp/pluginSrc/NDFileHDF5Layout.h. That file references the ostream class, but did not include . I added the include and that fixed the problem.

I’m not sure why it only gave errors on older compilers, but it was definitely an error.

Mark

From: Mark Rivers Sent: Wednesday, December 10, 2014 5:02 PM To: 'areaDetector/ADCore'; areaDetector/ADCore Cc: 'Glowacki, Arthur T.'; 'ulrik.pedersen@diamond.ac.uk' Subject: Problem with HDF5 includes

Hi Arthur and Ulrik,

I am working on the new demonstration file ADCore/iocs/simDetectorNoIOC/simDetectorNoIOCApp/src/simDetectorNoIOC.cpp. This is an example application that instantiates a simDetector outside the context of an IOC.

This application builds fine on Windows with Visual Studio, and on Linux with gcc 4.6.3. However, with gcc 4.3.0 and 4.2.1 I get the following errors:

/usr/bin/g++ -c -D_POSIX_C_SOURCE=199506L -D_POSIX_THREADS -D_XOPEN_SOURCE=500 -DX86 -DUNIX -D_BSD_SOURCE -Dlinux -D_REENTRANT -O3 -Wall -m32 -MMD -I. -I../O.Common -I. -I.. -I../../../include/os/Linux -I../../../include -I/corvette/usr/local/epics/base-3.14.12.4/include/os/Linux -I/corvette/usr/local/epics/base-3.14.12.4/include -I/corvette/home/epics/devel/asyn-4-24/include -I/corvette/home/epics/devel/areaDetector-2-1/ADBinaries/include -I/corvette/home/epics/devel/areaDetector-2-1/ADCore/include -I/corvette/home/epics/devel/calc-3-4-1/include -I/corvette/home/epics/devel/busy-1-6/include -I/corvette/home/epics/devel/sscan-2-10/include -I/corvette/home/epics/devel/autosave-5-4-2/include/os/Linux -I/corvette/home/epics/devel/autosave-5-4-2/include -I/usr/local/hdf5/hdf5-1.8.7/hdf5/include -I/usr/include/libxml2 ../simDetectorNoIOC.cpp /usr/lib/gcc/i386-redhat-linux/4.3.0/../../../../include/c++/4.3.0/bits/ostream_insert.h: In function 'std::basic_ostream<_CharT, _Traits>& std::__ostream_insert(std::basic_ostream<_CharT, _Traits>&, const _CharT*, std::streamsize) [with _CharT = char, _Traits = std::char_traits]': /usr/lib/gcc/i386-redhat-linux/4.3.0/../../../../include/c++/4.3.0/bits/basic_string.h:2425: instantiated from 'std::basic_ostream<_CharT, _Traits>& std::operator<<(std::basic_ostream<_CharT, _Traits>&, const std::basic_string<_CharT, _Traits, _Alloc>&) [with _CharT = char, _Traits = std::char_traits, _Alloc = std::allocator]' /corvette/home/epics/devel/areaDetector-2-1/ADCore/include/NDFileHDF5Layout.h:155: instantiated from here /usr/lib/gcc/i386-redhat-linux/4.3.0/../../../../include/c++/4.3.0/bits/ostream_insert.h:83: error: invalid use of incomplete type 'struct std::basic_ostream<char, std::char_traits >' /usr/lib/gcc/i386-redhat-linux/4.3.0/../../../../include/c++/4.3.0/iosfwd:61: error: declaration of 'struct std::basic_ostream<char, std::char_traits >' /usr/lib/gcc/i386-redhat-linux/4.3.0/../../../../include/c++/4.3.0/bits/ostream_insert.h:85: error: invalid use of incomplete type 'struct std::basic_ostream<char, std::char_traits >' /usr/lib/gcc/i386-redhat-linux/4.3.0/../../../../include/c++/4.3.0/iosfwd:61: error: declaration of 'struct std::basic_ostream<char, std::char_traits >' /usr/lib/gcc/i386-redhat-linux/4.3.0/../../../../include/c++/4.3.0/bits/ostream_insert.h:85: error: invalid use of incomplete type 'struct std::basic_ostream<char, std::char_traits >' /usr/lib/gcc/i386-redhat-linux/4.3.0/../../../../include/c++/4.3.0/iosfwd:61: error: declaration of 'struct std::basic_ostream<char, std::char_traits >' /usr/lib/gcc/i386-redhat-linux/4.3.0/../../../../include/c++/4.3.0/bits/ostream_insert.h:90: error: invalid use of incomplete type 'struct std::basic_ostream<char, std::char_traits >' /usr/lib/gcc/i386-redhat-linux/4.3.0/../../../../include/c++/4.3.0/iosfwd:61: error: declaration of 'struct std::basic_ostream<char, std::char_traits >' /usr/lib/gcc/i386-redhat-linux/4.3.0/../../../../include/c++/4.3.0/bits/ostream_insert.h:95: error: invalid use of incomplete type 'struct std::basic_ostream<char, std::char_traits >' /usr/lib/gcc/i386-redhat-linux/4.3.0/../../../../include/c++/4.3.0/iosfwd:61: error: declaration of 'struct std::basic_ostream<char, std::char_traits >' /usr/lib/gcc/i386-redhat-linux/4.3.0/../../../../include/c++/4.3.0/bits/basic_string.h:2425: instantiated from 'std::basic_ostream<_CharT, _Traits>& std::operator<<(std::basic_ostream<_CharT, _Traits>&, const std::basic_string<_CharT, _Traits, _Alloc>&) [with _CharT = char, _Traits = std::char_traits, _Alloc = std::allocator]' /corvette/home/epics/devel/areaDetector-2-1/ADCore/include/NDFileHDF5Layout.h:155: instantiated from here

Note that the simDetectorIOC builds fine, as do other real detector IOCs on these older compilers. So the compilers can handle the HDF5 code OK, it must just be the order of include files or something?

Any ideas?

Thanks, Mark

This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail. Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message. Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom