search

areaDetector / ADSupport

Source code for support libraries for areaDetector.
https://areadetector.github.io/areaDetector/user_guide.html
Other
1 stars 15 forks source link

Several fixes to build plugins with MinGW #36

Closed sudilav closed 3 years ago

sudilav commented 3 years ago

When running with MinGW, building plugins fails without the following small changes - I've maintained it such that it isolates changes to when you are specifically using MinGW or when the change is superficial (like changing Winsock to winsock to avoid case sensitivity issues between cross compiling on linux versus direct on windows which doesn't care for case sensitivity).

This was built with a static setup but I also saw the same errors a few months back when using shared.

sudilav commented 3 years ago

Hey @MarkRivers,

Are you able to review these commits? Right now we're keeping an internal copy at my company with my changes but would really like it that these get implemented and subsequently pulled into areaDetector (updating the submodule SHA) so we can use the community standard repo rather than our own internal one.

Thanks,

Ed

MarkRivers commented 3 years ago

@sudilav you did not respond to my review comment about ftdebug.c.

sudilav commented 3 years ago

@MarkRivers Apologies, just testing out VS building and will get back to you with a complete response

sudilav commented 3 years ago

Hey @MarkRivers,

So the reason for ftdebug.c was that we have explicitly defined the variable which only effect is to include that header file when you dive into the code. For some reason in MinGW this doesn't have an effect so there is a build failure that the header isn't included. Since because the variable is always defined and thus that header should always be included by the logic my workaround was to just include it at a higher level.

I make the modification as you suggested to WIN32/config.h, that was a good suggestion.

I'm trying to now test with VS building but I've just hit a hurdle where it can't find zlib.h:

epics-ci\support\areaDetector\ADSupport\supportApp\GraphicsMagickSrc\magick\blob.c(61): fatal error C1083: Cannot open include file: 'zlib.h': No such file or directory

However this is placed in epics-ci\support\areaDetector\ADSupport\include\os\WIN32\zlib.h so not sure why it can't find it.

My CONFIG_SITE.local (in the configure directory of areaDetector) is:

# To build the IOC applications set BUILD_IOCS to YES
# Otherwise set it to NO
BUILD_IOCS=NO

# To build areaDetector so that it only depends on libCom and asyn set the following flag
#EPICS_LIBCOM_ONLY=YES

# Configure which 3rd party libraries to use and where to find them.
# For each library XXX the following definitions are used:
# WITH_XXX      Build the plugins and drivers that require this library.
#               Build the source code for this library in ADSupport if XXX_EXTERNAL=NO.
# XXX_EXTERNAL  If NO then build the source code for this library in ADSupport.
# XXX_INCLUDE   If XXX_EXTERNAL=YES then this is the path to the include files for XXX.
#               However, if XXX is a system library whose include files are in a 
#               standard include search path then do not define XXX_INCLUDE.
# XXX_LIB       If XXX_EXTERNAL=YES then this is the path to the library files for XXX.
#               However, if XXX is a system library whose library files in a 
#               standard library search path then do not define XXX_LIB.

# The exception to the above is XML2. 
# As of ADCORE R3-0 libxml2 is required for ADCore so WITH_XML2 is not supported
# but the other XML2 flags are supported.

# The default defined here is appropriate for most Linux systems.

# BOOST is required for the ADCore/pluginTests unit tests
WITH_BOOST     = NO
BOOST_EXTERNAL = YES
#BOOST_LIB     =
#BOOST_INCLUDE =

# Uncomment this line if you want to use STATIC Linking when building
# ADCore/pluginTests unit tests.
#BOOST_USE_STATIC_LINK=YES

# EPICS PVA is required for NDPluginPva, pvaDriver, and qsrv
WITH_PVA  = YES

# WITH_QSRV controls whether IOCs are built with support for qsrv.
# This is separate from WITH_PVA to allow PVA to be enabled but QSRV to be disabled.
# This is needed on 3.14.12 where PVA is supported but QSRV is not.
# Note that if WITH_QSRV=YES then WITH_PVA must also be YES
WITH_QSRV = YES

# BLOSC is required to use the blosc compressors in the HDF5 plugin
WITH_BLOSC     = YES
BLOSC_EXTERNAL = NO
#BLOSC_INCLUDE =
#BLOSC_LIB     =

# BITSHUFFLE is required to use the bitshuffle/LZ4 decompressors in the ADEiger driver
# It can also be used to generate shareable libraries for the HDF5 compressor/decompressor
WITH_BITSHUFFLE     = YES
BITSHUFFLE_EXTERNAL = NO

# GraphicsMagick is required for NDFileMagick and the URL driver.
WITH_GRAPHICSMAGICK     = YES
GRAPHICSMAGICK_EXTERNAL = NO
# GraphicsMagick defines many global symbols, and one of these (SetImage) conflicts with a symbol
# in the Andor vendor library.  GraphickMagick provides a solution by allowing all external library
# symbols to be prefixed with "Gm".  Enable using this by default.  However, if a system library
# is used it may not be built with this option so this line should be commented out.
GRAPHICSMAGICK_PREFIX_SYMBOLS = YES
#GRAPHICSMAGICK_INCLUDE =
#GRAPHICSMAGICK_LIB     =

# HDF5 is required for the NDFileHDF5 and NDFileNexus plugins
WITH_HDF5     = YES
HDF5_EXTERNAL = NO
#HDF5_INCLUDE =
#HDF5_LIB     =

# JSON is required for the NDPluginBadPixel plugin
# The JSON parser requires C++11 and does not work on vxWorks 9.4
WITH_JSON     = YES

# JPEG is required for the NDFileJPEG and NDFileTIFF plugins
WITH_JPEG     = YES
JPEG_EXTERNAL = NO
#JPEG_INCLUDE =
#JPEG_LIB     =

# netCDF is required for the NDFileNetCDF plugin
WITH_NETCDF     = YES
NETCDF_EXTERNAL = NO
#NETCDF_INCLUDE =
#NETCDF_LIB     =

# NEXUS is required for the NDFileNexus plugin
WITH_NEXUS     = YES
NEXUS_EXTERNAL = NO
#NEXUS_INCLUDE =
#NEXUS_LIB     =

# OPENCV is required for the ADPluginEdge plugin
WITH_OPENCV     = NO
OPENCV_EXTERNAL = YES
#OPENCV_INCLUDE =
#OPENCV_LIB     =

# SZIP is required for the NDFileHDF5 and NDFileNexus plugins.  
# This may be optional in the future.
WITH_SZIP     = YES
SZIP_EXTERNAL = NO
#SZIP_INCLUDE =
#SZIP_LIB     =

# TIFF is required for the NDFileTIFF plugin.
WITH_TIFF     = YES
TIFF_EXTERNAL = NO
#TIFF_INCLUDE =
#TIFF_LIB     =

# XML2 is required for ADCore so there is no WITH_XML2.  
XML2_EXTERNAL = NO
#XML2_INCLUDE =
#XML2_LIB     =

# ZLIB is required for the NDFileHDF5 and NDFileNexus plugins.  
# This may be optional in the future.
WITH_ZLIB     = YES
ZLIB_EXTERNAL = NO
#ZLIB_INCLUDE =
#ZLIB_LIB     =

# ARAVIS is required for aravisGigE and ADAravis
# Only available on Linux
#ARAVIS_LIB      = /usr/local/lib
ARAVIS_INCLUDE  = /usr/local/include/aravis-0.8
# Location of GLIB include files and libraries.  This is needed for aravis.
# Our system has include files for glib-2.0 in 2 different places
GLIB_INCLUDE = /usr/include/glib-2.0 /usr/lib64/glib-2.0/include
glib-2.0_DIR = /usr/lib64

# The definitions above can be overridden in the following files.
# The files are searched in this order, with the last definition being used.
#    CONFIG_SITE.local.$(OS_CLASS)
#    CONFIG_SITE.local.$(EPICS_HOST_ARCH) 
#    CONFIG_SITE.local.$(EPICS_HOST_ARCH).$(T_A)

I've not hit this issue before or with MingW. I'm running VS Build Tools, 2019, amd64 setup as per the instructions from the EPICS documentation on Windows builds/installs. I don't see how my code changes could have caused this either. Any ideas?

MarkRivers commented 3 years ago

@sudilav

epics-ci\support\areaDetector\ADSupport\supportApp\GraphicsMagickSrc\magick\blob.c(61): fatal error C1083: Cannot open include file: 'zlib.h': No such file or directory However this is placed in epics-ci\support\areaDetector\ADSupport\include\os\WIN32\zlib.h so not sure why it can't find it.

I don't think your code changes caused this, something must be different with your build system. I just tested recompiling blob.c with Visual Studio 2015 and it worked fine. This is the output. I added line breaks to make the it more legible. 

J:\epics\devel\areaDetector\ADSupport\supportApp\GraphicsMagickSrc\magick>del O.windows-x64-static\blob.obj

J:\epics\devel\areaDetector\ADSupport\supportApp\GraphicsMagickSrc\magick>make
make -C O.windows-x64-static -f ../Makefile TOP=../../../.. \
    T_A=windows-x64-static install
make[1]: Entering directory 'J:/epics/devel/areaDetector/ADSupport/supportApp/GraphicsMagickSrc/magick/O.windows-x64-static'
cl               -nologo -FC -D__STDC__=0 -D_CRT_SECURE_NO_DEPRECATE -D_CRT_NONSTDC_NO_DEPRECATE    -Ox -Oy-   -W3
     -D_MAGICKLIB_ -DHAVE_VSNPRINTF -DPREFIX_MAGICK_SYMBOLS -DWIN64 -D_VISUALC_ -D_LIB    -MT    
-I. -I../O.Common -I. -I. -I.. -I../../../../include/magick/compiler/msvc 
-I../../../../include/magick/os/WIN32 
-I../../../../include/magick      
-IJ:/epics/devel/asyn-4-42/include     
-IJ:/epics/devel/areaDetector-3-11/ADSupport/include/os/WIN32 
-IJ:/epics/devel/areaDetector-3-11/ADSupport/include   
-IJ:/epics/devel/areaDetector-3-11/ADCore/include 
-IH:/epics-devel/base-7.0.5/include/compiler/msvc 
-IH:/epics-devel/base-7.0.5/include/os/WIN32 
-IH:/epics-devel/base-7.0.5/include     
-I../../../../supportApp/GraphicsMagickSrc 
-I../../../../supportApp/GraphicsMagickSrc/lcms/include 
-I../../../../supportApp/GraphicsMagickSrc/ttf/include    
-c ../blob.c
blob.c
Magick lib static interface
Magick module static interface
BZIP compiling as library
lib -nologo -verbose -out:Magick.lib     animate.obj annotate.obj attribute.obj average.obj bit_stream.obj blob.obj cdl.obj
channel.obj color.obj colormap.obj colorspace.obj color_lookup.obj command.obj compare.obj composite.obj compress.obj
confirm_access.obj constitute.obj decorate.obj delegate.obj deprecate.obj describe.obj display.obj draw.obj effect.obj enhance.obj enum_strings.obj error.obj export.obj floats.obj fx.obj gem.obj gradient.obj hclut.obj image.obj import.obj list.obj
locale.obj log.obj magic.obj magick.obj magick_endian.obj map.obj memory.obj module.obj monitor.obj montage.obj
nt_base.obj nt_feature.obj omp_data_view.obj operator.obj paint.obj pixel_cache.obj pixel_iterator.obj plasma.obj PreRvIcccm.obj
profile.obj quantize.obj random.obj registry.obj render.obj resize.obj resource.obj segment.obj semaphore.obj shear.obj
signature.obj statistics.obj tempfile.obj texture.obj timer.obj transform.obj tsd.obj type.obj unix_port.obj utility.obj version.obj
widget.obj xwindow.obj analyze_Win32.obj
Appending analyze_Win32.obj
Appending xwindow.obj
Appending widget.obj
Appending version.obj
Appending utility.obj
Appending unix_port.obj
Appending type.obj
Appending tsd.obj
Appending transform.obj
Appending timer.obj
Appending texture.obj
Appending tempfile.obj
Appending statistics.obj
Appending signature.obj
Appending shear.obj
Appending semaphore.obj
Appending segment.obj
Appending resource.obj
Appending resize.obj
Appending render.obj
Appending registry.obj
Appending random.obj
Appending quantize.obj
Appending profile.obj
Appending PreRvIcccm.obj
Appending plasma.obj
Appending pixel_iterator.obj
Appending pixel_cache.obj
Appending paint.obj
Appending operator.obj
Appending omp_data_view.obj
Appending nt_feature.obj
Appending nt_base.obj
Appending montage.obj
Appending monitor.obj
Appending module.obj
Appending memory.obj
Appending map.obj
Appending magick_endian.obj
Appending magick.obj
Appending magic.obj
Appending log.obj
Appending locale.obj
Appending list.obj
Appending import.obj
Appending image.obj
Appending hclut.obj
Appending gradient.obj
Appending gem.obj
Appending fx.obj
Appending floats.obj
Appending export.obj
Appending error.obj
Appending enum_strings.obj
Appending enhance.obj
Appending effect.obj
Appending draw.obj
Appending display.obj
Appending describe.obj
Appending deprecate.obj
Appending delegate.obj
Appending decorate.obj
Appending constitute.obj
Appending confirm_access.obj
Appending compress.obj
Appending composite.obj
Appending compare.obj
Appending command.obj
Appending color_lookup.obj
Appending colorspace.obj
Appending colormap.obj
Appending color.obj
Appending channel.obj
Appending cdl.obj
Appending blob.obj
Appending bit_stream.obj
Appending average.obj
Appending attribute.obj
Appending annotate.obj
Appending animate.obj
"Installing library ../../../../lib/windows-x64-static/Magick.lib"
make[1]: Leaving directory 'J:/epics/devel/areaDetector/ADSupport/supportApp/GraphicsMagickSrc/magick/O.windows-x64-static'

Note that the compile command contains this:

-IJ:/epics/devel/areaDetector-3-11/ADSupport/include/os/WIN32

That is how it finds zlib.h. Does your compile command contain the equivalent of that line?

sudilav commented 3 years ago

Hi @MarkRivers,

It did not nor did it contain /ADSupport/include/ for including bzlib.h but I added them explicitly into magick Makefile as USR_INCLUDES and got it to compile. This was within the if statement as to whether we were building external ZLIB set to NO. Strange why that didn't work but its not part of my changes and I did a info print to check that ZLIB_EXTERNAL was set to NO.

Anyway, pushing past that error I got a successful compile from ADSupport using Visual Studio Build Tools 2019. Should I investigate that error further as it seems unrelated?

Are we happy with the changes to merge or do you need more info from me?

Thanks,

Ed

sudilav commented 3 years ago

Hey @MarkRivers,

Any news? Would really like this merged so we can drop our internal version and go back to using the community version

MarkRivers commented 3 years ago

I am not satisfied with your changes to ftdebug.c. That file includes ftbuild.h, which includes freetype/config/ftheader.h, which contains this:

/*
   * Include internal headers definitions from <internal/...>
   * only when building the library.
   */
#ifdef FT2_BUILD_LIBRARY
#define  FT_INTERNAL_INTERNAL_H  <freetype/internal/internal.h>
#include FT_INTERNAL_INTERNAL_H
#endif /* FT2_BUILD_LIBRARY */

Note that it only includes <freetype/internal/internal.h> if FT2_BUILD_LIBRARY is defined. However, you have changed the logic so it always includes that file. You have also deleted this line which includes a completely different file.

#include FT_INTERNAL_DEBUG_H

It appears to me that the original version of ftdebug.c should always include <freetype/internal/internal.h> when it is needed. I think you need to dig deeper and understand why the original logic is not working for you with MINGW. You are introducing changes that could have subtle side effects.

sudilav commented 3 years ago

@MarkRivers I just ran the compile again and reverted my changes to freetype, it worked fine without needing me to explicitly include the header file like I did before.

I've removed that change and left everything else - given that change is no longer being implemented - are we happy to merge this pull now?

Thanks,

Ed