search

astromatic / sextractor

Extract catalogs of sources from astronomical images
http://astromatic.net/software/sextractor
GNU General Public License v3.0
148 stars 43 forks source link

Discrepancy between intel and gcc compiler #28

Open g-braeunlich opened 1 year ago

g-braeunlich commented 1 year ago

Did a quick test using sextractor version 2.25.0. Note: although, I was using an old gcc version on a HPC cluster, also up to date versions of gcc seem to lead to almost the same result.

  1. gcc/4.8.5 + fftw/3.3.9 + atlas/3.10.2

    #   1 NUMBER                 Running object number                                     
#   2 FLUX_RADIUS            Fraction-of-light radii                                    [pixel]
#   3 FLUX_AUTO              Flux within a Kron-like elliptical aperture                [count]
#   4 MAG_AUTO               Kron-like elliptical aperture magnitude                    [mag]
#   5 MAG_WIN                Gaussian-weighted magnitude                                [mag]
     1     18.416   0.01265051  36.5447  37.2098
     2     17.155   0.02413109  35.8436  36.3271
     3      9.331   0.07458996  34.6183  35.4000
     4      3.109 0.0005396747  39.9697  40.7248
     5      9.440  0.006560602  37.2576  38.0095
     6      3.450 0.0005385359  39.9720  40.6494
     7      4.610  0.000417984  40.2471  40.9967
     8      4.811  0.000814142  39.5233  40.2831
     9      7.209  0.002019732  38.5368  39.3207
    10      0.630  8.24541e-05  42.0095  42.4871
    11      0.644 0.0003812417  40.3470  40.9652
    12      2.444 0.0002358917  40.8682  41.6207
    13      6.857 0.0005290835  39.9912  40.6536
    14      5.277   0.02970535  35.6179  36.3627
    15      3.904  0.004263627  37.7256  38.4955
    16      5.558 0.0002740797  40.7053  41.1688
    17      2.816  0.009487203  36.8572  37.6168
    18      2.636 0.0001669553  41.2435  42.0243
    19      1.829   0.02703672  35.7201  36.5313
    20      0.693 6.265169e-05  42.3077  42.9687
    21      2.634  0.003446309  37.9566  38.6929
    22      1.176 0.0001651363  41.2554  42.0858
    23      1.994  0.000298967  40.6109  41.4154

  2. intel/2018.1 (+ mkl)

    #   1 NUMBER                 Running object number                                     
#   2 FLUX_RADIUS            Fraction-of-light radii                                    [pixel]
#   3 FLUX_AUTO              Flux within a Kron-like elliptical aperture                [count]
#   4 MAG_AUTO               Kron-like elliptical aperture magnitude                    [mag]
#   5 MAG_WIN                Gaussian-weighted magnitude                                [mag]
     1     18.373   0.01261778  36.5475  37.1998
     2      5.339    0.0300436  35.6056  36.3512
     3     16.941   0.02385332  35.8561  36.3346
     4      9.292   0.07417104  34.6244  35.4043
     5      3.109 0.0005396748  39.9697  40.7248
     6      9.447  0.006565565  37.2568  38.0088
     7      3.450 0.0005385373  39.9720  40.6494
     8      4.610 0.0004179853  40.2471  40.9967
     9      4.811 0.0008141441  39.5232  40.2831
    10      7.225  0.002027673  38.5325  39.3185
    11      0.630 8.245417e-05  42.0095  42.4871
    12      0.644 0.0003812418  40.3470  40.9652
    13      2.634  0.003446304  37.9566  38.6929
    14      3.904  0.004263623  37.7256  38.4955
    15      1.829   0.02703672  35.7201  36.5313
    16      5.558 0.0002740777  40.7053  41.1689
    17      2.816    0.0094872  36.8572  37.6168
    18      2.636 0.0001669546  41.2435  42.0243
    19      0.693 6.265133e-05  42.3077  42.9687
    20      6.857 0.0005290824  39.9912  40.6536
    21      1.176 0.0001651343  41.2554  42.0858
    22      1.994  0.000298965  40.6109  41.4154
    23      2.444 0.0002358903  40.8682  41.6207

Do you have an idea, what could be the cause of this? I debugged a bit the field FLUX_RADIUS. I saw that this is mostly computed in src/growth.c. Already there, there seems to be a slight discrepancy: pix = 6.5818e-07 (gcc) pix = 6.5819e-07 (intel)

Input

deafult.conv

CONV NORM
1 2 1
2 4 2
1 2 1

default.nnw

NNW
3 10 10 1

-1.56604e+00 -2.48265e+00 -1.44564e+00 -1.24675e+00 -9.44913e-01 -5.22453e-01 4.61342e-02 8.31957e-01 2.15505e+00 2.64769e-01
3.03477e+00 2.69561e+00 3.16188e+00 3.34497e+00 3.51885e+00 3.65570e+00 3.74856e+00 3.84541e+00 4.22811e+00 3.27734e+00

-3.22480e-01 -2.12804e+00 6.50750e-01 -1.11242e+00 -1.40683e+00 -1.55944e+00 -1.84558e+00 -1.18946e-01 5.52395e-01 -4.36564e-01 -5.30052e+00
4.62594e-01 -3.29127e+00 1.10950e+00 -6.01857e-01 1.29492e-01 1.42290e+00 2.90741e+00 2.44058e+00 -9.19118e-01 8.42851e-01 -4.69824e+00
-2.57424e+00 8.96469e-01 8.34775e-01 2.18845e+00 2.46526e+00 8.60878e-02 -6.88080e-01 -1.33623e-02 9.30403e-02 1.64942e+00 -1.01231e+00
4.81041e+00 1.53747e+00 -1.12216e+00 -3.16008e+00 -1.67404e+00 -1.75767e+00 -1.29310e+00 5.59549e-01 8.08468e-01 -1.01592e-02 -7.54052e+00
1.01933e+01 -2.09484e+01 -1.07426e+00 9.87912e-01 6.05210e-01 -6.04535e-02 -5.87826e-01 -7.94117e-01 -4.89190e-01 -8.12710e-02 -2.07067e+01
-5.31793e+00 7.94240e+00 -4.64165e+00 -4.37436e+00 -1.55417e+00 7.54368e-01 1.09608e+00 1.45967e+00 1.62946e+00 -1.01301e+00 1.13514e-01
2.20336e-01 1.70056e+00 -5.20105e-01 -4.28330e-01 1.57258e-03 -3.36502e-01 -8.18568e-02 -7.16163e+00 8.23195e+00 -1.71561e-02 -1.13749e+01
3.75075e+00 7.25399e+00 -1.75325e+00 -2.68814e+00 -3.71128e+00 -4.62933e+00 -2.13747e+00 -1.89186e-01 1.29122e+00 -7.49380e-01 6.71712e-01
-8.41923e-01 4.64997e+00 5.65808e-01 -3.08277e-01 -1.01687e+00 1.73127e-01 -8.92130e-01 1.89044e+00 -2.75543e-01 -7.72828e-01 5.36745e-01
-3.65598e+00 7.56997e+00 -3.76373e+00 -1.74542e+00 -1.37540e-01 -5.55400e-01 -1.59195e-01 1.27910e-01 1.91906e+00 1.42119e+00 -4.35502e+00

-1.70059e+00 -3.65695e+00 1.22367e+00 -5.74367e-01 -3.29571e+00 2.46316e+00 5.22353e+00 2.42038e+00 1.22919e+00 -9.22250e-01 -2.32028e+00

0.00000e+00
1.00000e+00

run.sh

#!/bin/bash

sex \
   -DETECT_TYPE CCD \
   -DETECT_MINAREA 5 \
   -DETECT_THRESH 1.5 \
   -ANALYSIS_THRESH 1.5 \
   -FILTER Y \
   -DEBLEND_NTHRESH 32 \
   -DEBLEND_MINCONT 0.005 \
   -CLEAN Y \
   -CLEAN_PARAM 1.0 \
   -MASK_TYPE CORRECT \
   -PHOT_APERTURES 5 \
   -PHOT_AUTOPARAMS '2.5, 3.5' \
   -PHOT_PETROPARAMS '2.0, 3.5' \
   -SATUR_LEVEL 50000.0 \
   -SATUR_KEY SATURATE \
   -MAG_ZEROPOINT 31.8 \
   -MAG_GAMMA 4.0 \
   -GAIN 1.0 \
   -GAIN_KEY GAIN \
   -PIXEL_SCALE 0 \
   -SEEING_FWHM 0.8 \
   -BACK_SIZE 64 \
   -BACK_FILTERSIZE 3 \
   -BACKPHOTO_TYPE GLOBAL \
   -MEMORY_OBJSTACK 3000 \
   -MEMORY_PIXSTACK 300000 \
   -MEMORY_BUFSIZE 1024 \
   -HEADER_SUFFIX .head \
   -CHECKIMAGE_NAME '' \
   -CATALOG_TYPE ASCII_HEAD \
   -CHECKIMAGE_TYPE NONE \
   -VERBOSE_TYPE QUIET \
   -CATALOG_NAME out.cat \
   -PARAMETERS_NAME user.param \
   -FILTER_NAME default.conv \
   -STARNNW_NAME default.nnw \
   gc_msx_e.fits

user.param

NUMBER
FLUX_RADIUS
FLUX_AUTO
MAG_AUTO
MAG_WIN

gc_msx_e.fits: Taken from here: https://www.astropy.org/astropy-data/galactic_center/gc_msx_e.fits

uweschmitt commented 1 year ago

Did anybody have a look at this already?

tinhle01 commented 1 year ago

Hi, It does not relate to your topic but I want to ask you how can you install sextractor, I get problem when trying to install atlas. Thank you for reading, I am looking forward to your response.

g-braeunlich commented 1 year ago

I just used the package from ubuntu https://packages.ubuntu.com/source/jammy/atlas / the preinstalled version of atlas on our cluster. Other than that, I dont have experience with installing atlas.

tinhle01 commented 1 year ago

I follow the steps to install atlas 1) mkdir my_build_dir ; cd my_build_dir 2) /path/to/ATLAS/configure [flags] 3) make ! tune and compile library 4) make check ! perform sanity tests 5) make ptcheck ! checks of threaded code for multiprocessor systems 6) make time ! provide performance summary as % of clock rate 7) make install ! Copy library and include files to other directories then in step 2 I put the path of configure then run and get trouble home/isaac/Downloads/atlas_3.10.3.orig/atlas-3.10.3/configure [flags]

gcc -I/home/isaac/Downloads/atlas_3.10.3.orig/atlas-3.10.3//CONFIG/include -g -w -c /home/isaac/Downloads/atlas_3.10.3.orig/atlas-3.10.3//CONFIG/src/atlconf_misc.c gcc -I/home/isaac/Downloads/atlas_3.10.3.orig/atlas-3.10.3//CONFIG/include -g -w -o xconfig /home/isaac/Downloads/atlas_3.10.3.orig/atlas-3.10.3//CONFIG/src/config.c atlconf_misc.o /usr/bin/ld: /tmp/ccLh2Z8e.o: in function ATL_tmpnam': /home/isaac/Downloads/atlas_3.10.3.orig/atlas-3.10.3//CONFIG/include/atlas_sys.h:224: warning: the use oftmpnam' is dangerous, better use `mkstemp' ./xconfig -d s /home/isaac/Downloads/atlas_3.10.3.orig/atlas-3.10.3/ -d b /home/isaac/Downloads/atlas_3.10.3.orig/atlas-3.10.3/my_build_dir [flags]

ERROR around arg 7 ([flags]). USAGE: ./xconfig [flags] where flags are: -v : verbosity level -O <enum OSTYPE #> : set OS type -s <enum ASMDIA #> : set assembly dialect -A <enum MACHTYPE #> : set machine/architecture -V # # = ((1<<vecISA1) | (1<<vecISA2) | ... | (1<<vecISAN)) -b <32/64> : set pointer bitwidth -o -C [xc,ic,if,sk,dk,sm,dm,al,ac] -F [xc,ic,if,sk,dk,sm,dm,al,ac,gc] '' -Fa [xc,ic,if,sk,dk,sm,dm,al,ac,gc] '' al: append flags to all compilers ac: append flags to all C compilers gc: append flags to gcc compiler used in user-contributed index files. acg: append to all C compilers & the index gcc alg: append to all compilers & the index gcc -T : ssh target for cross-compilation (probably broken) -D [c,f] -D= : cpp #define to add to [CDEFS,F2CDEFS] eg. -D c -DL2SIZE=8388604 -D f -DADD__ -D f -DStringSunStyle -d [s,b] : set source/build directory -f <#> : size (in KB) to flush before timing -t <#> : set # of threads (-1: autodect; 0: no threading) -m : set clock rate -S[i/s] : special int/string arg -Si bozol1 <0/1> : supress/enable bozo L1 defaults -Si archdef <1/0> : enable/supress arch default use -Si ieee <1/0> : dis/allow optimizations that break IEEE FP standard (eg., NEON, 3DNow!) -Si latune <1/0> : do/don't tune F77 LAPACK routines -Si nof77 <0/1> : Have/don't have fortran compiler -Si nocygwin <0/1> : Do/don't depend on GPL cygwin library (Windows compiler/cygwin install only) -tl <#> : set # of threads, use list of affinity IDs -Si omp <0/1> : don'tuse/use OpenMP for threading -Si antthr <0/1/2> : nobuild/build/use Antoine's code for threading -Si lapackref <0/1>: Netlib lapack is not/is unpacked to $BLDdir/src/lapack/ref -Ss kern <path/to/comp> : use comp for all kernel compilers -Ss ADdir <path/to/archdefs> : Get archdefs frm custom path -Ss pmake <parallel make invocation (eg '$(MAKE) -j 4')> -Ss f77lib -Ss flapack : used to build full lapack lib -Ss [s,d]maflags 'flags' NOTE: enum #s can be found by : make xprint_enums ; ./xprint_enums

 Can you help me to install that.
g-braeunlich commented 1 year ago

Sorry, as I said: I did not compile it myself and I dont know your system (probably Windows). This problem also does not belong to to this issue.

tinhle01 commented 1 year ago

yes, I use ubuntu 20.04 to install that, so I want to ask you steps you do to install atlas.

g-braeunlich commented 1 year ago

I just used sudo apt-get install atlas. Note: I wont reply any further in this issue about installing atlas

rmathar commented 8 months ago

You seem to be compiling without any gcc optimization flags; at least they are missing in the few gcc log-lines posted 2022-11-14. This probably means that for basically every operation only the 32 or 64bits of the memory representation of the (floating point) numbers will be used. To get the standard implementation of using all bits on your processor registers use at least -O2 .