SYNAPPS fails (but SYN++ doesn't)

HeloiseS commented 8 years ago

Hello!

I am using Ubuntu 16.04, syn++ runs fine and gives me outputs, Synapps on the other hand doesn't, it keeps crashing and returns what I have pasted below. Could you help me figure it out? I don't really understand what's going on or how to fix it.

Thank you!

APPSPACK: Asynchronous Parallel Pattern Search Written by T. G. Kolda et al., Sandia National Labs For more information visit http://software.sandia.gov/appspack

###########################################

APPSPACK Initialization Results

Parameter List Add Projected Compass = false [default] Add Projected Normals = true [default] Bounds Tolerance = 0.005 [default] Cache Comparison Tolerance = 0.005 [default] Cache File Precision = 14 [default] Cache Input File = "target.cache" Cache Output File = "target.cache" Contraction Factor = 0.5 [default] Debug = 3 [default] Epsilon Max = 0.02 [default] Initial Step = 1 [default] Initial X = [ 1.00e+00 -2.60e+00 -5.00e+00 1.07e+01 3.00e+01 1.14e+01 1.14e+00 1.07e+01 3.00e+01 1.35e+00 2.02e+01 ] Max Queue Size = 0 [default] Maximum Exchange Return = 1000 [default] Minimum Exchange Return = 1 [default] Minimum Step = 0.02 [default] Precision = 2 Snap To Boundary = true [default] Step Tolerance = 0.01 [default] Sufficient Decrease Factor = 0.01 [default] Synchronous = false [default] Use Projected Compass = true [unused] Use Random Order = true

Constraints

Bound Constraints lower = [ 0.00e+00 -1.00e+01 -1.00e+01 5.00e+00 1.50e+01 5.00e+00 -2.00e+00 5.00e+00 1.50e+01 1.00e-01 5.00e+00 ] upper = [ 1.00e+01 1.00e+01 1.00e+01 1.50e+01 3.00e+01 2.50e+01 2.00e+00 1.50e+01 3.00e+01 5.00e+00 2.50e+01 ] scaling = [ 1.00e+01 2.00e+01 2.00e+01 1.00e+01 1.00e+00 2.00e+01 1.00e+00 1.00e+00 1.00e+00 1.00e+00 1.00e+00 ]

Inequality Constraints Inequality matrix = [ 0.00e+00 0.00e+00 0.00e+00 -1.00e+00 1.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00 -1.00e+00 0.00e+00 0.00e+00 0.00e+00 1.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00 -1.00e+00 1.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00 1.00e+00 0.00e+00 0.00e+00 0.00e+00 -1.00e+00 0.00e+00 0.00e+00 ] Inequality lower = [ 0.00e+00 0.00e+00 0.00e+00 0.00e+00 ] Inequality upper = [ DNE DNE DNE DNE ]

Equality Constraints Equality matrix = [ 0.00e+00 0.00e+00 0.00e+00 -1.00e+00 0.00e+00 0.00e+00 0.00e+00 1.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00 1.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00 1.00e+00 0.00e+00 0.00e+00 ] Equality bound = [ 0.00e+00 3.00e+01 3.00e+01 ] Problem size summary: Number of variables: 11 Number of variable bounds: 22 (11 lower, 11 upper) Number of inequality constraints: 4 (4 lower, 0 upper) Number of equality constraints: 3

Conveyor

Using MPI Executor with 1 workers

End APPSPACK Initialization Results

###########################################

New Min: f= DNE x=[ 1.00e+00 -2.60e+00 -5.00e+00 1.07e+01 3.00e+01 1.14e+01 1.14e+00 1.07e+01 3.00e+01 1.35e+00 2.02e+01 ] step=1.00e+00 tag=0 state=Evaluated (Initial Point) dim. of lineality space: 8 no. of extreme rays in pointy part: 0

New Min: f= -nan x=[ 1.00e+00 -2.60e+00 -5.00e+00 1.07e+01 3.00e+01 5.00e+00 1.14e+00 1.07e+01 3.00e+01 1.35e+00 2.02e+01 ] step=1.00e+00 tag=1 state=Cached dim. of lineality space: 7 no. of extreme rays in pointy part: 1

Final State: Step Converged

Final Min: f= -nan x=[ 1.00e+00 -2.60e+00 -5.00e+00 1.07e+01 3.00e+01 5.00e+00 1.14e+00 1.07e+01 3.00e+01 1.35e+00 2.02e+01 ] step=1.00e+00 tag=1 state=Cached

Number of Evaluations : 1 Number of Cached Evaluations: 127 Evaluation Breakdown by Message Type... Success: 1 Evaluation Breakdown by Processor and Message Type... Worker #1 Success: 1 Final Directions: 0 : d = [ 1.00e+01 0.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00 ] step = 7.81e-03 1 : d = [ 0.00e+00 2.00e+01 0.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00 ] step = 7.81e-03 2 : d = [ 0.00e+00 0.00e+00 2.00e+01 0.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00 ] step = 7.81e-03 3 : d = [ 0.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00 1.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00 ] step = 7.81e-03 4 : d = [ 0.00e+00 0.00e+00 0.00e+00 9.95e-01 0.00e+00 0.00e+00 0.00e+00 9.95e-01 0.00e+00 0.00e+00 0.00e+00 ] step = 7.81e-03 5 : d = [ 0.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00 1.00e+00 0.00e+00 ] step = 7.81e-03 6 : d = [ 0.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00 1.00e+00 ] step = 7.81e-03 7 : d = [-1.00e+01 -0.00e+00 -0.00e+00 -0.00e+00 -0.00e+00 -0.00e+00 -0.00e+00 -0.00e+00 -0.00e+00 -0.00e+00 -0.00e+00 ] step = 7.81e-03 8 : d = [-0.00e+00 -2.00e+01 -0.00e+00 -0.00e+00 -0.00e+00 -0.00e+00 -0.00e+00 -0.00e+00 -0.00e+00 -0.00e+00 -0.00e+00 ] step = 7.81e-03 9 : d = [-0.00e+00 -0.00e+00 -2.00e+01 -0.00e+00 -0.00e+00 -0.00e+00 -0.00e+00 -0.00e+00 -0.00e+00 -0.00e+00 -0.00e+00 ] step = 7.81e-03 10 : d = [-0.00e+00 -0.00e+00 -0.00e+00 -0.00e+00 -0.00e+00 -0.00e+00 -1.00e+00 -0.00e+00 -0.00e+00 -0.00e+00 -0.00e+00 ] step = 7.81e-03 11 : d = [-0.00e+00 -0.00e+00 -0.00e+00 -9.95e-01 -0.00e+00 -0.00e+00 -0.00e+00 -9.95e-01 -0.00e+00 -0.00e+00 -0.00e+00 ] step = 7.81e-03 12 : d = [-0.00e+00 -0.00e+00 -0.00e+00 -0.00e+00 -0.00e+00 -0.00e+00 -0.00e+00 -0.00e+00 -0.00e+00 -1.00e+00 -0.00e+00 ] step = 7.81e-03 13 : d = [-0.00e+00 -0.00e+00 -0.00e+00 -0.00e+00 -0.00e+00 -0.00e+00 -0.00e+00 -0.00e+00 -0.00e+00 -0.00e+00 -1.00e+00 ] step = 7.81e-03 14 : d = [ 0.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00 2.00e+01 0.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00 ] step = 7.81e-03 15 : d = [ 0.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00 -2.00e+01 0.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00 ] step = 5.00e-03 16 : d = [ 8.29e-16 0.00e+00 0.00e+00 8.24e-18 0.00e+00 0.00e+00 0.00e+00 8.24e-18 0.00e+00 0.00e+00 0.00e+00 ] step = 7.81e-03 Number of times directions calculated by... LAPACK: 0 CDDLIB: 2 Cached: 0 Max directions in single iteration : 18 Number of times directions appended: 0

Thank you! APPSPACK: Asynchronous Parallel Pattern Search Written by T. G. Kolda et al., Sandia National Labs For more information visit http://software.sandia.gov/appspack

terminate called after throwing an instance of 'std::bad_alloc' what(): std::bad_alloc [Lola:21249] Process received signal [Lola:21249] Signal: Aborted (6) [Lola:21249] Signal code: (-6) [Lola:21249] [ 0] /lib/x86_64-linux-gnu/libc.so.6(+0x354b0)[0x2b45dee7b4b0] [Lola:21249] [ 1] /lib/x86_64-linux-gnu/libc.so.6(gsignal+0x38)[0x2b45dee7b428] [Lola:21249] [ 2] /lib/x86_64-linux-gnu/libc.so.6(abort+0x16a)[0x2b45dee7d02a] [Lola:21249] [ 3] /usr/lib/x86_64-linux-gnu/libstdc++.so.6(_ZN9gnu_cxx27__verbose_terminate_handlerEv+0x16d)[0x2b45de41284d] [Lola:21249] [ 4] /usr/lib/x86_64-linux-gnu/libstdc++.so.6(+0x8d6b6)[0x2b45de4106b6] [Lola:21249] [ 5] /usr/lib/x86_64-linux-gnu/libstdc++.so.6(+0x8d701)[0x2b45de410701] [Lola:21249] [ 6] /usr/lib/x86_64-linux-gnu/libstdc++.so.6(+0x8d919)[0x2b45de410919] [Lola:21249] [ 7] /usr/lib/x86_64-linux-gnu/libstdc++.so.6(+0xb6082)[0x2b45de439082] [Lola:21249] [ 8] /usr/local/lib/libappspack.so.0(+0x47579)[0x2b45dd09f579] [Lola:21249] [ 9] synapps[0x40fa25] [Lola:21249] [10] /lib/x86_64-linux-gnu/libc.so.6(libc_start_main+0xf0)[0x2b45dee66830] [Lola:21249] [11] synapps[0x422a19] [Lola:21249] End of error message

mpirun noticed that process rank 0 with PID 21249 on node Lola exited on signal 6 (Aborted).

rcthomas commented 8 years ago

Some things to look at:

Is the target spectrum you are fitting a 3-column (wavelength, flux, flux error) ASCII file?
Have your fluxes been rescaled to be about order unity, with the corresponding flux errors rescaled appropriately as well?

Check those things, if neither of them apply I'll need to see the configuration file you used, and possibly your spectrum file to debug further. If you'd rather not share those over github let me know.

HeloiseS commented 8 years ago

thanks for the quick reply! Yeah O checked the target spectrum and it's as described, I did rescale the spectrum and the error is sqrt(signal). Here is the synapps.yaml file used:

grid : bin_width : 0.3 # opacity bin size in kkm/s v_size : 100 # size of line-forming region grid v_outer_max : 30.0 # fastest ejecta velocity in kkm/s opacity : line_dir : /usr/local/share/es/lines # path to atomic line data ref_file : /usr/local/share/es/refs.dat # path to ref. line data form : exp # parameterization (only exp for now) v_ref : 10.0 # reference velocity for parameterization log_tau_min : -2.0 # opacity threshold source : mu_size : 10 # number of angles for source integration spectrum : p_size : 60 # number of phot. impact parameters for spectrum flatten : No # divide out continuum or not evaluator : target_file : "target.dat" # spectrum to fit (format: wl, flux, flux_error) vector_norm : 2 # objective function norm regions : apply : [ Yes, No, Yes ] # fit this wavelength region or not weight : [ 0, 1, 1 ] # weight the region by this number lower : [ 0, 3900, 5600 ] # min. wavelength for region definition upper : [ 10000, 4100, 6400 ] # max. wavelength for region definition config : fit_file : "target.fit" # when done, put answer here cache_file : "target.cache" # evaluated point cache

# Various bounds and scaling for parameters, see syn++.yaml example 
# for definition of each parameter.  Parameters can be fixed to a 
# set value --- others can be made to float.

# Make note of the "detach" setting --- if set to Yes then the ion
# will have a v_min equal to v_phot.

a0          :  { fixed:  No, start:    1, lower:   0, upper: 10, scale: 10 }
a1          :  { fixed:  No, start: -2.6, lower: -10, upper: 10, scale: 20 }
a2          :  { fixed:  No, start: -5.0, lower: -10, upper: 10, scale: 20 }
v_phot      :  { fixed:  No, start: 10.7, lower:   5, upper: 15, scale: 10 }
v_outer     :  { fixed: Yes, start:   30, lower:  15, upper: 30, scale:  1 }
t_phot      :  { fixed:  No, start: 11.4, lower:   5, upper: 25, scale: 20 }

ions        :  [ 1401, 1100, ]
active      :  [  Yes,   No, ]
detach      :  [   No,   No, ] 

log_tau     :               
    fixed   :  [   No,   No, ]
    start   :  [ 1.14,    0, ]
    lower   :  [   -2,   -2, ]
    upper   :  [    2,    2, ]
    scale   :  [    1,    1, ]
v_min       :               
    fixed   :  [   No,   No, ]
    start   :  [ 10.7,   10, ]
    lower   :  [    5,    5, ]
    upper   :  [   15,   15, ]
    scale   :  [    1,    1, ]
v_max       :               
    fixed   :  [  Yes,  Yes, ]
    start   :  [   30,   30, ]
    lower   :  [   15,   15, ]
    upper   :  [   30,   30, ]
    scale   :  [    1,    1, ]
aux         :               
    fixed   :  [   No,   No, ]
    start   :  [ 1.35,  1.6, ]
    lower   :  [  0.1,  0.1, ]
    upper   :  [    5,    5, ]
    scale   :  [    1,    1, ]
temp        :               
    fixed   :  [   No,   No, ]
    start   :  [ 20.2,   10, ]
    lower   :  [    5,    5, ]
    upper   :  [   25,   25, ]
    scale   :  [    1,    1, ]

rcthomas commented 8 years ago

OK, looks like the example synapps.yaml file that comes with the package. Just checking a couple things here:

The file is really called target.dat (not something else) and it's located at the path stated here in the synapps.yaml file?
The contents of target.dat --- you say error is sqrt(signal) (more flux is larger uncertainty?); in any case can you verify all the values are really numbers and not NaN (as if your signal had gone negative someplace maybe)?

Running out of ideas. Usually the bad_alloc thing is about not being able to read the target spectrum file, like the path to it is wrong, or it's empty, or something similar.

HeloiseS commented 8 years ago

-Yeah the file is called target.dat and it's int he same folder as the synapps.yaml file -yeah I know the errors aren't right but extracting the error spectrum from iraf is a pain in the bum so i left it until i got the program working, i thought poisson noise was good enough but maybe not. -None of the values are NaN thanks for the help :)

rcthomas commented 8 years ago

Darn. If you're willing to email me the target spectrum file (I won't scoop you, I promise) then I can see if I can spot anything and/or test it here.

HeloiseS commented 8 years ago

if I don't tell you what the target is there's no way you can scoop me :p but I'd rather send it to you by e-mail if you don't mind.

rcthomas commented 8 years ago

That's good, you can use my work email (rcthomas@lbl.gov).

HeloiseS commented 8 years ago

Actually I'm sorry, I can't really do that without the approval of my supervisor (I'm a PhD student), but thank you for offering! I'll get back to you if I get the green light.

Thank you again so much for all the help. If there are some developments (or the code somehow fixes itself) I'll update the thread.

Bye!

rcthomas / es