maindlt
commented
1 month ago Hi @avichaturvedi,
thanks for reaching out!
It is a bit strange that the nan messages in the log (which originate from calculating the gh moments from the orbit libraries before invoking the weight solver) only occur for orbit no. 0. This lets me guess that there might be an issue with starting to count from 0 or 1. On the other hand, why does this not occur in your limit to 25 arcsec...?
Could you perhaps provide me with the data and config file (per email or Slack so it is kept out of GitHub)? Then I can reproduce (with a smaller orbit library if necessary), debug and find out what's happening...
Thanks and have a great day, Thomas
avichaturvedi
Dear Dynamite team (@maindlt),
I am running dynamite modelling on a galaxy that has kinematics extending out to around 60 arcsec. When I am using the full available kinematics out to 60 arcsec, the dynamite fails (see the error message from the run at the bottom) whereas, on running models within 25 arcsec, the models runs fine.
This is exactly what I meant : Image1 : Here is the velocity map extending out to around 60 arcsec, In this case, the models try to run the nnls but its produces 'NaN' output in 'nn_nnls.out' and 'nn_kinem.out' file.
Image2: Screenshot of attached 'nn_nnls.out' file.
If I use kinematics maps limited to 25 arcsec then the models work fine. Image3:
Can you please help me to sort out this error? I think it has something to do with the spatial extension of the kinematical map? For your reference here, I have attached the log file as well. dynamite.log
Many thanks for your help and suggestions.
Regards Avinash
############### Error message from job run #####
The following have been reloaded with a version change: 1) openmpi4/4.1.1 => openmpi4/4.1.6
[INFO] 11:18:42 - dynamite.config_reader.Configuration - Config file IC4296_const_config_run21100703.yaml read. [INFO] 11:18:42 - dynamite.config_reader.Configuration - io_settings... [INFO] 11:18:42 - dynamite.config_reader.Configuration - Output directory tree: Output211007/. [INFO] 11:18:42 - dynamite.config_reader.Configuration - system_attributes... [INFO] 11:18:42 - dynamite.config_reader.Configuration - model_components... [WARNING] 11:18:42 - dynamite.mges.MGE - changing q=1.0 to q=0.9999 for numerical stability. [WARNING] 11:18:42 - dynamite.mges.MGE - New mge: I sigma q PA_twist
73217.772016 0.01729 0.96411 0.0 77279.338488 1.129681 0.906801 0.0 31479.949768 2.986875 0.904637 0.0 8388.363024 7.365392 0.90585 0.0 2736.037653 18.781604 0.884365 0.0 680.577546 40.507354 0.979502 0.0 175.839183 121.59474 0.9999 0.0 [WARNING] 11:18:42 - dynamite.mges.MGE - changing q=1.0 to q=0.9999 for numerical stability. [WARNING] 11:18:42 - dynamite.mges.MGE - New mge: I sigma q PA_twist
73217.772016 0.01729 0.96411 0.0 77279.338488 1.129681 0.906801 0.0 31479.949768 2.986875 0.904637 0.0 8388.363024 7.365392 0.90585 0.0 2736.037653 18.781604 0.884365 0.0 680.577546 40.507354 0.979502 0.0 175.839183 121.59474 0.9999 0.0 [INFO] 11:18:42 - dynamite.config_reader.Configuration - system_parameters... [INFO] 11:18:42 - dynamite.config_reader.Configuration - orblib_settings... [INFO] 11:18:42 - dynamite.config_reader.Configuration - weight_solver_settings... [INFO] 11:18:42 - dynamite.config_reader.Configuration - Will attempt to recover partially run models. [INFO] 11:18:42 - dynamite.config_reader.Configuration - parameter_space_settings... [INFO] 11:18:42 - dynamite.config_reader.Configuration - legacy_settings... [INFO] 11:18:42 - dynamite.config_reader.Configuration - multiprocessing_settings... [INFO] 11:18:42 - dynamite.config_reader.Configuration - ... using 40 CPUs for orbit integration. [INFO] 11:18:42 - dynamite.config_reader.Configuration - ... using 20 CPUs for weight solving. [INFO] 11:18:42 - dynamite.config_reader.Configuration - System assembled [INFO] 11:18:42 - dynamite.config_reader.Configuration - Configuration validated [INFO] 11:18:42 - dynamite.config_reader.Configuration - Instantiated parameter space [INFO] 11:18:42 - dynamite.model.AllModels - No previous models (file Output211007/all_models.ecsv) have been found: Made an empty table in AllModels.table [INFO] 11:18:42 - dynamite.config_reader.Configuration - Instantiated AllModels object [INFO] 11:18:42 - dynamite.model.AllModels - No all_models table update required. [INFO] 11:18:42 - dynamite.model_iterator.ModelIterator - LegacyGridSearch: iterations 0 and 1 [INFO] 11:18:42 - dynamite.parameter_space.LegacyGridSearch - LegacyGridSearch added 1 new model(s) out of 1 [INFO] 11:18:42 - dynamite.parameter_space.LegacyGridSearch - LegacyGridSearch added 8 new model(s) out of 8 [INFO] 11:18:43 - dynamite.model_iterator.SplitModelIterator - ... running model 1 out of 9 [INFO] 11:18:43 - dynamite.model_iterator.SplitModelIterator - ... running model 2 out of 9 [INFO] 11:18:43 - dynamite.orblib.LegacyOrbitLibrary - Calculating initial conditions [INFO] 11:18:43 - dynamite.model_iterator.SplitModelIterator - ... running model 3 out of 9 [INFO] 11:18:43 - dynamite.orblib.LegacyOrbitLibrary - Calculating initial conditions [INFO] 11:18:43 - dynamite.model_iterator.SplitModelIterator - ... running model 4 out of 9 [INFO] 11:18:43 - dynamite.orblib.LegacyOrbitLibrary - Calculating initial conditions [INFO] 11:18:43 - dynamite.model_iterator.SplitModelIterator - ... running model 5 out of 9 [INFO] 11:18:43 - dynamite.orblib.LegacyOrbitLibrary - Calculating initial conditions [INFO] 11:18:43 - dynamite.model_iterator.SplitModelIterator - ... running model 6 out of 9 [INFO] 11:18:43 - dynamite.orblib.LegacyOrbitLibrary - Calculating initial conditions [INFO] 11:18:43 - dynamite.model_iterator.SplitModelIterator - ... running model 7 out of 9 [INFO] 11:18:43 - dynamite.orblib.LegacyOrbitLibrary - Calculating initial conditions [INFO] 11:18:43 - dynamite.orblib.LegacyOrbitLibrary - Calculating initial conditions [INFO] 11:23:24 - dynamite.orblib.LegacyOrbitLibrary - ...done - cmd_orb_start exit code 0. Logfile: Output211007/models/orblib_001_005/datfil/orbstart.log. [INFO] 11:23:24 - dynamite.orblib.LegacyOrbitLibrary - Integrating orbit library tube and box orbits [INFO] 11:23:30 - dynamite.orblib.LegacyOrbitLibrary - ...done - cmd_orb_start exit code 0. Logfile: Output211007/models/orblib_001_001/datfil/orbstart.log. [INFO] 11:23:30 - dynamite.orblib.LegacyOrbitLibrary - Integrating orbit library tube and box orbits [INFO] 11:23:33 - dynamite.orblib.LegacyOrbitLibrary - ...done - cmd_orb_start exit code 0. Logfile: Output211007/models/orblib_000_000/datfil/orbstart.log. [INFO] 11:23:33 - dynamite.orblib.LegacyOrbitLibrary - Integrating orbit library tube and box orbits [INFO] 11:23:35 - dynamite.orblib.LegacyOrbitLibrary - ...done - cmd_orb_start exit code 0. Logfile: Output211007/models/orblib_001_002/datfil/orbstart.log. [INFO] 11:23:35 - dynamite.orblib.LegacyOrbitLibrary - Integrating orbit library tube and box orbits [INFO] 11:23:38 - dynamite.orblib.LegacyOrbitLibrary - ...done - cmd_orb_start exit code 0. Logfile: Output211007/models/orblib_001_003/datfil/orbstart.log. [INFO] 11:23:38 - dynamite.orblib.LegacyOrbitLibrary - Integrating orbit library tube and box orbits [INFO] 11:23:49 - dynamite.orblib.LegacyOrbitLibrary - ...done - cmd_orb_start exit code 0. Logfile: Output211007/models/orblib_001_000/datfil/orbstart.log. [INFO] 11:23:49 - dynamite.orblib.LegacyOrbitLibrary - Integrating orbit library tube and box orbits [INFO] 11:24:07 - dynamite.orblib.LegacyOrbitLibrary - ...done - cmd_orb_start exit code 0. Logfile: Output211007/models/orblib_001_004/datfil/orbstart.log. [INFO] 11:24:07 - dynamite.orblib.LegacyOrbitLibrary - Integrating orbit library tube and box orbits [INFO] 12:19:24 - dynamite.orblib.LegacyOrbitLibrary - ...done - cmd_tube_box_orbs exit code 0. Logfiles: Output211007/models/orblib_001_000/datfil/orblib.log, Output211007/models/orblib_001_000/datfil/orblibbox.log, Output211007/models/orblib_001_000/datfil/triaxmass.log, Output211007/models/orblib_001_000/datfil/triaxmassbin.log. [INFO] 12:19:24 - dynamite.model_iterator.SplitModelIterator - Model 2: Output211007/models/orblib_001_000/ml04.50/model_done_staging.ecsv written. [INFO] 12:19:44 - dynamite.orblib.LegacyOrbitLibrary - ...done - cmd_tube_box_orbs exit code 0. Logfiles: Output211007/models/orblib_000_000/datfil/orblib.log, Output211007/models/orblib_000_000/datfil/orblibbox.log, Output211007/models/orblib_000_000/datfil/triaxmass.log, Output211007/models/orblib_000_000/datfil/triaxmassbin.log. [INFO] 12:19:44 - dynamite.model_iterator.SplitModelIterator - Model 1: Output211007/models/orblib_000_000/ml04.50/model_done_staging.ecsv written. [INFO] 12:20:06 - dynamite.orblib.LegacyOrbitLibrary - ...done - cmd_tube_box_orbs exit code 0. Logfiles: Output211007/models/orblib_001_005/datfil/orblib.log, Output211007/models/orblib_001_005/datfil/orblibbox.log, Output211007/models/orblib_001_005/datfil/triaxmass.log, Output211007/models/orblib_001_005/datfil/triaxmassbin.log. [INFO] 12:20:06 - dynamite.model_iterator.SplitModelIterator - Model 7: Output211007/models/orblib_001_005/ml04.50/model_done_staging.ecsv written. [INFO] 12:20:29 - dynamite.orblib.LegacyOrbitLibrary - ...done - cmd_tube_box_orbs exit code 0. Logfiles: Output211007/models/orblib_001_002/datfil/orblib.log, Output211007/models/orblib_001_002/datfil/orblibbox.log, Output211007/models/orblib_001_002/datfil/triaxmass.log, Output211007/models/orblib_001_002/datfil/triaxmassbin.log. [INFO] 12:20:29 - dynamite.model_iterator.SplitModelIterator - Model 4: Output211007/models/orblib_001_002/ml04.50/model_done_staging.ecsv written. [INFO] 12:21:16 - dynamite.orblib.LegacyOrbitLibrary - ...done - cmd_tube_box_orbs exit code 0. Logfiles: Output211007/models/orblib_001_003/datfil/orblib.log, Output211007/models/orblib_001_003/datfil/orblibbox.log, Output211007/models/orblib_001_003/datfil/triaxmass.log, Output211007/models/orblib_001_003/datfil/triaxmassbin.log. [INFO] 12:21:16 - dynamite.model_iterator.SplitModelIterator - Model 5: Output211007/models/orblib_001_003/ml04.50/model_done_staging.ecsv written. [INFO] 12:21:59 - dynamite.orblib.LegacyOrbitLibrary - ...done - cmd_tube_box_orbs exit code 0. Logfiles: Output211007/models/orblib_001_001/datfil/orblib.log, Output211007/models/orblib_001_001/datfil/orblibbox.log, Output211007/models/orblib_001_001/datfil/triaxmass.log, Output211007/models/orblib_001_001/datfil/triaxmassbin.log. [INFO] 12:21:59 - dynamite.model_iterator.SplitModelIterator - Model 3: Output211007/models/orblib_001_001/ml04.50/model_done_staging.ecsv written. [INFO] 12:22:22 - dynamite.orblib.LegacyOrbitLibrary - ...done - cmd_tube_box_orbs exit code 0. Logfiles: Output211007/models/orblib_001_004/datfil/orblib.log, Output211007/models/orblib_001_004/datfil/orblibbox.log, Output211007/models/orblib_001_004/datfil/triaxmass.log, Output211007/models/orblib_001_004/datfil/triaxmassbin.log. [INFO] 12:22:22 - dynamite.model_iterator.SplitModelIterator - Model 6: Output211007/models/orblib_001_004/ml04.50/model_done_staging.ecsv written. [INFO] 12:22:22 - dynamite.model_iterator.SplitModelIterator - 7 orblib(s) calculated. [INFO] 12:22:22 - dynamite.model_iterator.SplitModelIterator - ... running model 1 out of 9 [INFO] 12:22:22 - dynamite.model_iterator.SplitModelIterator - ... running model 2 out of 9 [INFO] 12:22:22 - dynamite.weight_solvers.LegacyWeightSolver - Using WeightSolver: LegacyWeightSolver [INFO] 12:22:22 - dynamite.weight_solvers.LegacyWeightSolver - Fitting orbit library to the kinematic data: ml04.50 [INFO] 12:22:22 - dynamite.model_iterator.SplitModelIterator - ... running model 3 out of 9 [INFO] 12:22:22 - dynamite.weight_solvers.LegacyWeightSolver - Using WeightSolver: LegacyWeightSolver [INFO] 12:22:22 - dynamite.weight_solvers.LegacyWeightSolver - Fitting orbit library to the kinematic data: ml04.50 [INFO] 12:22:22 - dynamite.model_iterator.SplitModelIterator - ... running model 4 out of 9 [INFO] 12:22:22 - dynamite.weight_solvers.LegacyWeightSolver - Using WeightSolver: LegacyWeightSolver [INFO] 12:22:22 - dynamite.weight_solvers.LegacyWeightSolver - Fitting orbit library to the kinematic data: ml04.50 [INFO] 12:22:22 - dynamite.model_iterator.SplitModelIterator - ... running model 5 out of 9 [INFO] 12:22:22 - dynamite.weight_solvers.LegacyWeightSolver - Using WeightSolver: LegacyWeightSolver [INFO] 12:22:22 - dynamite.weight_solvers.LegacyWeightSolver - Fitting orbit library to the kinematic data: ml04.50 [INFO] 12:22:22 - dynamite.model_iterator.SplitModelIterator - ... running model 6 out of 9 [INFO] 12:22:22 - dynamite.weight_solvers.LegacyWeightSolver - Using WeightSolver: LegacyWeightSolver [INFO] 12:22:22 - dynamite.weight_solvers.LegacyWeightSolver - Fitting orbit library to the kinematic data: ml04.50 [INFO] 12:22:22 - dynamite.model_iterator.SplitModelIterator - ... running model 7 out of 9 [INFO] 12:22:22 - dynamite.weight_solvers.LegacyWeightSolver - Using WeightSolver: LegacyWeightSolver [INFO] 12:22:22 - dynamite.weight_solvers.LegacyWeightSolver - Fitting orbit library to the kinematic data: ml04.50 [INFO] 12:22:22 - dynamite.model_iterator.SplitModelIterator - ... running model 8 out of 9 [INFO] 12:22:22 - dynamite.weight_solvers.LegacyWeightSolver - Using WeightSolver: LegacyWeightSolver [INFO] 12:22:22 - dynamite.weight_solvers.LegacyWeightSolver - Fitting orbit library to the kinematic data: ml04.50 [INFO] 12:22:22 - dynamite.model_iterator.SplitModelIterator - ... running model 9 out of 9 [INFO] 12:22:22 - dynamite.weight_solvers.LegacyWeightSolver - Using WeightSolver: LegacyWeightSolver [INFO] 12:22:22 - dynamite.weight_solvers.LegacyWeightSolver - Fitting orbit library to the kinematic data: ml03.50 [INFO] 12:22:22 - dynamite.weight_solvers.LegacyWeightSolver - Using WeightSolver: LegacyWeightSolver [INFO] 12:22:22 - dynamite.weight_solvers.LegacyWeightSolver - Fitting orbit library to the kinematic data: ml05.50 [INFO] 12:27:12 - dynamite.weight_solvers.LegacyWeightSolver - ...done, NNLS problem solved - cmd_nnls_4.5 exit code 0. Logfile: Output211007/models/orblib_001_005/ml04.50/nnls.log. [INFO] 12:27:12 - dynamite.config_reader.Configuration - Config file copied to Output211007/models/orblib_001_005/ml04.50/IC4296_const_config_run21100703.yaml. [INFO] 12:27:21 - dynamite.weight_solvers.LegacyWeightSolver - ...done, NNLS problem solved - cmd_nnls_5.5 exit code 0. Logfile: Output211007/models/orblib_000_000/ml05.50/nnls.log. [INFO] 12:27:21 - dynamite.weight_solvers.LegacyWeightSolver - ...done, NNLS problem solved - cmd_nnls_3.5 exit code 0. Logfile: Output211007/models/orblib_000_000/ml03.50/nnls.log. [INFO] 12:27:21 - dynamite.config_reader.Configuration - Config file copied to Output211007/models/orblib_000_000/ml03.50/IC4296_const_config_run21100703.yaml. [INFO] 12:27:21 - dynamite.config_reader.Configuration - Config file copied to Output211007/models/orblib_000_000/ml05.50/IC4296_const_config_run21100703.yaml. [INFO] 12:27:23 - dynamite.weight_solvers.LegacyWeightSolver - ...done, NNLS problem solved - cmd_nnls_4.5 exit code 0. Logfile: Output211007/models/orblib_001_001/ml04.50/nnls.log. [INFO] 12:27:23 - dynamite.config_reader.Configuration - Config file copied to Output211007/models/orblib_001_001/ml04.50/IC4296_const_config_run21100703.yaml. [INFO] 12:27:24 - dynamite.weight_solvers.LegacyWeightSolver - ...done, NNLS problem solved - cmd_nnls_4.5 exit code 0. Logfile: Output211007/models/orblib_001_000/ml04.50/nnls.log. [INFO] 12:27:24 - dynamite.config_reader.Configuration - Config file copied to Output211007/models/orblib_001_000/ml04.50/IC4296_const_config_run21100703.yaml. [INFO] 12:27:28 - dynamite.analysis.Analysis - Getting model projected masses and losvds. [INFO] 12:27:29 - dynamite.weight_solvers.LegacyWeightSolver - ...done, NNLS problem solved - cmd_nnls_4.5 exit code 0. Logfile: Output211007/models/orblib_000_000/ml04.50/nnls.log. [INFO] 12:27:29 - dynamite.config_reader.Configuration - Config file copied to Output211007/models/orblib_000_000/ml04.50/IC4296_const_config_run21100703.yaml. [INFO] 12:27:30 - dynamite.weight_solvers.LegacyWeightSolver - ...done, NNLS problem solved - cmd_nnls_4.5 exit code 0. Logfile: Output211007/models/orblib_001_002/ml04.50/nnls.log. [INFO] 12:27:30 - dynamite.config_reader.Configuration - Config file copied to Output211007/models/orblib_001_002/ml04.50/IC4296_const_config_run21100703.yaml. [INFO] 12:27:33 - dynamite.weight_solvers.LegacyWeightSolver - ...done, NNLS problem solved - cmd_nnls_4.5 exit code 0. Logfile: Output211007/models/orblib_001_003/ml04.50/nnls.log. [INFO] 12:27:33 - dynamite.config_reader.Configuration - Config file copied to Output211007/models/orblib_001_003/ml04.50/IC4296_const_config_run21100703.yaml. [INFO] 12:27:35 - dynamite.analysis.Analysis - Getting model projected masses and losvds. [INFO] 12:27:36 - dynamite.weight_solvers.LegacyWeightSolver - ...done, NNLS problem solved - cmd_nnls_4.5 exit code 0. Logfile: Output211007/models/orblib_001_004/ml04.50/nnls.log. [INFO] 12:27:36 - dynamite.config_reader.Configuration - Config file copied to Output211007/models/orblib_001_004/ml04.50/IC4296_const_config_run21100703.yaml. [INFO] 12:27:36 - dynamite.analysis.Analysis - Getting model projected masses and losvds. [INFO] 12:27:38 - dynamite.analysis.Analysis - Getting model projected masses and losvds. [INFO] 12:27:39 - dynamite.analysis.Analysis - Getting model projected masses and losvds. [INFO] 12:27:43 - dynamite.analysis.Analysis - Getting model projected masses and losvds. [INFO] 12:27:45 - dynamite.analysis.Analysis - Getting model projected masses and losvds. [INFO] 12:27:48 - dynamite.analysis.Analysis - Getting model projected masses and losvds. [INFO] 12:27:50 - dynamite.analysis.Analysis - Getting model projected masses and losvds. /home/achaturv/.conda/envs/avipy38/lib/python3.8/site-packages/dynamite-4.2.0-py3.8.egg/dynamite/kinematics.py:830: RuntimeWarning: invalid value encountered in divide mean /= norm [INFO] 12:36:25 - dynamite.kinematics.Histogram - orbit=0, aperture=0: mean or sigma is nan. [INFO] 12:36:25 - dynamite.kinematics.Histogram - orbit=0, aperture=1: mean or sigma is nan. [INFO] 12:36:25 - dynamite.kinematics.Histogram - orbit=0, aperture=2: mean or sigma is nan. [INFO] 12:36:25 - dynamite.kinematics.Histogram - orbit=0, aperture=3: mean or sigma is nan. [INFO] 12:36:25 - dynamite.kinematics.Histogram - orbit=0, aperture=4: mean or sigma is nan. [INFO] 12:36:25 - dynamite.kinematics.Histogram - orbit=0, aperture=5: mean or sigma is nan. [INFO] 12:36:25 - dynamite.kinematics.Histogram - orbit=0, aperture=6: mean or sigma is nan. [INFO] 12:36:25 - dynamite.kinematics.Histogram - orbit=0, aperture=7: mean or sigma is nan. [INFO] 12:36:25 - dynamite.kinematics.Histogram - orbit=0, aperture=8: mean or sigma is nan. [INFO] 12:36:25 - dynamite.kinematics.Histogram - orbit=0, aperture=9: mean or sigma is nan.