timothydmorton / isochrones

Pythonic stellar model grid access; easy MCMC fitting of stellar properties
http://isochrones.readthedocs.org
MIT License
123 stars 63 forks source link

Nosetests crashes because of memory issues #172

Open sjshamsi opened 2 years ago

sjshamsi commented 2 years ago

Hi! I hope you're well. I recently installed isochromes and upon running nosetests isochromes, I get the following output:

(astrolab_hrd) sshamsi@SHAMSI-LAP:~$ nosetests isochrones
/home/sshamsi/miniconda3/envs/astrolab_hrd/lib/python3.9/site-packages/isochrones/tests/test_fits.py:6: FutureWarning: pandas.util.testing is deprecated. Use the functions in the public API at pandas.testing instead.
  from pandas.util.testing import assert_frame_equal
/home/sshamsi/miniconda3/envs/astrolab_hrd/lib/python3.9/site-packages/pandas/core/arraylike.py:397: RuntimeWarning: divide by zero encountered in log10
  result = getattr(ufunc, method)(*inputs, **kwargs)
/home/sshamsi/miniconda3/envs/astrolab_hrd/lib/python3.9/site-packages/isochrones/models.py:148: SettingWithCopyWarning:
A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_indexer,col_indexer] = value instead

See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy
  subdf.loc[:, "dm_deep"] = deriv
/home/sshamsi/miniconda3/envs/astrolab_hrd/lib/python3.9/site-packages/tables/attributeset.py:459: NaturalNameWarning: object name is not a valid Python identifier: 'axis1_name[Fe/H]'; it does not match the pattern ``^[a-zA-Z_][a-zA-Z0-9_]*$``; you will not be able to use natural naming to access this object; using ``getattr()`` will still work, though
  check_attribute_name(name)
interpolating missing values in evolution tracks (feh=-4.0)': 100%|████████████████████████████████| 196/196 [00:00<00:00, 866.28it/s]
interpolating missing values in evolution tracks (feh=-3.5)': 100%|████████████████████████████████| 196/196 [00:00<00:00, 196.93it/s]
interpolating missing values in evolution tracks (feh=-3.0)': 100%|████████████████████████████████| 196/196 [00:01<00:00, 129.69it/s]
interpolating missing values in evolution tracks (feh=-2.5)': 100%|████████████████████████████████| 196/196 [00:01<00:00, 110.23it/s]
interpolating missing values in evolution tracks (feh=-2.0)': 100%|█████████████████████████████████| 196/196 [00:01<00:00, 99.46it/s]
interpolating missing values in evolution tracks (feh=-1.75)': 100%|███████████████████████████████| 196/196 [00:00<00:00, 253.08it/s]
interpolating missing values in evolution tracks (feh=-1.5)': 100%|████████████████████████████████| 196/196 [00:00<00:00, 290.92it/s]
interpolating missing values in evolution tracks (feh=-1.25)': 100%|███████████████████████████████| 196/196 [00:00<00:00, 342.94it/s]
interpolating missing values in evolution tracks (feh=-1.0)': 100%|████████████████████████████████| 196/196 [00:00<00:00, 319.67it/s]
interpolating missing values in evolution tracks (feh=-0.75)': 100%|███████████████████████████████| 196/196 [00:00<00:00, 320.18it/s]
interpolating missing values in evolution tracks (feh=-0.5)': 100%|████████████████████████████████| 196/196 [00:00<00:00, 331.73it/s]
interpolating missing values in evolution tracks (feh=-0.25)': 100%|███████████████████████████████| 196/196 [00:00<00:00, 362.00it/s]
interpolating missing values in evolution tracks (feh=0.0)': 100%|█████████████████████████████████| 196/196 [00:00<00:00, 293.63it/s]
interpolating missing values in evolution tracks (feh=0.25)': 100%|████████████████████████████████| 196/196 [00:01<00:00, 140.77it/s]
interpolating missing values in evolution tracks (feh=0.5)': 100%|██████████████████████████████████| 196/196 [00:02<00:00, 74.35it/s]
Killed
(astrolab_hrd) sshamsi@SHAMSI-LAP:~$ nosetests isochrones
/home/sshamsi/miniconda3/envs/astrolab_hrd/lib/python3.9/site-packages/isochrones/tests/test_fits.py:6: FutureWarning: pandas.util.testing is deprecated. Use the functions in the public API at pandas.testing instead.
  from pandas.util.testing import assert_frame_equal
...Killed

I tried installing Multinest but that then chashed without any output.

The logs suggest an OOM issue. Is this a normal amount of memory to use?

[ 1069.646175] oom_reaper: reaped process 226 (nosetests), now anon-rss:0kB, file-rss:0kB, shmem-rss:0kB
[ 2021.171512] nosetests invoked oom-killer: gfp_mask=0x100dca(GFP_HIGHUSER_MOVABLE|__GFP_ZERO), order=0, oom_score_adj=0
[ 2021.171515] CPU: 2 PID: 251 Comm: nosetests Not tainted 5.10.16.3-microsoft-standard-WSL2 #1
[ 2021.171520] Call Trace:
[ 2021.171648]  dump_stack+0x57/0x6a
[ 2021.171683]  dump_header+0x4a/0x196
[ 2021.171701]  oom_kill_process.cold+0xb/0x10
[ 2021.171705]  out_of_memory+0x1fb/0x2a0
[ 2021.171733]  __alloc_pages_slowpath.constprop.0+0x86f/0xc30
[ 2021.171769]  __alloc_pages_nodemask+0x258/0x2c0
[ 2021.171772]  handle_mm_fault+0xf8b/0x1630
[ 2021.171806]  do_user_addr_fault+0x1b1/0x3e0
[ 2021.171825]  exc_page_fault+0x63/0x110
[ 2021.171857]  ? asm_exc_page_fault+0x8/0x30
[ 2021.171859]  asm_exc_page_fault+0x1e/0x30
[ 2021.171910] RIP: 0033:0x7ff489e8d9e6
[ 2021.171922] Code: Unable to access opcode bytes at RIP 0x7ff489e8d9bc.
[ 2021.171923] RSP: 002b:00007ffe1519d5c8 EFLAGS: 00010206
[ 2021.171928] RAX: 00007ff2edf00568 RBX: 00007ffe1519d680 RCX: 00007ff2ee0f8e20
[ 2021.171929] RDX: 000000000002edc0 RSI: 00007ff3aae2a478 RDI: 00007ff2ee0ca000
[ 2021.171945] RBP: 00007ffe1519d780 R08: ffffffffffffffe8 R09: 00007ffe1519d648
[ 2021.171948] R10: 00007ff2ee0f8e40 R11: 00007ff2edf00568 R12: 00007ffe1519d650
[ 2021.171949] R13: 00007ffe1519d630 R14: 00007ffe1519d880 R15: 00007ffe1519d980
[ 2021.171951] Mem-Info:
[ 2021.171954] active_anon:755122 inactive_anon:142056 isolated_anon:32
                active_file:97 inactive_file:9 isolated_file:0
                unevictable:0 dirty:0 writeback:36
                slab_reclaimable:6638 slab_unreclaimable:7806
                mapped:11 shmem:10 pagetables:2691 bounce:0
                free:20906 free_pcp:500 free_cma:0
[ 2021.171959] Node 0 active_anon:3020232kB inactive_anon:568224kB active_file:388kB inactive_file:36kB unevictable:0kB isolated(anon):128kB isolated(file):0kB mapped:44kB dirty:0kB writeback:144kB shmem:40kB shmem_thp: 0kB shmem_pmdmapped: 0kB anon_thp: 73728kB writeback_tmp:0kB kernel_stack:2768kB all_unreclaimable? no
[ 2021.171964] DMA free:14884kB min:256kB low:320kB high:384kB reserved_highatomic:0KB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB writepending:0kB present:14972kB managed:14884kB mlocked:0kB pagetables:0kB bounce:0kB free_pcp:0kB local_pcp:0kB free_cma:0kB
[ 2021.171967] lowmem_reserve[]: 0 3791 3813 3813 3813
[ 2021.171971] DMA32 free:67176kB min:66936kB low:83668kB high:100400kB reserved_highatomic:0KB active_anon:3020332kB inactive_anon:567544kB active_file:724kB inactive_file:132kB unevictable:0kB writepending:408kB present:4046848kB managed:3887112kB mlocked:0kB pagetables:10428kB bounce:0kB free_pcp:2000kB local_pcp:496kB free_cma:0kB
[ 2021.171974] lowmem_reserve[]: 0 0 21 21 21
[ 2021.171978] Normal free:1564kB min:384kB low:480kB high:576kB reserved_highatomic:2048KB active_anon:16kB inactive_anon:0kB active_file:0kB inactive_file:12kB unevictable:0kB writepending:0kB present:30720kB managed:22480kB mlocked:0kB pagetables:336kB bounce:0kB free_pcp:0kB local_pcp:0kB free_cma:0kB
[ 2021.171981] lowmem_reserve[]: 0 0 0 0 0
[ 2021.171983] DMA: 1*4kB (U) 0*8kB 0*16kB 1*32kB (U) 2*64kB (U) 1*128kB (U) 1*256kB (U) 0*512kB 0*1024kB 1*2048kB (M) 3*4096kB (M) = 14884kB
[ 2021.171991] DMA32: 131*4kB (ME) 62*8kB (E) 174*16kB (UE) 223*32kB (UE) 188*64kB (UE) 90*128kB (UME) 52*256kB (UME) 28*512kB (UME) 5*1024kB (UME) 0*2048kB 0*4096kB = 67260kB
[ 2021.172000] Normal: 1*4kB (H) 2*8kB (H) 2*16kB (H) 5*32kB (H) 3*64kB (H) 1*128kB (H) 0*256kB 2*512kB (H) 0*1024kB 0*2048kB 0*4096kB = 1556kB
[ 2021.172007] Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=1048576kB
[ 2021.172008] Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=2048kB
[ 2021.172009] 278 total pagecache pages
[ 2021.172010] 161 pages in swap cache
[ 2021.172011] Swap cache stats: add 1252221, delete 1251949, find 363173/418998
[ 2021.172012] Free swap  = 0kB
[ 2021.172013] Total swap = 1048576kB
[ 2021.172014] 1023135 pages RAM
[ 2021.172015] 0 pages HighMem/MovableOnly
[ 2021.172016] 42016 pages reserved
[ 2021.172017] Tasks state (memory values in pages):
[ 2021.172017] [  pid  ]   uid  tgid total_vm      rss pgtables_bytes swapents oom_score_adj name
[ 2021.172024] [    142]     0   142      397        0    24576       19             0 gns
[ 2021.172025] [    145]     0   145      412        0    40960       13             0 localhost
[ 2021.172026] [    154]     0   154      416        0    40960       17             0 init
[ 2021.172027] [    159]     0   159      436        0    36864       18             0 init
[ 2021.172029] [    163]     0   163      414        0    40960       16             0 init
[ 2021.172029] [    164]     0   164     3510        0    65536      175             0 WSLGd
[ 2021.172031] [    166]  1000   166   197724        0   245760     5335             0 weston
[ 2021.172032] [    167]  1000   167      391        0    28672       15             0 mstsc.exe
[ 2021.172032] [    168]    18   168     1791        0    53248      125             0 dbus-daemon
[ 2021.172033] [    169]  1000   169    58475        0    86016      754             0 pulseaudio
[ 2021.172034] [    173]  1000   173     1758        0    49152       92             0 dbus-daemon
[ 2021.172035] [    182]     0   182      438        0    36864       20             0 init
[ 2021.172051] [    183]     0   183      438        0    36864       28             0 init
[ 2021.172053] [    184]  1000   184     2510        1    65536      448             0 bash
[ 2021.172054] [    242]  1000   242    63230        1   315392     2309             0 Xwayland
[ 2021.172055] [    251]  1000   251  1551289   896931  9969664   252786             0 nosetests
[ 2021.172056] oom-kill:constraint=CONSTRAINT_NONE,nodemask=(null),cpuset=/,mems_allowed=0,global_oom,task_memcg=/,task=nosetests,pid=251,uid=1000
[ 2021.172166] Out of memory: Killed process 251 (nosetests) total-vm:6205156kB, anon-rss:3587720kB, file-rss:4kB, shmem-rss:0kB, UID:1000 pgtables:9736kB oom_score_adj:0

Thank you!