avivajpeyi commented 2 years ago

All TOIs:

[114, 119, 124, 137, 147, 174, 175, 176, 187, 189, 191, 199, 201, 204, 206, 211, 214, 216, 217, 218, 240, 274, 275, 276, 282, 283, 284, 285, 286, 289, 385, 389, 412, 419, 428, 448, 512, 532, 533, 602, 604, 609, 623, 633, 634, 635, 639, 644, 669, 673, 678, 692, 699, 700, 704, 711, 712, 713, 714, 718, 721, 723, 731, 735, 765, 779, 780, 785, 786, 787, 788, 789, 793, 794, 796, 797, 798, 800, 801, 802, 804, 807, 809, 810, 817, 827, 831, 866, 867, 868, 872, 878, 888, 898, 920, 936, 966, 971, 974, 975, 976, 992, 1028, 1051, 1131, 1136, 1138, 1141, 1152, 1172, 1178, 1205, 1207, 1209, 1210, 1212, 1216, 1219, 1222, 1223, 1227, 1228, 1246, 1250, 1252, 1253, 1254, 1269, 1292, 1306, 1338, 1346, 1347, 1418, 1430, 1438, 1439, 1447, 1486, 1487, 1488, 1489, 1555, 1589, 1609, 1633, 1640, 1678, 1689, 1706, 1729, 1741, 1747, 1783, 1791, 1816, 1819, 1847, 1941, 1942, 2003, 2006, 2014, 2077, 2078, 2079, 2090, 2091, 2095, 2098, 2099, 2220, 2250, 2269, 2277, 2281, 2283, 2309, 2313, 2328, 2388, 2423, 2472, 2477, 2525, 2534, 2606, 2607, 3353, 3358, 3460, 3487, 3493, 3494, 4307, 4339, 4343, 4344, 4348, 4355, 4356, 4357, 4405, 4465, 4495, 4503, 4507, 4544, 4555, 4558, 4580, 5126, 5130, 5145, 5376, 5387, 5391, 5523, 5564]

24 / 219 TOIs: ConnectionResetError

This is sus -- why is it taking so long to sample?

avivajpeyi commented 2 years ago

Some of the others that dont have the error don't even look like they finished executing... EG 137

avivajpeyi commented 2 years ago

Maybe the easiest thing to do would be to restart the jobs, and then check again if these are the same TOIs with the issues?

avivajpeyi commented 2 years ago

For ref, code to get the TOIs without netcdfs

import glob
import tess_atlas
import pandas as pd
import os

def get_toi_number(path):
    toi_str = path.split("/")[-2].split("_")[1]
    return int(toi_str)

root = "jun11_cat"
notebook_root = f"{root}/0.2.1.dev64+gc7fa3a0"

tois = pd.read_csv(f"{root}/tois.csv", index_col="Unnamed: 0")
netcdfs_files = glob.glob(f"{notebook_root}/toi_*_files/*.netcdf")
notebooks = glob.glob(f"{notebook_root}/toi_*.ipynb")

tois_with_netcdf = [get_toi_number(f) for f in netcdfs_files]
has_netcdf = [True if i in tois_with_netcdf else False for i in tois.toi_numbers.values]
tois['has_netcdf']= has_netcdf
assert sum(tois.has_netcdf)==len(netcdfs_files)

print(f"num TOIs to analyse: {len(tois)}")
print(f"num notebooks: {len(notebooks)}")
print(f"num netcdfs: {len(netcdfs_files)}")
print(f"num tois without netcdfs: {len(tois)-len(netcdfs_files) }")
print("TOIs without netcdfs:")
print(tois[tois['has_netcdf']==False].toi_numbers.values.tolist())

avivajpeyi commented 2 years ago

Some of these TOIs have lightcurves with over .5M data points -- sampling can take more than 5hrs for these!

Dan had some thoughts:

'clip' data to only include lc data near the eclipse (see exoplanet's eclipsing binary tutorial)
increase the duration for which we run these specific analyses? (eg if data longer than XXX datapoints, have an independent slurm job for these (with a longer duration)

avivajpeyi commented 2 years ago

Changes made in #225 helped with this! seems like most are now finishing -- 25 / 219 analyses are still timing out. Closing for now

dfm / tess-atlas

219 analyses arnt able to finish sampling #214

24 / 219 TOIs: ConnectionResetError