afeinstein20 / stella

For characterizing flares with convolutional neural networks
https://adina.feinste.in/stella
MIT License
26 stars 18 forks source link

ff.flare_table comes out with fewer flares than the tutorial indicates #9

Open ekaterinailin opened 4 years ago

ekaterinailin commented 4 years ago

Hi @afeinstein20 !

I tried to use the FitFlares feature, but got stuck at the point where I wanted to show the table with the fitted flares.

This is the code I ran (trying to reproduce the Fitting Flares tutorial):

import stella

# Get the light curve

from lightkurve.search import search_lightcurvefile

lk = search_lightcurvefile(target='tic62124646', mission='TESS')
lk = lk.download().PDCSAP_FLUX

# get the model

cnn = stella.ConvNN(output_dir=OUT_DIR)

MODEL_DIR = 'run01/'
MODEL = [os.path.join(MODEL_DIR,i) for i in
          os.listdir(MODEL_DIR) if i.endswith('.h5')][0]

# predict flares from model

cnn.predict(modelname=MODEL,
            times=lk.time,
            fluxes=lk.flux,
            errs=lk.flux_err)

# fit flares

ff = stella.FitFlares(id=[lk.targetid],
                      time=[lk.time],
                      flux=[lk.flux],
                      flux_err=[lk.flux_err],
                      predictions=[cnn.predictions[0]])

# get flare table

ff.identify_flare_peaks(threshold=0.5)

ff.flare_table

The output is a single flares

Target_ID tpeak amp ed_s rise fall prob
62124646.0 1662.6966771763764 1.0031577638536382 2.5355761288160976 0.0028550217295811296 0.008296178315620328 0.976371169090271

which is not what I expected. Based on the cnn.predictions I expected a number of flares similar to what your tutorial shows (16?).

I tried several models from the quickstart collection, but they either return 1 or zero flares in the ff.flare_table.

Do you have an idea what I am doing wrong here?

Cheers :) Ekaterina

afeinstein20 commented 4 years ago

Hi @ekaterinailin ! Weird. Which version of stella are you using? If you're using the version that was pip install stella then I would believe there would be differences between that and the version now on GitHub.

I will be updating the PyPI version once the referee report is good to go and the associated AJ paper is accepted!

ekaterinailin commented 4 years ago

I think I tried both the pip version and the cloned repo. I now reproduced the same result with the most recent master branch version.

afeinstein20 commented 4 years ago

Okay great! Yeah I think there was a flub in me working in different branches, but now there's consistency between them. Please close the issue if you're satisfied with the fix!

ekaterinailin commented 4 years ago

I got a fresh installation from the master branch now, but unfortunately to no avail. I get a bunch of outputs tothe prompt, not sure if related to the actual issue, but perhaps worth posting anyways. Some I expected from the tutorial, others not:

/home/ekaterina/Documents/000_envs/stella/lib/python3.6/site-packages/lightkurve/lightcurvefile.py:47: LightkurveWarning: `LightCurveFile.header` is deprecated, please use `LightCurveFile.get_header()` instead.
  LightkurveWarning)

WARNING: No stella.DataSet object passed in.
Can only use stella.ConvNN.predict().

100%|██████████| 1/1 [00:01<00:00,  1.13s/it]
Finding & Fitting Flares:   0%|          | 0/1 [00:00<?, ?it/s]/home/ekaterina/Documents/000_envs/stella/lib/python3.6/site-packages/stella-0.1.0rc1-py3.6.egg/stella/mark_flares.py:92: VisibleDeprecationWarning: Creating an ndarray from ragged nested sequences (which is a list-or-tuple of lists-or-tuples-or ndarrays with different lengths or shapes) is deprecated. If you meant to do this, you must specify 'dtype=object' when creating the ndarray
  return np.array(results)
/home/ekaterina/Documents/000_envs/stella/lib/python3.6/site-packages/numpy/lib/nanfunctions.py:1665: RuntimeWarning: Degrees of freedom <= 0 for slice.
  keepdims=keepdims)
/home/ekaterina/Documents/000_envs/stella/lib/python3.6/site-packages/numpy/lib/nanfunctions.py:1114: RuntimeWarning: All-NaN slice encountered
  overwrite_input=overwrite_input)
Finding & Fitting Flares: 100%|██████████| 1/1 [00:00<00:00,  5.22it/s]
danirevi5 commented 2 years ago

Hi,

I am having the same problem, i find fewer flares than in the tutorial. I have tried both, the pip and the github installation and they all give the same result.

In other cases, although the star I am working with has many obvious flares, it only detect two small ones but not the obvious ones.

Has there been any change that could be causing this?

Thank you!

aftab234 commented 6 months ago

I get 16 flares as in the tutorial if I don't normalize the light curve and use the remove_nans() function along with the s004 pre-trained model. But the equivalent duration is in seconds now in the most recent version of stella, instead of minutes.

Edit: Not the same 16 flares. There are duplicates! 67 flares, again with many duplicates, if I normalize the light curve.