search

noraeisner / LATTE

Lightcurve Analysis Tool for Transiting Exoplanet
GNU Lesser General Public License v3.0
39 stars 10 forks source link

Hit ValueError during vetting #8

Closed orionlee closed 4 years ago

orionlee commented 4 years ago

In trying to vet TIC 329342112 (see PHT talk thread), I got the following exception while LATTE was processing.

...
Downloading https://mast.stsci.edu/api/v0.1/Download/file/?uri=mast:TESS/product/tess2020133194932-s0025-0000000329342112-0182-s_tp.fits
|===========================================|  45M/ 45M (100.00%)      5m25s
Aperture size plots... done.
Downloading https://skyview.gsfc.nasa.gov/tempspace/fits/skv26484726564662.fits
|===========================================| 371k/371k (100.00%)         3s
Star Aperture plots... done.
Traceback (most recent call last):
  File "C:\pkg\Anaconda3\envs\forLATTE\lib\runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "C:\pkg\Anaconda3\envs\forLATTE\lib\runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "c:\dev\_juypter\LATTE\LATTE\__main__.py", line 426, in <module>
    utils.interact_LATTE(tic, indir, syspath, sectors_all, sectors, ra, dec, args)  # the argument of whether to shos the images or not
  File "c:\dev\_juypter\LATTE\LATTE\LATTEutils.py", line 661, in interact_LATTE
    brew.brew_LATTE(tic, indir, syspath, transit_list, simple, BLS, model, save, DV, sectors, sectors_all, alltime, allflux, allflux_err, all_md, alltimebinned, allfluxbinned, allx1, allx2, ally1, ally2, alltime12, allfbkg, start_sec, end_sec, in_sec, tessmag, teff, srad, ra, dec, args)
  File "c:\dev\_juypter\LATTE\LATTE\LATTEbrew.py", line 237, in brew_LATTE
    X1_list, X4_list, oot_list, intr_list, bkg_list, apmask_list, arrshape_list, t_list, T0_list, tpf_filt_list = utils.download_tpf_mast(indir, transit_sec, transit_list, tic)
  File "c:\dev\_juypter\LATTE\LATTE\LATTEutils.py", line 3765, in download_tpf_mast
    trends = pca.fit_transform(X2)
  File "C:\pkg\Anaconda3\envs\forLATTE\lib\site-packages\sklearn\decomposition\_pca.py", line 376, in fit_transform
    U, S, V = self._fit(X)
  File "C:\pkg\Anaconda3\envs\forLATTE\lib\site-packages\sklearn\decomposition\_pca.py", line 425, in _fit
    return self._fit_truncated(X, n_components, self._fit_svd_solver)
  File "C:\pkg\Anaconda3\envs\forLATTE\lib\site-packages\sklearn\decomposition\_pca.py", line 545, in _fit_truncated
    random_state=random_state)
  File "C:\pkg\Anaconda3\envs\forLATTE\lib\site-packages\sklearn\utils\validation.py", line 72, in inner_f
    return f(**kwargs)
  File "C:\pkg\Anaconda3\envs\forLATTE\lib\site-packages\sklearn\utils\extmath.py", line 358, in randomized_svd
    Uhat, s, V = linalg.svd(B, full_matrices=False)
  File "C:\pkg\Anaconda3\envs\forLATTE\lib\site-packages\scipy\linalg\decomp_svd.py", line 106, in svd
    a1 = _asarray_validated(a, check_finite=check_finite)
  File "C:\pkg\Anaconda3\envs\forLATTE\lib\site-packages\scipy\_lib\_util.py", line 263, in _asarray_validated
    a = toarray(a)
  File "C:\pkg\Anaconda3\envs\forLATTE\lib\site-packages\numpy\lib\function_base.py", line 486, in asarray_chkfinite
    "array must not contain infs or NaNs")
ValueError: array must not contain infs or NaNs

It seemed to have created some array with infs or NaNs in making the following call in download_tpf_mast: https://github.com/noraeisner/LATTE/blob/f5423bcd0bcb7f26f593576c27dbea95db879201/LATTE/LATTEutils.py#L3765

The stacktrace is above generated from the latest LATTE codes in github. The same error happened with the released v1.0.0.

orionlee commented 4 years ago

Same error for the following input

TICID,sectors,transits,BLS,model,FFI
441808796,24,"1960.84,1976.32",,0,no
AJAX6255 commented 4 years ago

Sam: When I run the command: python -m tess_stars2px -t 441808796 I get the following table - so shouldn't your command be closer to: 441808796,24,"1231.69,1427.97",,0,no I am assuming that the two bold numbers are ColPix and RowPix. (latte) c:\Users\Allan\anaconda3\Lib\site-packages>python -m tess_stars2px -t 441808796

TIC | RA | Dec | EclipticLong | EclipticLat | Sector |

Camera | Ccd | ColPix | RowPix 441808796 | 266.495444 | 72.636084 | 99.733711 | 83.805616 | 14 | 3 | 2 | 493.363 | 1827.968 441808796 | 266.495444 | 72.636084 | 99.733711 | 83.805616 | 15 | 3 | 2 | 858.063 | 1502.832 441808796 | 266.495444 | 72.636084 | 99.733711 | 83.805616 | 16 | 3 | 2 | 1042.773 | 1054.941 441808796 | 266.495444 | 72.636084 | 99.733711 | 83.805616 | 17 | 4 | 2 | 1001.833 | 1788.701 441808796 | 266.495444 | 72.636084 | 99.733711 | 83.805616 | 18 | 4 | 2 | 769.184 | 1379.916 441808796 | 266.495444 | 72.636084 | 99.733711 | 83.805616 | 19 | 4 | 2 | 372.624 | 1108.236 441808796 | 266.495444 | 72.636084 | 99.733711 | 83.805616 | 20 | 4 | 1 | 2036.296 | 1040.201 441808796 | 266.495444 | 72.636084 | 99.733711 | 83.805616 | 21 | 4 | 1 | 1540.190 | 1227.923 441808796 | 266.495444 | 72.636084 | 99.733711 | 83.805616 | 22 | 4 | 1 | 1196.217 | 1634.838 441808796 | 266.495444 | 72.636084 | 99.733711 | 83.805616 | 24 | 3 | 1 | 1231.690 | 1427.971 441808796 | 266.495444 | 72.636084 | 99.733711 | 83.805616 | 25 | 3 | 1 | 1561.377 | 1777.150 441808796 | 266.495444 | 72.636084 | 99.733711 | 83.805616 | 26 | 3 | 1 | 1989.022 | 1943.922

I also have had trouble running LATTE on Anaconda3 Windows 10. I would love to run it on the cloud and not on a local machine, (specifically SciServer or Kaggle) but I can't get python tkinter to work when using a docker container.

On Fri, Aug 21, 2020 at 9:55 AM Sam Lee notifications@github.com wrote:

Same error for the following input

TICID,sectors,transits,BLS,model,FFI 441808796,24,"1960.84,1976.32",,0,no

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/noraeisner/LATTE/issues/8#issuecomment-678389792, or unsubscribe https://github.com/notifications/unsubscribe-auth/AI6H6K2PTDIMCDLO4JJU7SLSB2RHDANCNFSM4QCBA47Q .

orionlee commented 4 years ago

I get the following table - so shouldn't your command be closer to: 441808796,24,"1231.69,1427.97",,0,no I am assuming that the two bold numbers are ColPix and RowPix.

The two numbers in the input csv "1960.84,1976.32" are proposed transit epoch that I supplied to LATTE, not ColPix/RowPIx.

The issue I raised here seems to indicate there during the processing of target pixel files, the current logic seems not to handle some cases where some intermediary contain infs or NaNs.

AJAX6255 commented 4 years ago

[image: 20200822_195637_resized.jpg] YES. After three weeks of experiments I have results with tessLATTE! It needed to install it on a Ubuntu 20.04 OS. I also used anaconda3. This worked like a charm. Now I am in a position to try and repeat Sam Lee's issue. Thanks Nora.

On Fri, Aug 21, 2020 at 3:11 PM Allan Jackson allan.jackson6255@gmail.com wrote:

Sam: When I run the command: python -m tess_stars2px -t 441808796 I get the following table - so shouldn't your command be closer to: 441808796,24,"1231.69,1427.97",,0,no I am assuming that the two bold numbers are ColPix and RowPix. (latte) c:\Users\Allan\anaconda3\Lib\site-packages>python -m tess_stars2px -t 441808796

TIC | RA | Dec | EclipticLong | EclipticLat | Sector |

Camera | Ccd | ColPix | RowPix 441808796 | 266.495444 | 72.636084 | 99.733711 | 83.805616 | 14 | 3 | 2 | 493.363 | 1827.968 441808796 | 266.495444 | 72.636084 | 99.733711 | 83.805616 | 15 | 3 | 2 | 858.063 | 1502.832 441808796 | 266.495444 | 72.636084 | 99.733711 | 83.805616 | 16 | 3 | 2 | 1042.773 | 1054.941 441808796 | 266.495444 | 72.636084 | 99.733711 | 83.805616 | 17 | 4 | 2 | 1001.833 | 1788.701 441808796 | 266.495444 | 72.636084 | 99.733711 | 83.805616 | 18 | 4 | 2 | 769.184 | 1379.916 441808796 | 266.495444 | 72.636084 | 99.733711 | 83.805616 | 19 | 4 | 2 | 372.624 | 1108.236 441808796 | 266.495444 | 72.636084 | 99.733711 | 83.805616 | 20 | 4 | 1 | 2036.296 | 1040.201 441808796 | 266.495444 | 72.636084 | 99.733711 | 83.805616 | 21 | 4 | 1 | 1540.190 | 1227.923 441808796 | 266.495444 | 72.636084 | 99.733711 | 83.805616 | 22 | 4 | 1 | 1196.217 | 1634.838 441808796 | 266.495444 | 72.636084 | 99.733711 | 83.805616 | 24 | 3 | 1 | 1231.690 | 1427.971 441808796 | 266.495444 | 72.636084 | 99.733711 | 83.805616 | 25 | 3 | 1 | 1561.377 | 1777.150 441808796 | 266.495444 | 72.636084 | 99.733711 | 83.805616 | 26 | 3 | 1 | 1989.022 | 1943.922

I also have had trouble running LATTE on Anaconda3 Windows 10. I would love to run it on the cloud and not on a local machine, (specifically SciServer or Kaggle) but I can't get python tkinter to work when using a docker container.

On Fri, Aug 21, 2020 at 9:55 AM Sam Lee notifications@github.com wrote:

Same error for the following input

TICID,sectors,transits,BLS,model,FFI 441808796,24,"1960.84,1976.32",,0,no

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/noraeisner/LATTE/issues/8#issuecomment-678389792, or unsubscribe https://github.com/notifications/unsubscribe-auth/AI6H6K2PTDIMCDLO4JJU7SLSB2RHDANCNFSM4QCBA47Q .

orionlee commented 4 years ago

The ValueError appears to be an installation issue that is probably specific to Windows (or some subset of Windows): during the installation via pip, some dependent packages are installed with bad version (that aren't quite completely working).

The workaround I used:

  1. Before installing LATTE, first use conda to install the following packages (conda installation does a better job of skipping potentially bad versions, comparing to pip)

    conda install --channel=conda-forge numpy scikit-learn scipy

  2. Then follow the normal installation instruction

    pip install tessLATTE

  3. On Windows or Anaconda environment, you'd need to ensure python3 exists. See https://github.com/noraeisner/LATTE/issues/7

orionlee commented 4 years ago

If LATTE is available on conda-forge, then the issue can probably be resolved without any workaround.