guillochon / MOSFiT

Modular Open Source Fitter for Transients
http://mosfit.readthedocs.io/
MIT License
41 stars 53 forks source link

ssl certificates expired on sne.space #210

Closed JohannesBuchner closed 2 years ago

JohannesBuchner commented 2 years ago

The SSL certificate of https://sne.space/ expired last night (Wed, 07 Sep 2022 21:05:34 GMT)

This leads to mosfit examples failing with:

Warning: Could not downloadOSCnames (are you online?), using cached list.

which is occuring in fetcher.py line 107 due to urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: certificate has expired (_ssl.c:997)

Full output:


± mosfit -m slsn -e CSS110406:135058+261642
╔═════════════════════════════════════════════════════════════════════════════════════════╗
║ ;@@@#.@        :@@;       ``@@@@@            :@@@@@                      ::::,. `       ║
║ `@@@@@@'      @@@@@@@    @@@@@@@@@@':       @@@@@@@@@  @@@@@@@@@@@@@.    @@@@@@@@@@@@@@ ║
║ @@@@@@@     @@@@@@@#   #@@@@@@@@@@@@@,    @@@@@@@@@@@  @@@@@@@',`          @@@@@@@'     ║
║ `@@@@@@@     '@@@@@@   `@@@@@@    .@@@@   @@@@@     @@; .@@@@'       @@@@    @@@@@@     ║
║ @@@@@@#    '@@@@@@@   @@@@@       `;@@;  @@@@@      @.  @@@@@@@@@@  @@@;    @@@@@       ║
║ @@@@@@@@` @@@@@@@@   #@@@@         @@@    @@@@@@;       @@@:        #@@`    @@@@@`      ║
║ @@@@@@@@ #@@@@ @@@   @@@@@@       ;@@@`    .@@@@@@@'    @@@@       `@@@      @@@@       ║
║ @@@@ `@@@@@@   @@@   `@@@@@@      @@@@        #@@@@@'    @@@        @@@      @@@@       ║
║ @@@',@@@@@@   @@@    @@@@@@@    @@@    ;@@    .@@@@@@  '@@@        :@@     `@@@@        ║
║ ;@@@   @@@:   ,@@@      @@@@@@@@@@@     ;@@@@@@@@@@@    `@                   @'         ║
║ ;@@@   ,@#     @@#       ;@@@@@@@,       @@;@@@@@`                                      ║
╚═════════════════════════════════════════════════════════════════════════════════════════╝

                     ### MOSFiT -- Version 1.1.8 (4ee5977208fbbc32) ###                      
                         Authored by James Guillochon & Matt Nicholl                         
                 Additional contributions from Brenna Mockler & Ashley Villar                
                                Released under the MIT license                               

                         Code: https://github.com/guillochon/MOSFiT                          
                        Documentation: https://mosfit.readthedocs.io                         

Copying MOSFiT folder hierarchy to current working directory (disable with `--no-copy-at-launch`).
Event `CSS110406:135058+261642` interpreted as a transient name, downloading lists of transient
aliases...
<urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: certificate has expired (_ssl.c:997)> https://sne.space/astrocats/astrocats/supernovae/output/names.min.json
Warning: Could not download `OSC` names (are you online?), using cached list.
<urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: certificate has expired (_ssl.c:997)> https://tde.space/astrocats/astrocats/tidaldisruptions/output/names.min.json
Warning: Could not download `OTC` names (are you online?), using cached list.
HTTP Error 403: Forbidden https://kilonova.space/astrocats/astrocats/kilonovae/output/names.min.json
Warning: Could not download `OKC` names (are you online?), using cached list.
Warning: Could not read list of OSC names!
Warning: Could not read list of OTC names!
Warning: Could not read list of OKC names!
Traceback (most recent call last):
  File "/home/user/.local/bin/mosfit", line 33, in <module>
    sys.exit(load_entry_point('mosfit', 'console_scripts', 'mosfit')())
  File "/mnt/data/daten/PostDoc2/home/Downloads/MOSFiT/mosfit/main.py", line 974, in main
    Fitter(**fitargs).fit_events(**fitargs)
  File "/mnt/data/daten/PostDoc2/home/Downloads/MOSFiT/mosfit/fitter.py", line 307, in fit_events
    fetched_events = self._fetcher.fetch(
  File "/mnt/data/daten/PostDoc2/home/Downloads/MOSFiT/mosfit/fetcher.py", line 146, in fetch
    for name in self._names[catalog]:
KeyError: 'OSC'
JohannesBuchner commented 2 years ago

It seems even fixing this, the server is giving 500 errors and not delivering files.

mnicholl commented 2 years ago

Unfortunately the Open Supernova Catalog has been down for some time and is no longer maintained. Mosfit can still be run on local data, so for now the solution is to find the object you need on the Github pages for the Open Catalogs.

E.g: https://github.com/astrocatalogs/sne-2010-2014/blob/master/SN2011ke.json

Download and run with mostfit -e ./SN2011ke.json

JohannesBuchner commented 2 years ago

I see, thank you. I think it could be possible to rewire the code to download from Github. In any case, perhaps the docs at https://mosfit.readthedocs.io/en/latest/fitting.html#public-data could be updated to state that the instructions are outdated.

See #212.

mnicholl commented 2 years ago

The problem isn’t so much the amount of work. Unfortunately since James left nobody is responsible for maintaining Astrocats, so I’m not sure who knows how to do it (not me!) or who would have any necessary permissions

On 12 Sep 2022, at 12:24, Nikhil Sarin @.***> wrote:

Is it really too much work to renew the SSL certificate? That way the API with OACAPI https://github.com/astrocatalogs/OACAPI https://github.com/astrocatalogs/OACAPI would still work and restore all the other functionality. The website itself can remain broken.

— Reply to this email directly, view it on GitHub https://github.com/guillochon/MOSFiT/issues/210#issuecomment-1243597801, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABZ74QQKENBAGC64UMNRAO3V54HHBANCNFSM6AAAAAAQHQBQVI. You are receiving this because you commented.

guillochon commented 2 years ago

Hi all, apologies for this, my email reminder to renew the certs (as the renewal has to be done manually) got eaten by internet gremlins. I've renewed them now.

@JohannesBuchner there was a catastrophic database failure that killed the front end of the OSC/sister sites a few months back. This however did not kill the API, which is still active indefinitely. I have heard of two separate efforts to bring a replacement solution online but I do not know the timetables, until then my intention is to keep the API accessible.

JohannesBuchner commented 2 years ago

The server issue mentioned in https://github.com/guillochon/MOSFiT/issues/210#issuecomment-1240622963 still persist: https://sne.space/astrocats/astrocats/supernovae/output/names.min.json gives "Error establishing a database connection" (Error code 500) which leads to the following error:

Event `SN2011ke` interpreted as a transient name, downloading lists of transient aliases...
Warning: Could not download `OSC` names (are you online?), using cached list.
Warning: Could not download `OTC` names (are you online?), using cached list.
Warning: Could not download `OKC` names (are you online?), using cached list.
Warning: Could not read list of OSC names!
Warning: Could not read list of OTC names!
Warning: Could not read list of OKC names!
Traceback (most recent call last):
  File "/home/user/.local/bin/mosfit", line 33, in <module>
    sys.exit(load_entry_point('mosfit', 'console_scripts', 'mosfit')())
  File "/mnt/data/daten/PostDoc2/home/Downloads/MOSFiT/mosfit/main.py", line 973, in main
    Fitter(**fitargs).fit_events(**fitargs)
  File "/mnt/data/daten/PostDoc2/home/Downloads/MOSFiT/mosfit/fitter.py", line 307, in fit_events
    fetched_events = self._fetcher.fetch(
  File "/mnt/data/daten/PostDoc2/home/Downloads/MOSFiT/mosfit/fetcher.py", line 145, in fetch
    for name in self._names[catalog]:
KeyError: 'OSC'

when running mosfit -m slsn -e SN2011ke

Perhaps you have a cached name list on your computer and do not encounter the issue.

guillochon commented 2 years ago

Ah this is a different error, the JSON files are not being properly served. I'll take a look later today.

guillochon commented 2 years ago

J/k I think I already fixed it, haha. Try now.

JohannesBuchner commented 2 years ago

https://sne.space/astrocats/astrocats/supernovae/output/names.min.json causes a download, which contains:

<?php
/**
 * Front to the WordPress application. This file doesn't do anything, but loads
 * wp-blog-header.php which does and tells WordPress to load the theme.
 *
 * @package WordPress
 */

/**
 * Tells WordPress to load the WordPress theme and output it.
 *
 * @var bool
 */
define('WP_USE_THEMES', true);

/** Loads the WordPress Environment and Template */
require( dirname( __FILE__ ) . '/wp-blog-header.php' );

https://tde.space/astrocats/astrocats/tidaldisruptions/output/names.min.json -> 403 forbidden

https://kilonova.space/astrocats/astrocats/kilonovae/output/names.min.json -> html page

The new error is:

Event `SN2011ke` interpreted as a transient name, downloading lists of transient aliases...
https://sne.space/astrocats/astrocats/supernovae/output/names.min.json
https://tde.space/astrocats/astrocats/tidaldisruptions/output/names.min.json
Warning: Could not download `OTC` names (are you online?), using cached list.
https://kilonova.space/astrocats/astrocats/kilonovae/output/names.min.json
Warning: Could not download `OKC` names (are you online?), using cached list.
Traceback (most recent call last):
  File "/home/user/.local/bin/mosfit", line 33, in <module>
    sys.exit(load_entry_point('mosfit', 'console_scripts', 'mosfit')())
  File "/mnt/data/daten/PostDoc2/home/Downloads/MOSFiT/mosfit/main.py", line 973, in main
    Fitter(**fitargs).fit_events(**fitargs)
  File "/mnt/data/daten/PostDoc2/home/Downloads/MOSFiT/mosfit/fitter.py", line 307, in fit_events
    fetched_events = self._fetcher.fetch(
  File "/mnt/data/daten/PostDoc2/home/Downloads/MOSFiT/mosfit/fetcher.py", line 122, in fetch
    self._names[catalog] = json.load(
  File "/usr/lib/python3.10/json/__init__.py", line 293, in load
    return loads(fp.read(),
  File "/usr/lib/python3.10/json/__init__.py", line 359, in loads
    return cls(**kw).decode(s)
  File "/usr/lib/python3.10/json/decoder.py", line 337, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/usr/lib/python3.10/json/decoder.py", line 355, in raw_decode
    raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
guillochon commented 2 years ago

Looks like the permissions got mucked on the backend, should be fixed now.

FYI kilonova.space registration lapsed and we no longer control that domain.

JohannesBuchner commented 2 years ago

It works now, thanks a lot!