alercebroker / usecases

Collection of notebooks and scripts with science use cases
26 stars 11 forks source link

Number of objects #3

Open raphaelshirley opened 5 years ago

raphaelshirley commented 5 years ago

If I run the SN notebook:

https://github.com/alercebroker/usecases/blob/master/notebooks/ALeRCE_ZTF_SupernovaUseCase.ipynb

But change the date in the query in cell 12 to look for sn at least 70 days after first light to the last 100 days it returns no objects.

What am I doing wrong?

Many thanks,

Raphael. (postdoc at IAC)

query=''' select probabilities.oid, probabilities.other_prob, objects.meanra, objects.meandec, objects.nobs, objects.mean_magpsf_g, objects.mean_magpsf_r, objects.min_magpsf_g, objects.min_magpsf_r, objects.classxmatch, objects.firstMJD

from probabilities

inner join objects on probabilities.oid=objects.oid

where probabilities.sne_prob>0.7 and objects.classxmatch=%s and objects.firstMJD > %s ''' % (classmapper["SNe"], 58744. - 100.)

SNe = pd.read_sql_query(query, conn) SNe.set_index('oid', inplace=True) SNe.head()

fforster commented 5 years ago

Hi Raphael, the reason for this is that we haven't updated the crossmatches table for some time. We are working in a new version which is up to date, which will be released soon. For the moment you can remove the requirement of having a crossmatch. See attached Screenshot from 2019-09-18 13-31-39

raphaelshirley commented 5 years ago

Thank you. I am also getting an issue with pip install alerce_client:

"Collecting alerce_client Could not find a version that satisfies the requirement alerce_client (from versions: ) No matching distribution found for alerce_client"

Getting the code from https://github.com/alercebroker/alerce_client and installing does seem to work. Perhaps pip hasn't been updated yet...

Best,

Raphael.

fforster commented 5 years ago

Hi Raphael, thanks for the feedback. Can you send us more details about your operating system, python version, etc...? You can install locally from this repository: https://github.com/alercebroker/alerce_client You will need pandas >= 0.24

fforster commented 5 years ago

@cvalenzu any ideas what could be the problem?

raphaelshirley commented 5 years ago

I'm using a mac. Directly installing from the GitHub code works. I use conda and pip, pandas 0.24, python 3.6.

raphaelshirley commented 5 years ago

Hmm I am also getting an issue with the queries giving me objects with dates below the minimum I set. The following returns objects with firstmjd below 58700:

` total = 100 records_per_page = 100 page = 1 sortBy = "pclassrf" classrf = "SNe" pclassrf = 0.8 min_nobs = 10 max_nobs = 50 min_firstmjd = Time("2019-09-19T00:00:00", format="isot", scale="utc").mjd - 14

params = {

"total": total,

"records_per_pages": records_per_page,
"page": page,
"sortBy": sortBy,
"query_parameters": {
    "filters": {
        "nobs": {
            "min": min_nobs,
            "max": max_nobs
        },
        "firstmjd": { 
            "min": min_firstmjd,
        },
        "classrf": classrf,
        "pclassrf": pclassrf
    }
}

} SNe = alerce.query(params) print(SNe.shape) SNe.head() `

fforster commented 5 years ago

Thanks for the input, we can reproduce the error you mention. I think there is a bug in the API, let us fix it and we will let you know once it is fixed.

cvalenzu commented 5 years ago

@cvalenzu any ideas what could be the problem?

Hi, The package is called just alerce, to install it run pip install alerce and to import the client run from alerce.api import AlerceAPI

fforster commented 5 years ago

Hi Raphael

We found the problem, the params were incorrectly formatted. "firstmjd" must be inside the "dates" key. Try the following formatting:

total = 100 records_per_page = 100 page = 1 sortBy = "pclassrf" classrf = "SNe" pclassrf = 0.8 min_nobs = 10 max_nobs = 50 min_firstmjd = Time("2019-09-19T00:00:00", format="isot", scale="utc").mjd - 14 print(min_firstmjd) params = {

"total": total,

"records_per_pages": records_per_page, "page": page, "sortBy": sortBy, "query_parameters": { "dates": { "firstmjd": { "min": min_firstmjd, } }, "filters": { "nobs": { "min": min_nobs, "max": max_nobs }, "classrf": classrf, "pclassrf": pclassrf } } } SNe = alerce.query(params) SNe.head()

which returns one object