photon query returns events outside of requested time range

[cmillion]

Extremely rarely, but for example:

https://mastcomp.stsci.edu/portal/Mashup/MashupQuery.asmx/GalexPhotonListQueryTest?query=select time,ra,dec,xi,eta,x,y,flag from GPFCore.dbo.fGetNearbyObjEqNUVAllColumns(179.85709,-21.97462,0.025,823515332999.98,823515445995.0199,0) -- 1529527652&format=extjs

returns an event with a timestamp of 823515332.999 which is < 823515332.99998

Note that the timestamp has better precision than the normally quoted 0.005s, so this may only pop up in the rare observations that used the higher resolution mode.

[cmillion]

The issue is that the database stores the timestamps as BigInt(time * 1000), which truncates the timestamp precision at 1ms. (Truncates, not rounds!) My preference is for the database and the software to treat timestamps identically, so rather than rebuilding the entire database, the gPhoton tools will henceforth truncate timestamps at 1ms precision.

[MattSmiglarski]

Hi.

Unfortunately, the gQuery.exposure_ranges function calls truncate with large numbers plus an epsilon variable. This breaks an example from the docs, where the resulting query contains the string 1e+15 and is not understood by the web service.

As a workaround, the new truncate function can be clobbered with a version that does not use the str function, and avoids the exponential. Please see below for a demonstration of the problem, and the described workaround.

In [1]: import gPhoton

In [2]: gPhoton.gFind(band='NUV',skypos=[176.919525856024,0.255696872807351],maxgap=100.,minexp=100.)

ERROR
Unsuccessful query: https://mastcomp.stsci.edu/portal/Mashup/MashupQuery.asmx/GalexPhotonListQueryTest?query=select distinct time from GPFCore.dbo.fGetNearbyAspectEq(176.919525856024,0.255696872807351,((1.1/2.0)*60.0),1000,1e+15) where band='NUV' or band='FUV/NUV' order by time -- 1530847426&format=extjs
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
…

In [3]: def truncate_workaround(n):
   ...:     return '{:f}'.format(n)
   ...:

In [4]: import gPhoton.gQuery

In [5]: gPhoton.gQuery.truncate = truncate_workaround

In [6]: gPhoton.gFind(band='NUV',skypos=[176.919525856024,0.255696872807351],maxgap=100.,minexp=100.)
No NUV exposure time in database.
Out[6]: {'NUV': {'expt': 0, 't0': None, 't1': None}}

In [7]: str(1000000000000000.0001)
Out[7]: '1e+15'

In [8]: str(1000000000000000)
Out[8]: '1000000000000000'

[cmillion]

Thank you. I was not able to reproduce that ERROR in Python 3.6.6. What version of Python are you using?

[MattSmiglarski]

Python 2.7.10

On Fri, 6 Jul 2018 07:43 Chase Million, notifications@github.com wrote:

Thank you. I was not able to reproduce that ERROR in Python 3.6.6. What version of Python are you using?

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/cmillion/gPhoton/issues/280#issuecomment-403038205, or mute the thread https://github.com/notifications/unsubscribe-auth/AAJXxPjT3zLetqkUCaawFSv7y-DeUDEFks5uD2lsgaJpZM4Uycv2 .

[cmillion]

Ok. Following the lead of astropy, we have dropped Python 2 support in gPhoton.

That said, the truncate function shouldn't be this fragile, so I'll consider this change. I think it's not a priority for a new build right now, though, since it seems to only affect Python 2.