astrocatalogs / astrocats

Astrocats package for constructing astronomical catalogs
https://astrocats.space
MIT License
40 stars 12 forks source link

URL caching update #70

Closed lzkelley closed 8 years ago

lzkelley commented 8 years ago

This pull-request contains enhancements for the way in which URLs are downloaded and cached, in addition to how archived mode operates. The existing methods Catalog.load_cached_url and Task.load_archive provided some of the framework necessary for the overall caching/archiving/updating functionality --- but this was somewhat incomplete, and the additional pieces were put together in the individual tasks that needed them.

Additionally, there was previously an error in the archived-mode logic which made normal imports run in the same way as archived mode.

The new machinery should consolidate everything cleanly.

_BEFORE_ this is merged:

guillochon commented 8 years ago

Will test with Supernovae later today. Also it seems like this is breaking Python 3.4, but the reason isn't entirely clear to me looking at the travis test (one of the imports failed).

How should the calls change from the old way to the new way? Let's use the ASASSN task as an example: https://github.com/astrocatalogs/supernovae/blob/master/tasks/asassn.py#L14-L17

guillochon commented 8 years ago

Ah looks like 3.4 failing was a Travis goof (looks like some technical issue on their end for builds yesterday).

coveralls commented 8 years ago

Coverage Status

Coverage decreased (-0.8%) to 79.594% when pulling c6549714f1f2336c0eecf4287ed034e1a798abfe on url-caching-update into 0efa504f056e01eec2a8443d2918c6df07e19ddf on master.

coveralls commented 8 years ago

Coverage Status

Coverage decreased (-0.8%) to 79.634% when pulling 48b56b8d5405d0af5e78c208187a3d1278c556c6 on url-caching-update into 0efa504f056e01eec2a8443d2918c6df07e19ddf on master.