Closed mworion closed 3 years ago
brandon-rhodes
commented
3 years ago Interesting! Does it feel like it loads just as fast from JSON, or more slowly because of the larger file size? (I wish everyone would just move to CSV. It loads very quickly, and doesn’t have the vast redundancy of JSON.)
We should also take a look at whether it provides more decimal places in orbital parameters, as that could allow for more accurate orbits.
mworion
commented
3 years ago From what I see, there are no more decimal places, but adds some more fields than in the text file:
{ "H": 18.9, "G": 0.15, "Num_obs": 103, "rms": 0.41, "U": "1", "Arc_years": "1994-2019", "Perturbers": "M-v", "Perturbers_2": "38h", "Principal_desig": "1994 WY1", "Epoch": 2458600.5, "M": 298.5756, "Peri": 155.25303, "Node": 232.63124, "i": 5.79199, "e": 0.3747765, "n": 0.2341911, "a": 2.6066884, "Ref": "E2019-P40", "Num_opps": 3, "Computer": "MPCLINUX", "Hex_flags": "0005", "Last_obs": "2019-08-04", "Tp": 2458862.78324, "Orbital_period": 4.2085615, "Perihelion_dist": 1.6297628, "Aphelion_dist": 3.583614, "Semilatus_rectum": 1.1202798, "Synodic_period": 1.3116661, "Orbit_type": "Object with perihelion distance < 1.665 AU" }
and it avoids the nasty compressed dates etc. Docs are available: ()That's a real benefit for accessing. And as I'm a little bit lazy, it's easy as you could access directly the dict. Loading does like json does. If I try to load the full MPC file (, 613 MB)I ran out of memory.
I give you some times out of the list from https://www.minorplanetcenter.net/data
Orbits for Near Earth Asteroids (NEAs): download took 0.8510456085205078 unzip took 0.0049779415130615234 json load took 0.015008211135864258
Orbits from the latest DOU MPEC: download took 1.3170678615570068 unzip took 0.1071016788482666 json load took 0.2768704891204834
Orbits for Potentially Hazardous Asteroids (PHAs) download took 1.1340773105621338 unzip took 0.025032520294189453 json load took 0.03396964073181152
Orbits for TNOs, Centaurs and SDOs download took 0.9000532627105713 unzip took 0.01816582679748535 json load took 0.044861793518066406
if you still like my software, you could test this on windows (as my mount manufacturer only has a windows updater from programming the mount. But reading the data from internet and displaying it is available on Mac or Ubuntu, too
Michel
brandon-rhodes
commented
3 years ago From what I see, there are no more decimal places…
Loading does like json does. If I try to load the full MPC file (, 613 MB)I ran out of memory.
That is my general issue with JSON: it is massively redundant if instead of using it to transmit lists of coordinates ({"ra_list":[1,2,3]}), every single coordinate is given its own object ([{"ra": 1}, {"ra": 2}, {"ra": 3}]), which is the more typical way to use JSON as a data format. From other work over on my sgp4 library, I have come to strongly prefer formats like CSV which amortize the cost of the list of names across multiple rows or values.
So I think that Skyfield for now will stick with table-based MPC formats, especially if the JSON offers no additional precision.
Thanks for drawing our attention to the alternate format, though!
@brandon-rhodes,
You might not have seen it, but I recognized that the minor planets center offers all data already in json.gz format. So just a json.load() does everything for managing the data. Hope this might help lower the effort for parsing all the data.
Michel