B612-Asteroid-Institute / precovery

Fast precovery of small body observations at scale
BSD 3-Clause "New" or "Revised" License
6 stars 2 forks source link

Add multi-dataset synthetic test data and unit test [ADAM-47] #58

Closed moeyensj closed 1 year ago

moeyensj commented 1 year ago

This PR extends the synthetic test data generation to generate multiple different datasets of observation spanning about 2 calendar months with overlapping and non-overlapping regions of coverage.

From the tests/README.md:

#### `make_observations.py`:
Produces 4 synthetic observations CSV files containing observations for each object in the `sample_orbits.csv` file. The observations have the following properties:
- 4 observations each day (a "quad" tracklet) for 15 days for each observatory.
- Each observatory ('500', 'I11', 'I41', 'F51') is split into its own unique dataset.
- Each observatory has an observation window that starts at 0, 10, 20, and, 30 days after the initial epoch at which the orbits are defined for '500', 'I11', 'I41', 'F51', respectively. This, by design, creates periods of overlap and non-overlap for each dataset and spreads observations over a 2 calendar month period.
- The first observation on any night is always an integer number of days from the reference epoch at which the orbit is defined plus a ~few hours time shift to account for the location of the observatory relative to GMT. Observations 2, 3, and 4 in the same night as the first, are set to be 30, 60, and 90 minutes from the time of the first observation, respectively.
- The exposure times increase for each observation in the quad tracklet: 30, 60, 90, and 120 seconds.
- **No** astrometric errors are added. 
- The observation times are reported at a random time between the start of the exposure and the end of the exposure.  

Usage: ```python make_observations.py```  
This saves observations to `data/index/dataset_500/dataset_500_observations.csv`, `data/index/dataset_I11/dataset_I11_observations.csv`, `data/index/dataset_I41/dataset_I41_observations.csv`, `data/index/dataset_F51/dataset_F51_observations.csv`.
moeyensj commented 1 year ago

I've combined I41 and F51 into a single dataset and updated the tests README accordingly. The cpu_count=1 testing residual has also been reverted.