CombinedDataReader.read_block() can read from an external ephemeris file in two ways, controlled by the ephem_primary key. This dictates whether the ObjIDs for the chunk are derived from the relevant chunk of the orbit file (ephem_primary=False) or from reading a chunk of the ephemeris file (ephem_primary=True).
We have ephem_primary=True. This means that if the config file specifies size_serial_chunk=5, CombinedDataReader.read_block() would read the ephemeris file in chunks of 5, then use the ObjIDs from those chunks to pull data from orbit/physical parameters. As the external ephemeris file has several rows for each ObjID, this causes some unwanted behaviour - some objects may end up spilt across two chunks, for instance, and therefore wouldn't be linked properly.
What we want is ephem_primary=False. If size_serial_chunk=5, then CombinedDataReader.read_block() should read the orbit file in chunks of 5, then use the ObjIDs from those chunks to pull the relevant lines from the external ephemeris file.
CombinedDataReader.read_block()can read from an external ephemeris file in two ways, controlled by theephem_primarykey. This dictates whether the ObjIDs for the chunk are derived from the relevant chunk of the orbit file (ephem_primary=False) or from reading a chunk of the ephemeris file (ephem_primary=True).We have
ephem_primary=True. This means that if the config file specifiessize_serial_chunk=5,CombinedDataReader.read_block()would read the ephemeris file in chunks of 5, then use the ObjIDs from those chunks to pull data from orbit/physical parameters. As the external ephemeris file has several rows for each ObjID, this causes some unwanted behaviour - some objects may end up spilt across two chunks, for instance, and therefore wouldn't be linked properly.What we want is
ephem_primary=False. Ifsize_serial_chunk=5,thenCombinedDataReader.read_block()should read the orbit file in chunks of 5, then use the ObjIDs from those chunks to pull the relevant lines from the external ephemeris file.