jtec / prx

MIT License
8 stars 6 forks source link

Faster carrier frequency assignment #119

Closed jtec closed 1 month ago

jtec commented 1 month ago

By adding @line_profiler.profile, and running

https://github.com/jtec/prx/blob/85c006479a402380ecac369a7feacaf6bdf8a6c0/src/prx/test/test_main.py#L139

with LINE_PROFILE=1

we can see that assigning carrier frequencies to observations takes a whooping 13 % of the time spent in https://github.com/jtec/prx/blob/9a90fc804f81e29af47a296a41dfe951502b022c/src/prx/main.py#L211

   426         2     230039.0 115019.5     13.1      flat_obs.loc[:, "carrier_frequency_hz"] = flat_obs.apply(
   427         1          3.0      3.0      0.0          signal_2_carrier_frequency, axis=1
   428                                               )

This PR reduces this to 1.8%:

 413         2        234.0    117.0      0.0      glo_cdma = flat_obs[
   414         2       4526.0   2263.0      0.3          (flat_obs.satellite.str[0] == "R")
   415         1       4537.0   4537.0      0.3          & (flat_obs["observation_type"].str[1].astype(int) > 2)
   416                                               ]
   417         1        369.0    369.0      0.0      flat_obs.loc[glo_cdma.index, "frequency_slot"] = int(1)
   418                                           
   419         1          4.0      4.0      0.0      def assign_carrier_frequencies(flat_obs):
   420                                                   freq_dict = pd.json_normalize(carrier_frequencies_hz(), sep="_").to_dict(
   421                                                       orient="records"
   422                                                   )[0]
   423                                                   assignable = flat_obs.frequency_slot.notna()
   424                                                   keys = (
   425                                                       flat_obs.satellite[assignable].str[0]
   426                                                       + "_L"
   427                                                       + flat_obs["observation_type"][assignable].str[1]
   428                                                       + "_"
   429                                                       + flat_obs.frequency_slot[assignable].astype(int).astype(str)
   430                                                   )
   431                                                   flat_obs.loc[:, "carrier_frequency_hz"] = keys.map(freq_dict)
   432                                                   return flat_obs
   433                                           
   434         1      18716.0  18716.0      1.2      flat_obs = assign_carrier_frequencies(flat_obs)