MetPX / sarracenia

https://MetPX.github.io/sarracenia
GNU General Public License v2.0
44 stars 22 forks source link

Have the raw2bulletin renamer work with senders #1113

Open andreleblanc11 opened 2 months ago

andreleblanc11 commented 2 months ago

Originally, rename/raw2bulletin.py was ported from v2/Sundew to integrate the AM protocol into sr3.

Now that we look to port more configurations from Sundew to Sarra, we'll eventually need to have a way of generating bulletin filenames reliably. The way we currently do this is we send our unformated bulletin filenames to Sundew, where it receives the bulletins through an AM receiver. That receiver then auto-converts the filenames correctly.

When I tried to use the sr3 renamer with a sender (callback rename.some-other-plugin-linking-to-the-raw2bulletin-renamer in the config) , these are the problems that I've come accross.

  1. The Bulletin class is complaining that it is not able to find self.o. This isn't a problem with the AM flow config, and I thought this might be because we do a callback_prepend gather.am in the config. I tried doing the same with the renamer, but this didn't resolve the problem.
  2. Because the AM receiver gets data before going through the sarracenia filtering, it made sense at the time to use msg['new_dir'] and msg['new_file'] to get the file contents. However, for a sender, directory corresponds to the directory where we want to drop files in the transfer so using the new fields doesn't make sense for this component.
petersilva commented 2 months ago
 def  __init__( self, options):
      self.o = options

then ... b=Bulletin(self.o)
andreleblanc11 commented 2 months ago

The raw2bulletin renamer only renames the filename and does not alter the file contents so I think it should stay in the rename directory.

The getData method in bulletin.py fetches either the downloaded bulletin file or the bulletin contents from the sarracenia message. With this data, we fetch the station name, the BBB and the timestamp to generate the bulletin filename.

petersilva commented 2 months ago

Should probably remove getdata from bulletinand use the one in the Message class