Closed troyraen closed 4 months ago
https://github.com/mwvgroup/pittgoogle-client/pull/6 has been merged. What's the next step for this current Issue?
Thanks, we need to evaluate what still needs to be added to pittgoogle-client
. Below is my rundown from looking at the imports in classifier/main.py. Perhaps I can spend a little time this weekend to knock out the simpler ones. @wmwv if there are particular items that would be helpful to prioritize for #13, let me know.
from broker_utils import data_utils, gcp_utils, math
This is using:
get_table
causes problems at high alert rates (see https://github.com/mwvgroup/Pitt-Google-Broker/issues/96#issuecomment-983282727) and should be dropped; and 2) we should just ditch this call altogether and use a BigQuery subscription instead.from broker_utils.types import AlertIds
AlertIds
is awkward and clunky to use and I want to drop it. The new pittgoogle-client has an Alert
class and I can look at making the IDs easily accessible from there. In the meantime, the IDs can just be pulled directly from the alert.from broker_utils.schema_maps import load_schema_map, get_value
from broker_utils.data_utils import open_alert
Alert.from_cloud_run
, Alert.from_cloud_functions
, Alert.from_path
, etc.).The way that schema maps are currently implemented is a pain and I don't have a clear solution right now.
Jotting down some thoughts:
This should probably be handled by pittgoogle.pubsub.Alert
without the user having to do anything except specify which survey the alert is coming from (and possibly the schema version). The schema maps should be packaged with the pittgoogle-client so that they don't have to be supplied or retrieved from somewhere else.
I've started working on this, but realized that simply porting these functions over is really not the way to go. Most of this should be handled by the pittgoogle.pubsub.Alert
class, so I'm implementing new methods there.
Update all necessary code/files to remove the
pgb-broker-utils
python package, and instead use tools frompittgoogle-client
.mwvgroup/pittgoogle-client#6 is a PR containing a major refactor of
pittgoogle-client
, so this likely needs to wait until that is done. After that happens, we'll need to check and see which functions/classes still need to be added topittgoogle-client
for this to work (and then add them).