A custom logging handler that writes data to standard log tables.

Create a parsons logging handler that writes logging.log messages to a database, using standard Python interfaces.

Detailed Description

In the Mobilize to ActionNetwork sample, I noticed a logging pattern that has a few issues. Namely, what happens if there's a crash before log messages are written to the log table? Furthermore, wouldn't it be nice to get to use logger.info etc and have those messages appear in your data warehouse? Without duplicating code, you could capture all the console logs to a db.

The standard way to implement this in Python is to create a custom Handler class. The interface could look something like:

db = Redshift()
handler = ParsonsHandler(db, table_name='mobilize_schema.log_messages')
logger = logging.getLogger(__name__)
logger.addHandler(handler)

Context

Code deduplication (uses standard logging for all log destinations)
Less error prone
Encourage logging practice

Possible Implementation

(rough sketch)

@dataclasses.dataclass
class ParsonsHandler(logging.Handler):
    db: BaseTable
    table_name: str

    def emit(record) -> None:
        self.db.copy(Table(vars(record)), table_name=self.table_name, **other_kwargs)

    def close() -> None:
        self.db.close()

Better still: you may be able to subclass and directly use this: https://docs.python.org/3/library/logging.handlers.html#logging.handlers.HTTPHandler !

Priority

Low Priority – this is a nice-to-have.

move-coop / parsons