Open amarjandu opened 3 years ago
Since the stitched subgraphs are all from the same source we don't gain any information from logging the source
attribute of the FQIDs. If we change processed
and its related variables from Set[SourcedBundleFQID]
to Set[BundleFQID]
, or upcast the elements of processed
to BundleFQID
before logging, it would reduce the number of characters logged for each stitched subgraph from approximately 343 (it varies slightly on the source spec) to 94, a 72% decrease.
@hannes-ucsc to devise solution.
Something like
log.info('Stitched on %i bundles' len(fqids))
if log.isEnabledFor(log.DEBUG):
for fqid in fqids:
log.debug('Stitched on bundle %s', BundleFQID.__repr__(fqid))
This is followed by 2244 instances of SourcedBundleFQID. This causes Cloudwatch to split the log line in multiple log entries. It's also impractically large.