Boto DEBUG output floods mrun logs

[rkingsbury]

The boto package used by the S3Store to access MinIO generates a very large amount of DEBUG output, making it difficult to see the output from maggma itself.

[sivonxay]

I noticed this as well. I believe the offending lines are the following (the latter for serial running within a jupyter notebooks):

https://github.com/materialsproject/maggma/blob/9e037d7349e531be7d5f05096815781ab6e8faa6/src/maggma/cli/__init__.py#L70-L71 https://github.com/materialsproject/maggma/blob/672c6aedc8af97029b14a73849b39a9c45988e96/src/maggma/core/builder.py#L127-L128

Not sure what's the best way to deal with it, but that depends on whether or not it is wanted. In the case that they aren't wanted, I would have no issue silencing them with something like this (as per github issue on boto3):

def silence_boto3_logger():
    logging.getLogger('boto3').setLevel(logging.CRITICAL)
    logging.getLogger('botocore').setLevel(logging.CRITICAL)
    logging.getLogger('nose').setLevel(logging.CRITICAL)
    logging.getLogger('s3transfer').setLevel(logging.CRITICAL)
    logging.getLogger('urllib3').setLevel(logging.CRITICAL)

root = logging.getLogger()
root.setLevel(level)
silence_boto3_logger()

alternatively, the log level update should be applied to the builder logs, not the root logger

for builder in builders:
    logger = logging.getLogger(type(builder).__name__)
    logger.setLevel(level)

[munrojm]

Having that information has been useful for me in the past when debugging connections to S3 for different builders. Not sure what the best course of action is.