matthewgilbert / pdblp

pandas wrapper for Bloomberg Open API
MIT License
242 stars 67 forks source link

Logging compatibility with user code #39

Closed matthewgilbert closed 6 years ago

matthewgilbert commented 6 years ago

Currently the way logging in pdblp is handled, it does not play nice with a user configuring top level logging. The following example illustrates the problematic behavior:

import logging
logger = logging.getLogger()
h = logging.StreamHandler()
formatter = logging.Formatter(
        '%(asctime)s %(name)-12s %(levelname)-8s %(message)s',
        '%Y-%m-%dT%H:%M:%S')
h.setFormatter(formatter)
logger.addHandler(h)
logger.setLevel(logging.DEBUG)

Setting debug=False ignores all logger settings

import pdblp
con = pdblp.BCon(debug=False).start()
con.bdh("SPX Index", "PX_LAST", "20180101", "20180103")
ticker     SPX Index
field        PX_LAST
date                
2018-01-02   2695.81
2018-01-03   2713.06

Setting debug=True when using user defined config settings will ignore these. Note that the user defines here that only CRITICAL level things are logged however this is ignored by pdblp

logger.setLevel(logging.CRITICAL)
con.debug = True
con.bdh("SPX Index", "PX_LAST", "20180101", "20180103")
2018-06-20T17:38:50 root         DEBUG    Sending Request:
 HistoricalDataRequest = {
    securities[] = {
        "SPX Index"
    }
    fields[] = {
        "PX_LAST"
    }
    startDate = "20180101"
    endDate = "20180103"
    overrides[] = {
    }
}

2018-06-20T17:38:51 root         DEBUG    Message Received:
 HistoricalDataResponse = {
    securityData = {
        security = "SPX Index"
        eidData[] = {
        }
        sequenceNumber = 0
        fieldExceptions[] = {
        }
        fieldData[] = {
            fieldData = {
                date = 2018-01-02
                PX_LAST = 2695.810000
            }
            fieldData = {
                date = 2018-01-03
                PX_LAST = 2713.060000
            }
        }
    }
}

ticker     SPX Index
field        PX_LAST
date                
2018-01-02   2695.81
2018-01-03   2713.06
matthewgilbert commented 6 years ago

While not recommended as best practices by the logging tutorial, providing a default logger for users is desired since pdblp is an interactive library mostly used from the shell. Commit 82819186c3d49d9daa94f0c59cdef75678106cac facilitates this by only attaching a default logger if the user has not preconfigured some logging upstream. This is done at run time instead of when importing pdblp to support cases where the user imports pdblp prior to configuring upstream logging.

The behaviour is now consistent with upstream user specifications

import logging
logger = logging.getLogger()
h = logging.StreamHandler()
formatter = logging.Formatter(
        '%(asctime)s %(name)-12s %(levelname)-8s %(message)s',
        '%Y-%m-%dT%H:%M:%S')
h.setFormatter(formatter)
logger.addHandler(h)
logger.setLevel(logging.DEBUG)

import pdblp
con = pdblp.BCon(debug=False).start()
con.bdh("SPX Index", "PX_LAST", "20180101", "20180103")
2018-06-20T17:54:08 pdblp.pdblp  INFO     Sending Request:
 HistoricalDataRequest = {
    securities[] = {
        "SPX Index"
    }
    fields[] = {
        "PX_LAST"
    }
    startDate = "20180101"
    endDate = "20180103"
    overrides[] = {
    }
}

2018-06-20T17:54:08 pdblp.pdblp  INFO     Message Received:
 HistoricalDataResponse = {
    securityData = {
        security = "SPX Index"
        eidData[] = {
        }
        sequenceNumber = 0
        fieldExceptions[] = {
        }
        fieldData[] = {
            fieldData = {
                date = 2018-01-02
                PX_LAST = 2695.810000
            }
            fieldData = {
                date = 2018-01-03
                PX_LAST = 2713.060000
            }
        }
    }
}

ticker     SPX Index
field        PX_LAST
date                
2018-01-02   2695.81
2018-01-03   2713.06
logger.setLevel(logging.CRITICAL)
con.debug = True
con.bdh("SPX Index", "PX_LAST", "20180101", "20180103")
ticker     SPX Index
field        PX_LAST
date                
2018-01-02   2695.81
2018-01-03   2713.06