Merge generally useful updates from htrc-features branch

[organisciak]

I pulled out some changes from the htrc-features branch of my fork that I expect may be generally useful.

• Fast feature counter code for large datasets with feature input
• Updates to createDatabase: logging and various fixes
  • recurse DB.connect() without engine setting on MySQL versions that expect "default_storage_engine" to be set on the daemon level.
• other small fixes

[bmschmidt]

One thing I take from this is that you are using the logger module in places where we currently warn or write to sys.stderr. I'm happy to get on board with this, but don't know what the advantages are. Should we have a feature request about bringing some of the existing code into alignment with your practice here?

— Sent from Mailbox

On Tue, Mar 10, 2015 at 4:03 PM, Peter Organisciak notifications@github.com wrote:

Merged #58.

Reply to this email directly or view it on GitHub: https://github.com/Bookworm-project/BookwormDB/pull/58#event-249621803

[organisciak]

I should have consulted you first, here's why I like using it, the decision whether to keep doing so is up to you.

Logging lets you have different levels (debug, info, warn, error), allows you to route to a log file, and outputs in a standard format -- particularly log time and log level are helpful. Messages are only logged when they're equal or greater than a specific logging level; i.e. with the default 'warn' level, info and debug messages are not logged but warn and error are.

I started using it in this case a) because I can keep code to print verbose debugging in, rather than temporary putting in print statements and subsequently having to hunt them down before commits, and b) finding messages in the log file by log level is useful, so I can grep for caught-but-logged errors after running a big process.

Logging doesn't write anything unless you set it. It's global, so you could set it in OneClick.py or another top-level script, and the imported modules will log at that level. Remembering that I'm on mobile and there may be typos, here's an example:

logging.basicConfig("filename.log", level=logging.INFO)

On Wed, Mar 11, 2015, 4:12 PM Benjamin Schmidt notifications@github.com wrote:

One thing I take from this is that you are using the logger module in places where we currently warn or write to sys.stderr. I'm happy to get on board with this, but don't know what the advantages are. Should we have a feature request about bringing some of the existing code into alignment with your practice here?

— Sent from Mailbox

On Tue, Mar 10, 2015 at 4:03 PM, Peter Organisciak notifications@github.com wrote:

Merged #58.

Reply to this email directly or view it on GitHub: https://github.com/Bookworm-project/BookwormDB/pull/58#event-249621803

— Reply to this email directly or view it on GitHub https://github.com/Bookworm-project/BookwormDB/pull/58#issuecomment-78372742 .