search

INWTlab / dbrequests

python package built for easy use of raw SQL within python and pandas projects
MIT License
5 stars 2 forks source link

Logging #50

Open Cattes opened 3 years ago

Cattes commented 3 years ago

To be able to process more closely which queries / query file was used to connect to the database, it should be possible for the user to enable logging. Using the debug log level will allow the user to set the log level for the dbrequest package to debug to get the detailed query information. Open question: If files are used, should only the .sql file name be logged or the content of the .sql query sent to the server? The content may contain sensible information which should not appear in a logfile.

phainom commented 3 years ago

Please don't log any query content, especially not any insert, copy or load/unload statements, as it can not only contain sensitive information, but also provide access information to eg the unload targets.

wahani commented 3 years ago

I agree, we should not log any queries or parameters passed to them. However we may log which file is being executed, how many rows are processed and a timing. @Cattes is that what you had in mind?

Cattes commented 3 years ago

However we may log which file is being executed, how many rows are processed and a timing.

Yes having some meta information about the execution would be useful for monitoring and debugging.

I understand logging query content per default is a severe security risk, but do you think enabling it manually via the debug log level (or a -vvv flag) would be a good idea for debugging?