Wikia / sroka

Python library for API access and data analysis in Product, BI, Revenue Operations (GAM, GA, Athena etc.)
MIT License
71 stars 11 forks source link

Add MySQL connector #23

Closed Ullaakut closed 4 years ago

Ullaakut commented 4 years ago

Goal of this PR

Fixes #22

This PR implements a simple MySQL connector for Sroka. Just like other connectors, it supports fetching data into pandas dataframes as well as csv files.

It also:

How to test it

I personally tested this feature by using this public MySQL server. You can do the same by using the following configuration values in your config.ini file:

[mysql]
host=mysql-rfam-public.ebi.ac.uk
port=4497
unix_socket=
user=rfamro
password=
database=Rfam

Here is an example of running it in the sample Jupyter Notebook on this public database:

Screenshot 2019-10-12 at 10 43 39 PM

And here is the CSV file created when an argument is given for the filename:

Screenshot 2019-10-12 at 10 16 12 PM

Additional Notes

Feel free to be nit-picky about the PR, it's my first ever Python contribution so I don't know any of the best practices that are specific to Python.

martynaut commented 4 years ago

@Ullaakut could you please do a PR to dev branch? dev branch is where most recent code lives. We move code to master when we plan to do a release. I guess we need to make it clear for contributors in README.md to avoid future confusions. Sorry for that!

Ullaakut commented 4 years ago

@martynaut I changed the base of the PR and there are no conflicts. It should be okay now :)

Ullaakut commented 4 years ago

Pushed a fix for the path issues, did the following tests:

Ensure that it works when given a path to a folder that doesn't exist yet, and handles absolute paths

Screenshot 2019-10-25 at 12 17 23 PM Screenshot 2019-10-25 at 12 17 13 PM

Ensure that relative paths are handled correctly

Screenshot 2019-10-25 at 12 15 43 PM Screenshot 2019-10-25 at 12 15 54 PM

Ensure OS errors (no read/write access, file system errors, etc.) are caught properly

Screenshot 2019-10-25 at 12 15 21 PM