seanharr11 / etlalchemy

Extract, Transform, Load: Any SQL Database in 4 lines of Code.
MIT License
556 stars 83 forks source link

Add test framework and begin creating tests #42

Closed tpow closed 5 years ago

tpow commented 5 years ago

I've added a test framework using pytest and have started creating test cases. I started with tests of the schema transformations since it is relatively straightforward to test.

You can call it using either the pytest command or python setup.py test. It also supports passing arguments so you can do python setup.py test -a '--verbose'

I also refined the version numbers of the various packages so they aren't fixed to out of date versions. At least one of the changes (SQLAlchemy-Utils) requires python 2.7 and above. That seems reasonable, but I'm not sure which versions this project is intending to support. I'd recommend 2.7+ and 3.5+ now.

Tim

seanharr11 commented 5 years ago

@tpow thanks for this! Keep 'em coming...

In the meantime I'm writing some functional tests, leveraging Docker, to test real-world migrations between PostgreSQL, SQL Server, MySQL, PostgreSQL and SQLite. I plan on putting a testing TODO together for the functional stuff, but more importantly, we need the unit tests, mucho appreciated.

Once we have these, we can bump to python3.6, address scalability (not loading entire tables into memory), and other issues...

tpow commented 5 years ago

@seanharr11 Glad I could help. Thanks for merging my PR. I'll see if I can build more of the unit tests. As is expected when adding tests after the code, I've thought of ways to both improve testing and simplify the code, so I'll probably be making some other pull requests.

For functional tests, do you already have a test database framework built? Even a fairly simple one that exercises the various supported column types would be helpful. Maybe this latter question should be a new issue to add functional tests.

Tim

seanharr11 commented 5 years ago

@tpow moved here: https://github.com/seanharr11/etlalchemy/issues/45