search

chromaway / ngcccbase

next-gen colored coin client base
MIT License
85 stars 46 forks source link

system and integration testing #105

Open killerstorm opened 10 years ago

killerstorm commented 10 years ago

While unit tests can detect breakage in individual units, it would be nice to know whether the whole system works together.

I think we can set up some virtual machines and do automated tests on testnet or testnet-in-a-box.

I'm not familiar with best practices in this area, so I'd like to know which options are available. (Just some bash/python scripts?)

jimmysong commented 10 years ago

Squish seems like a decent possibility: http://stackoverflow.com/questions/4129430/qt-automated-testing

http://www.froglogic.com/squish/gui-testing/index.php

I can start on this after I finish up some more unit tests.

jimmysong commented 10 years ago

Hm, it looks like we can get pretty far with just python unit tests: http://www.voom.net/pyqt-qtest-example

killerstorm commented 10 years ago

Actually I'd rather test command-line (or RPC) versions first. GUI testing can be cumbersome,

jimmysong commented 10 years ago

Can we close this?

killerstorm commented 10 years ago

If I understand correctly, we have unit tests now, but they are no substitute for system tests (i.e. checking it live incl. interactions with bitcoind and so on).

arichnad commented 10 years ago

color-explorer-vagrant sets up a VM, downloads the blockchain, and I can have it do some simple color transactions. The problem is, to do some actual live color transactions, you'd have to feed it some test-coins every time you want to do a test. Wouldn't that be hard to automate?

arichnad commented 10 years ago

I guess it could feed the test coin back when it's done testing.

jimmysong commented 10 years ago

test_wallet_model.py does actual integration testing using testnet and bitcoind.

p2p has mostly integrated tests in test_ewctrl and test_agent. let me know if you feel like that's not enough and I can make more stuff.