关于测试

[zhyq0826]

http://devdocs.io/python~2.7/library/unittest https://wiki.python.org/moin/PythonTestingToolsTaxonomy http://pythontesting.net/books/python-testing-ebook/

单元测试 unit test

如果测试:

• 与数据库产生交互
• 与网络产生交互
• 使用了文件系统
• 不能和其他单元测试一起执行
• 需要特殊的运行环境保障测试的执行

这些测试都不算是单元测试 unit test

if test:

• talk to the datebase
• communicates across the network
• touch the file system
• can't run at the same time as any of other unit tests run
• have to do special things to run it

集成测试 integration test

1.integration tests do verify that your code works with external dependencies correctly 2.the dependencies can be broken into two types: the ones that under control(like datebase, filesystem), the ones that you don't have control over(like network)

1.集成测试确保应用对外部依赖可以正确的执行 2.这些依赖一般可以分为：可控和不可控的，可控比如数据库，文件系统等，不可控如网络等

what type of code benefit if unit testing 什么样的代码能从单元测试中收益

1.Complex code with few dependencies

This cost-benefit argument goes strongly in favour of unit testing this code, because it’s cheap to do and highly beneficial

2.Trivial code with many dependencies

This cost-benefit argument is in favour of not unit testing this code

3.Complex code with many dependencies

This code is very expensive to write with unit tests, but too risky to write without.Usually you can sidestep this dilemma by decomposing the code into two parts: the complex logic (algorithm) and the bit that interacts with many dependencies (coordinator)

4.Trival code with few dependencies

In cost-benefit terms, it doesn’t matter whether you unit test it or not

1.复杂且依赖很少，最能受益于单元测试

2.琐碎不重要且依赖多，不需要单元测试

3.复杂而且依赖多

写单元测试的代价很大，但是如果没有单元测试，引入风险的可能性很高，建议把这样的代码拆成负责逻辑和负责依赖的俩部分

4.琐碎不重要且依赖少，不值得测试

acceptance test

http://istqbexamcertification.com/what-is-acceptance-testing/

fter the system test has corrected all or most defects, the system will be delivered to the user or customer for Acceptance Testing or User Acceptance Testing (UAT).

• Acceptance testing is basically done by the user or customer although other stakeholders may be involved as well.
• The goal of acceptance testing is to establish confidence in the system.
• Acceptance testing is most often focused on a validation type testing.
• Acceptance testing may occur at more than just a single level, for example: A Commercial Off the shelf (COTS) software product may be acceptance tested when it is installed or integrated.
• Acceptance testing of the usability of the component may be done during component testing.
• Acceptance testing of a new functional enhancement may come before system testing. The types of acceptance testing are:

-The User Acceptance test: focuses mainly on the functionality thereby validating the fitness-for-use of the system by the business user. The user acceptance test is performed by the users and application managers.

• The Operational Acceptance test: also known as Production acceptance test validates whether the system meets the requirements for operation. In most of the organization the operational acceptance test is performed by the system administration before the system is released. The operational acceptance test may include testing of backup/restore, disaster recovery, maintenance tasks and periodic check of security vulnerabilities.
• Contract Acceptance testing: It is performed against the contract’s acceptance criteria for producing custom developed software. Acceptance should be formally defined when the contract is agreed.
• Compliance acceptance testing: It is also known as regulation acceptance testing is performed against the regulations which must be adhered to, such as governmental, legal or safety regulations.

---

为什么需要自动化测试

• 可重复执行
• 有利于代码反复修改和迭代
• 初期投入时间巨大，长久看来是节约大量时间
• 如果编写的功能的正确性需要第三者来验证，为了能最大程度上的节约寻找问题的时间和保证代码经过反复更改之后的正确性，测试的重要性在这种场景之下非常高。
• 自动化测试时持续集成的基础

测试用例的分层设计

单元测试只测试函数的功能，不依赖数据库、网络，单元测试保证是在函数粒度下输入正确的情况下输出正确。

单元测试应该先从完全不依赖编写功能中任何其他函数(这些函数本身也需要测试)的函数开始，在此称为这些函数为原子函数: 不依赖任何其他需要测试函数的函数。

在保证原子函数正确性之后，测试就应该过渡到仅仅只依赖原子函数的函数，我们称这些函数为分子函数：分子函数是仅仅只依赖原子函数的函数。

在保证分子函数正确性之后，测试过渡到需要依赖外部环境的函数，比如数据库、网络等，这些函数或实例的方法调用最终将会把计算的数据以各种各样的方式输出到外部软件，比如数据库、缓存、磁盘或者一次 http 请求。在此我们称这样的函数为实体函数：与外界进行交互的函数。这样的测试我们称之为集成测试。

任何测试都离不开输入和输出，为了能够保证测试的顺利和快速执行以及代码的迅速变更和重构，测试要具有足够的灵活性。有几个原则可以先确立下来执行过程中慢慢改进。

如果测试输入样例少于 3 个，这三个样例完全可以手动编写完成 如果输入样例多余 5 个以上，需要编写测试程序来保证输入在任何时候都是一致的。 对于需要依赖时间的测试，一定要模拟清楚测试边界的条件和范围，事先可以通过手动描述出不同的边界和范围。

测试工具

• http://devdocs.io/python~2.7/library/unittest unitest
• https://nose.readthedocs.io nose
• https://pypi.python.org/pypi/mock python standard mock
• https://github.com/gabrielfalcao/lettuce bdd
• https://pythonhosted.org/should_dsl/ should dsl
• https://github.com/robotframework/robotframework automate test framework

nose

利用 nose 的特性为不同的测试函数打上不同的标签，把原子函数测试，分子函数和实体函数分开，按修改之后的影响程度进行测试。

利用 nose 的 attr plugin 可以对测试进行分类

@attr(tags=['rtime_zero', 'api'])
@attr(tag='api')

nosetests -a tag=b test_package
nosetests -a tags=b test_package

测试用例编写

分层

测试用例按照原子函数、分子函数、实体函数进行分层，测试顺序

1. 原子函数
2. 分子函数
3. 实体函数

回归测试：修改 1，测试 1，2，3；修改 2，测试 2，3；修改 3 测试 3

测试条件编写

所有的逻辑都可以用分支和循环来表示，为了编写满足覆盖所有分支的测试条件，几个原则

1. 优先测试范围（0~0.5, 0.5~1, 1~1.5,....），重点测试边界(0, 0.5, 1, 1.5, ...)
2. 借用分支和循环，排列组合所有测试条件
3. 手动计算每个范围和每个边界的正确输出，一个就可以，要确保足够正确

可测性

为了让测试案例编写容易，测试准确，函数要具备为很好的可测性，衡量可测性的标准

• 函数职责单一
• 单一的输入，必然产生单一的输出，不具有二义性

测试的环境依赖

集成测试对外部软件和环境的依赖很多，不同测试场景对配置要求也不相同，比如本地开发测试，ci 测试，功能集成测试等等

借助 nose plugin https://pypi.python.org/pypi/nose-testconfig/0.10 可以对环境依赖进行配置

#config_loca.ini

[v12]
host = http://127.0.0.1:5000

#bash
nosetests -v  --nocapture -a tags=leap_year --tc-file=tests/test_rest/config_local.ini

#test.py
from testconfig import config
config['v12']['host']