Work out test strategy and test plan

[skasberger]

Purpose

Work out test strategy and test plan for pyDataverse, with a test plan.

Scope

In:

• Detailed test plan for API module

Out:

• Detailed test plan for models and utils module.
• Test data

Requirements

• Create test strategy
• Create test plan

ACTIONS

0. Pre-Requisites

• [ ] Apply modern development practices
• [ ] Specification
• [ ] Risk Assessment #82

1. Research

• [x] API testing best practices
  • [x] 3 Ways to Test API Client Applications in Python
  • [x] Forming an API Test Strategy - Where to Start
  • [x] Dependency injection
  • [x] API testing strategy: Who is testing your API-driven product?
  • [x] Quality Assurance, Quality Control and Testing — the Basics of Software Quality Management
  • [x] Three aspects of software quality: functional, structural and process
  • [x] Test Strategy and Test Plan
  • [x] The 10 Minute Test Plan
  • [x] Rapid Software Testing
  • [x] A sneak peek into ‘test framework’, ‘test pyramid’ & ‘testing pyramid’
  • [x] SoapUI
  • [x] Software Testing - Wikipedia
  • [x] Regression Testing - Wikipedia
  • [x] Risk-based testing - Wikipedia
• [x] Research other projects test strategy
  • [x] Pandas
  • [x] Requests
  • [x] Spotify
  • [x] Wikipedia API
  • [x] Facebook
  • [x] Twitter
  • [x] Slack
  • [x] Google Places
• [ ] Check solutions
  • [x] vcrpy
  • [ ] VCR.py at Venmo
  • [x] pytest
  • [x] Check: re-use unit tests for integrity tests
  • [x] Check: Plugins
  • [x] Check: tmp directories
  • [x] 5 Pytest Best Practices for Writing Great Python Tests
  • [x] Tips and tricks for unit tests (with Python & Pytest)
• [x] Check Pytest docs
  • [x] Check Fixtures
  • [x] Check parametrize
  • [x] Check Monkeypatching
  • [x] Check Hypothesis
• [ ] Watch python testing tutorials
• [ ] Read QA/Testing notes

2. Plan

• [ ] Define requirements
• [ ] Documentation of test strategy:
  • [ ] location: offline/online; GitHub, RTD
  • [ ] file format
  • [ ] content
• [ ] Required environment
  • [ ] Dataverse installation
  • [ ] Version
  • [ ] Continous Integration: Travis-CI

3. Implement

• [ ] Write tests
• [ ] Write/Update code
• [ ] Write/Update Docs
• [ ] Write/Update Docstrings
• [ ] Run pytest
• [ ] Run tox
• [ ] Run pylint
• [ ] Run mypy

4. Follow Ups

• [ ] Apply test strategy to
  • [ ] api #83 (release 0.3.0)
  • [ ] models (release 0.4.0)
  • [ ] utils (later)
• [ ] Review extension of test strategy to
  • [ ] tox
  • [ ] linting
  • [ ] coverage
  • [ ] docstrings
  • [ ] docs
• [ ] Review
  • [ ] Code
  • [ ] Tests
  • [ ] Docs
  • [ ] Strategy
  • [ ] Content
  • [ ] Documentation

[pdurbin]

As discussed during the 2024-02-14 meeting of the pyDataverse working group, we are closing old milestones in favor of a new project board at https://github.com/orgs/gdcc/projects/1 and removing issues (like this one) from those old milestones. Please feel free to join the working group! You can find us at https://py.gdcc.io and https://dataverse.zulipchat.com/#narrow/stream/377090-python