Thesis Meeting | June 17, 2024 - 10:30am - ITB 167

[samm82]

I'm assuming that the synchronous part of this meeting will be just @smiths and I, but that @JacquesCarette would want to review/comment on it afterwards.

Progress Update

I've started looking at resources to fill in some of the "gaps" in my spreadsheet:

• Assertion Checking: LahiriEtAl2013, ChalinEtAl2006, BerdineEtAl2006
• Loop Testing: DhokAndRamanathan2016, GodefroidAndLuchaup2011, PreußeEtAl2012, ForsythEtAl2004
  • Used as reference for terms but not fully "scraped": ISO2015, ISO2022
  • Added as out-of-scope examples: Goralski1999, Dominguez-PumarEtAl2020
  • Added as potentially in scope: TrudnowskiEtAl2017, PierreEtAl2017
• EMSEC Testing (out of scope; not fully "scraped"): ZhouEtAl2012, ISO2021
• Asynchronous Testing: JardEtAl1999
• Performance(-related) Testing: Moghadam2019
• Web Application Testing: DoğanEtAl2014, Kam2008 (in progress)

Summary

This totals to 10 "fully" investigated sources, with 1 in progress and 3 standards that may be helpful (but are probably mainly out of scope).

Before these resources, I had 403 testing approaches, with 146 (~36%) without definitions. After these resources, I have 466 testing approaches, with 175 (~38%) without definitions.

Questions

Haven't made into separate issues yet, but I can if that would be helpful! 😅

Scope

1. Are language-specific test approaches meaningful? (e.g., JavaScript testing in Doğan et al., 2014, p. 192) - my instinct is no, unless they imply a new testing approach due to constraints of the language

Areas of Potentially Implied Testing

1. Coverage-based testing (see Table 13 from Doğan et al., 2014)
2. Types of assertion checking: input, loop, and output (IEEE, 2017)
3. Types of fault-based testing (see Table 27 from Doğan et al., 2014)
4. Types of performance testing based on issues: repetitive computations, redundant loops, object bloat, latent performance bugs, and performance issues in clouds and smart phones (Dhok and Ramanathan, 2016, p. 895)

[samm82]

The only thing that seems meaningful enough to explicitly note is that Table 13 (the types of coverage) seem to imply other forms of testing; language-specific testing methods are out of scope (and I should explicitly note this), and the implied "sub-approaches" for other kinds of testing should just be added as notes where relevant.

The bottom line on this method of research @JacquesCarette is that it seems to be working. Not only am I able to fill in some definitions, but I'm also discovering new kinds of testing!

TO-DO

• [x] Add "coverage-based" kinds of testing from Table 13
• [x] Add notes on rest of testing (implied/language-specific)
• [x] Add notes to methodology on this type of "snowballing"
• [x] Investigate multi-lingual faults

[samm82]

"Multi-lingual faults" in this context referred to spoken languages, as you suspected @smiths. I skimmed through the list, and none of them seem to suggest a new approach to testing, so this table was just added as a note on fault-based testing as we discussed. All items addressed; issue closed!

[JacquesCarette]

The bottom line on this method of research [...] I'm also discovering new kinds of testing!

Excellent.