search

samm82 / TestGen-Thesis

My MASc thesis for generating test cases in Drasil
Other
0 stars 0 forks source link

Thesis Meeting | Mar 18, 2023 - 10:30am - ITB 167 #44

Closed samm82 closed 8 months ago

samm82 commented 8 months ago

Conferences/Publications

See #42 and the wiki page. Don't really have anything specific to ask except for what the next steps are. How much time should I devote into looking for other venues? Is there one here that we think we should commit to?

  1. I made some scripts to help with my search, one to scrape/filter CFPs from the resource mentioned in #41 and one to estimate the amount of committee members from top-100 universities; would these be helpful to upload somewhere? This repo? A separate repo? Drasil?
  2. I'm assuming natural-language-processing-centric conferences are probably not the best fit for us? đŸ˜…

Software Qualities

  1. General check-in: am I doing too much here?
  2. Are the following actually qualities worth tracking? a. "diversity": the "realization of the same function by different means" (IEEE, 2017, p. 143) b. "effectiveness": the "accuracy and completeness with which users achieve specified goals" (IEEE, 2017, p. 153) c. "presentability", implied by "presentable": able to "be retrieved and viewed" (IEEE, 2017, p. 333); probably refers mostly to documentation, etc., but could also apply to code? d. satisfaction-related "qualities"; see below (from ISO/IEC 25010:2011) image

General Testing Questions

  1. "Safety" is described as an "expectation" (IEEE, 2017, p. 397), not a quality, but "safety requirement" (which "ensure the safety of the product" (IEEE, 2017, p. 398)) seems to imply it could be.…
  2. Could "software quality evaluation" (the "systematic examination of the extent to which a software product is capable of satisfying stated and implied needs" (IEEE, 2017, p. 425)) be considered a synonym of "(non-)requirements-based testing"?
  3. As mentioned before about other terms, "software product evaluation" (a "technical operation that consists of producing an assessment of one or more characteristics of a software product according to a specified procedure" (IEEE, 2017, p. 424)) seems too vague to be useful; thoughts?
  4. Are "safety demonstrations", defined by IEEE as "bod[ies] of evidence and rationale that show[] an item is justified as being safe within allowed limits on risk" (2017, p. 397) included in static testing? It seems like the term refers to the process, but the definition refers to the artifact - should this be recorded, even if static testing will be removed from our scope?

Minor Observations

Not super related to my specific research

  1. "Data": plural? singular? I've always treated it as a collective plural, so "data" refers to a "group of datums", and such, use it like "the data is correct", although IEEE2017 (and others?) would use it as "the data are correct".
  2. IEEE2017 also defines the "software design technique" of "input-process-output (IPO)" (p. 226) in case we want to pivot to that term instead of "input-calculate-output (ICO)".

For Later

Overflow from previous meeting(s) so I don't forget to revisit; I'm planning on doing more investigation before we discuss!

  1. Is "data use" (IEEE, 2017, p. 120) a standard term? Can I assume its meaning is well-known, at least for my glossary?
    • We concluded that it is not a well-understood term and should be defined somewhere, but made a note to revisit this
samm82 commented 8 months ago

We decided on the 47th International Conference on Software Engineering (ICSE 2025); if this doesn't pan out, there will likely be a related workshop we can submit to. We'll aim for sometime in June as an internal deadline, and the specific purpose/title of this paper will become apparent as this research progresses, although it will likely be along the lines of "software testing terminology is messy, vague, and unorganized; here is a better way to classify/talk about software testing".

Since software qualities are (likely) going to be out of scope, conversations about what to record and what not to record aren't that fruitful. I should just be recording everything, since it doesn't take that much time/effort, and we can analyze if further if that becomes necessary.

General Testing Questions

  1. "Safety" is a quality, and should be recorded as such, along with this extra information about safety requirements (as a precedence for "safety testing")
  2. Yes, these could be treated as synonyms
  3. As previously mentioned in #39, "evaluation" is too vague to be useful; the same is true of "software product evaluation"; however, this should be explicitly recorded. Software product evaluations are a type of "metamodel" that is outside of our scope
  4. Safety demonstrations are artifacts (I was thinking of demonstrations = demos, not demonstrations = proofs) that should be included in the first pass of my glossary (to be excluded later once static testing is removed as out of scope)