acmsigsoft / EmpiricalStandards

Tools and standards for conducting and evaluating research in software engineering
https://acmsigsoft.github.io/EmpiricalStandards/
Creative Commons Zero v1.0 Universal
288 stars 59 forks source link

Method selection decision tree #34

Open drpaulralph opened 3 years ago

drpaulralph commented 3 years ago

Create a flow chart or decision tree to help users determine which standard(s) to select. (Suggestions welcome).

rindPHI commented 3 years ago

Do you know about any kind of statistics about how frequently a given research method occurs in practice? For instance, Engineering Research could apply for much more empirical SE papers than, e.g., Quantitative Simulation. In that case, one should begin the decision tree with a question regarding the characteristics of Engineering Research rather than Quantitative Simulation.

drpaulralph commented 3 years ago

Many SLRs classify primary studies in terms of methods used, and I think there's an ICSE paper that classifies all the ICSE papers for several years, but you have to take it all with a grain of salt because many, many SE papers misrepresent the methods they're using. Quantitative simulations are often referred to as "experiments" or "case studies". Action research is rarely called action research. Engineering research is typically mislabled "design research" (don't get me started on this). Qualitative surveys are often mislabeled as grounded theory, etc.

rindPHI commented 3 years ago

I see 😬 Still, such a classification could be useful for designing the decision tree; it won't make it incorrect, rather influence its structure. I just sent 10 minutes trying to find that ICSE paper you mentioned, unfortunately unsuccessful. If I find some time, I'll continue the search and maybe draft a decision tree, but can't promise anything ;)