search

akondrahman / 59115ASE

This repository contains all the necessary materials required for grading for CSC 591 (Automated Software Engineering) in Fall 2015 at the Department of CSC at North Carolina State University
1 stars 4 forks source link

Research Questions and Hypothesis to be verified/reviewed #2

Closed manish211 closed 9 years ago

manish211 commented 9 years ago

The paper does not explicitly mention research questions like RQ1,RQ2. Hence the research questions need to be verified/reviewed by the teammates. Same for hypothesis.

timm commented 9 years ago

why not list what you think are the RQs in this comment and let the team debate them here?

bynarron commented 9 years ago

I put that in there for elaboration to keep our notes as congruent as possible from poster to poster. You won't always find research questions, goals, or hypotheses started in a paper, especially if there is no experiment being conducted. There are typically implied version of them though. On Aug 24, 2015 6:09 PM, "Tim Menzies" notifications@github.com wrote:

why not list what you think are the RQs in this comment and let the team debate them here?

— Reply to this email directly or view it on GitHub https://github.com/akondrahman/59115ASE/issues/2#issuecomment-134394825.

bynarron commented 9 years ago

*Paper to paper On Aug 24, 2015 6:16 PM, "Bennett Narron" bynarron@ncsu.edu wrote:

I put that in there for elaboration to keep our notes as congruent as possible from poster to poster. You won't always find research questions, goals, or hypotheses started in a paper, especially if there is no experiment being conducted. There are typically implied version of them though. On Aug 24, 2015 6:09 PM, "Tim Menzies" notifications@github.com wrote:

why not list what you think are the RQs in this comment and let the team debate them here?

— Reply to this email directly or view it on GitHub https://github.com/akondrahman/59115ASE/issues/2#issuecomment-134394825 .

manish211 commented 9 years ago

Here are the questions that I put in our notes.

While pursuing this goal,this paper tries to answer the following research questions:

  1. What is the right level of aggregation to study?
  2. How does the level of study affect the truth, meaning and relevance of the findings?
  3. Are prediction models subject to ecological inference risk?
  4. Is hypothesis testing subject to ecological inference risk?
  5. What are the effects of aggregation on model quality?
  6. Do inferences drawn from models built at aggregated levels transfer to disaggregated levels used to build the aggregations?

Below are the two hypothesis:

bynarron commented 9 years ago

Sounds both applicable and intelligible. I would say, from only having read the abstract yet, it sounds appropriate. On Aug 24, 2015 7:37 PM, "manish211" notifications@github.com wrote:

Here are the questions that I put in our notes.

While pursuing this goal,this paper tries to answer the following research questions:

  1. What is the right level of aggregation to study?
  2. How does the level of study affect the truth, meaning and relevance of the findings?
  3. Are prediction models subject to ecological inference risk?
  4. Is hypothesis testing subject to ecological inference risk?
  5. What are the effects of aggregation on model quality?
  6. Do inferences drawn from models built at aggregated levels transfer to disaggregated levels used to build the aggregations?

Below are the two hypothesis:

  • “Prediction models are subject to ecological inference risk”
  • “Hypothesis testing is subject to ecological risk”

— Reply to this email directly or view it on GitHub https://github.com/akondrahman/59115ASE/issues/2#issuecomment-134414614.

manish211 commented 9 years ago

Let's keep this open until we have read the entire paper! I read the first four pages thoroughly, skimmed through the methodology, findings and conclusion. The questions are relevant but we might want to reduce the number of questions.

akondrahman commented 9 years ago

These are my views, but I might be wrong:

  1. What is the right level of aggregation to study? I agree
  2. How does the level of study affect the truth, meaning and relevance of the findings? Partially agree. To be more accurate, I would say 'which factors associated with aggregated and dis-aggregated level of software affect empirical results '
  3. Are prediction models subject to ecological inference risk? I disagree. I would say 'are empirical findings that propose and test software models, subject to ecological inference risk ?'
  4. Is hypothesis testing subject to ecological inference risk? I disagree. I don't think the paper does that at all.
  5. What are the effects of aggregation on model quality? Falls under point # 3
  6. Do inferences drawn from models built at aggregated levels transfer to disaggregated levels used to build the aggregations? I disagree. I don't think the paper does that at all.
timm commented 9 years ago

profound questions. you (and/or your group) free thurs 3pm to talk about this?

p.s. not for your class work but just so you know it, i got into ecological and local/global in http://menzies.us/pdf/12gense.pdf

akondrahman commented 9 years ago

Professor @timm, I have a class at 3PM on main campus on Thursday. Can we meet some other time at your convenience ?

manish211 commented 9 years ago

@akondrahman Thanks for the review. Here's some pointers.

akondrahman commented 9 years ago

@manish211 I re-read it, I agree with you point # 6.

But, I don't think the authors answered # 4. Even though they say Is hypothesis testing subject to ecological inference risk?, they are not looking at hypotheses testing; they are looking at inferences that are drawn from different levels of analysis. Hypothesis testing is just a tool to do that. I would argue the wording is confusing, at the very least.

I also think Are prediction models subject to ecological inference risk? is a wide claim; there are many prediction models (defect, performance, popularity etc.) specific to a certain application (security in open source software, Github repos, HCI in Software Engg. ); I don't thin we cannot generalize this paper's findings and apply it everywhere. That is still an open research area

Thanks for the feedback. Interesting discussion !

timm commented 9 years ago

; I don't thin we cannot generalize this paper's findings and apply it everywhere. That is still an open research area

fyi: i think we can. i did. i do. see http://menzies.us/pdf/12gense.pdf

timm commented 9 years ago

Professor @timm, I have a class at 3PM on main campus on Thursday. Can we meet some other time at your convenience ?

not this week. icse deadline is friday and there are grants to write and new phd students to process and and and.

maybe we need to grab coffee after class on tuesday? i hate our campus... nowhere we can really go that is not a hike.

akondrahman commented 9 years ago

Professor @timm, I am fine with coffee after today's class.

timm commented 9 years ago

Anyone else in your group can come? I'll get a grad student to truck in coffee and we'll sit on the foyer outside the lecture.

manish211 commented 9 years ago

Dr. Tim, I will join too !

bynarron commented 9 years ago

Dr. Menzies,

I also am able to join after class.

Thanks! Bennett On Aug 25, 2015 10:25 AM, "manish211" notifications@github.com wrote:

Dr. Tim, I will join too !

— Reply to this email directly or view it on GitHub https://github.com/akondrahman/59115ASE/issues/2#issuecomment-134603466.

timm commented 9 years ago

ok. coffee and doughnuts organized.

akondrahman commented 9 years ago

Thanks Professor !

bynarron commented 9 years ago

I was on the fence.... now I think I'm in. har

On Tue, Aug 25, 2015 at 4:26 PM, Tim Menzies notifications@github.com wrote:

ok. coffee and doughnuts organized.

— Reply to this email directly or view it on GitHub https://github.com/akondrahman/59115ASE/issues/2#issuecomment-134730238.