hanam_patterns

[yasutaka-kamei]

@qhanam

I've cloned bugaid and tried mvn install, but failed it. Would you please help me to solve the problem? I've attached two log files when I conduct mvn install and mvn --version

mvninstall-X.txt mvn_--version.txt

[qhanam]

Thanks for bringing this up. Not sure what the problem is yet, but we'll have a look this weekend.

[qhanam]

@yasutaka-kamei I've updated the build dependencies. Please re-build with mvn -U clean install.

Also, the instructions for running BugAID have been updated in the README at https://github.com/saltlab/BugAID

[yasutaka-kamei]

@qhanam thank you for updating the build dependencies. I re-build with mvn -U clean install, but got another error.

mvn_-U_clean_install.txt

[qhanam]

@yasutaka-kamei we haven't been able to reproduce this error. I've run it on a few different machines now, including Windows. Is there another machine you could run it on? The only thing I can think of is that there is another version of Gumtree that is cached in your local maven repo.

Otherwise, the executable isn't a super important part of the artifact.

[yasutaka-kamei]

@qhanam when I use another machine with Ubuntu 10.4 + JDK 8.0, mvn install works. Thank you so much for your effort to solve my issue!

Btw, I could not run on the machine with JDK 6, which is originally installed in Ubuntu 10.4. I share this information with you for people who may later face the same issue I got.

[qhanam]

Oh good! Yes, BugAID requires JDK 8 to run.

[brusso123]

Dear authors,

The artefact is to me the most insightful, useful, and usable artefact that I reviewed. It is useful to researchers that can use the tool on their own Java script programs, navigate the results and use the bug patterns as benchmark of their analysis from the online web page and contribute to the tool from github.

The description of how to use the tool is in github. There is also a very detailed and interactive page that helps the users install and run the tool. Buttons on the main page do not work though. To use and update the tool one installs and uses maven. The tool is portable on JVM.

Barbara

[yasutaka-kamei]

This artifact paper provides a replication package including an executable for reproducing the dataset of the author's FSE'16 paper, the raw data and a graphical web interface for exploring the data.

• Insightful
  • Timely (i.e., addresses a problem that is most current and most pressing)? Yes. This artifact paper provides a replication package for the author's FSE'16 paper titled "Discovering Bug Patterns in JavaScript". JavaScript is one of the languages that are frequently used in GitHub. Therefore, the analysis of JavaScript is a very timely topic in research and in practice.
  • Makes researchers “smarter” in some way (e.g., identifies and fills some significant gap in prior work)? Yes. This artifact paper encourages researchers to explore more bug patterns in JavaScript. Discovering more bug patterns makes researchers smarter.
• Useful
  • Serves a useful purpose? Yes. As I described in Insightful, researchers can explore more bug patterns in JavaScript. It is a useful purpose.
  • Cost-effective? Yes. Researchers can save their effort to re-implement the same tool and reproduce the same results.
• Usable
  • Easy to understand? The graphical web interface provided at http://salt.ece.ubc.ca/software/bugaid/ helps users to understand and explore the data.
  • Accompanied by tutorial notes? Yes. README.md mentions how to build BugAID using Maven. Please describe "BugAID requires JDK 8 to run" in README.
  • Artifacts need not be executable but if so, are they: Yes.
  • Easy to download, install, or execute? Yes. However, I cannot access the three URLs that are linked at http://salt.ece.ubc.ca/software/bugaid/ due to 404 error.
  • http://salt.ece.ubc.ca/software/bugaid/bugaid/results/tech-report.pdf
  • http://salt.ece.ubc.ca/software/bugaid/bugaid/results/tuning_data.tar.gz
  • http://salt.ece.ubc.ca/software/bugaid/bugaid/results/empirical_data.tar.gz
  • Available in a virtual machine image? No.
  • Available online? Yes.
  • Supported by configuration management tools to permit easy updates? Yes. The tool is supported by Git and Maven. Therefore, it can be easily updated.
• Score I think that this artifact is insightful, useful and usable. So I would like to say "Platinum" for this paper.

[timm]

Note these labels are still "under discussion" and are still subject to change prior to the final notifications Friday.

[shaiduc]

[Recommendation]: Maybe Platinum

The paper does not follow the instructions provided in the CFP. The first page must describe the artifact and the scorecard dimensions, and the second page must show how to use the artifact. However, the first page of this paper includes instructions about how to use the artifact, and the second contains the scorecard dimension. Reordering the pages is enough to comply with the CFP.

[Insightful]

• Timely: Web development practitioners use JavaScript + HTML + CSS as the major technologies to develop their apps. As JavaScript is an important element of this set, tools and datasets that allow to analyze large sets of web applications are always needed. Also, the bug dataset can guide developers in designing good programming language features and APIs.
• Makes researchers smarter: It provides a good picture of the most frequent bugs present in JavaScript code, which can inspire researchers and help them advance their research if they are working on this topic.

[Useful]:

• Serves a useful purpose?: Definitely. The tool and the bug dataset will help researchers understand bug patterns in JavaScript.
• Serves a purpose that would otherwise be tedious, prolonged, awkward, or impossible? Collecting this dataset is a time-consuming task. Now it is available for general use.
• Cost-effective? The code is ready for further analysis and execution. However, it seems that a single execution takes a long time. Perhaps the provided code is not suitable for repeated executions.

[Usable]:

• Easy to understand? Yes. There is a web page that explains each component of this set of artifacts. Also, the web page implements search boxes that help navigate thought the set of results.
• Accompanied by tutorial notes? The web page does a good job explaining the components. Also, the tool is hosted in GitHub, including a README file with additional instructions about how to execute it.
• Easy to download: Yes. It is in GitHub. However, it seems that the raw results are not available. Users can get the results only by navigating the provided web page.
• Available in a virtual machine image? No. It does not need it.
• Supported by configuration management tools to permit easy updates? The tool uses GitHub and Maven which help with the configuration process.

[obaysal]

@timm since @shaiduc agrees with "platinum", I am keeping the label as is.