search

researchart / rose6icse

12 stars 71 forks source link

icse20-main-1239-heterorefactor #111

Closed minkull closed 4 years ago

minkull commented 4 years ago

https://github.com/researchart/rose6icse/tree/master/submissions/available/icse20-main-1239-heterorefactor https://github.com/researchart/rose6icse/tree/master/submissions/reusable/icse20-main-1239-heterorefactor

Note to reviewers: these authors want multiple badges

icse2020review commented 4 years ago

The authors provide the source code of the HeteroRefactor tool along with detailed instructions to reproduce results presented in the paper.

Given that HeteroRefactor depends on various OS-specific libraries, I am wondering if the authors can provide a ready to use VM image of HeteroRefactor in addition to the source code.

timm commented 4 years ago

@miryung : @icse2020review is having start up issues with your code. is a docker image available?

@icse2020review : without executables, can you at least assess everything else?

timm commented 4 years ago

@andromeda04 : I look forward to your review

andromeda04 commented 4 years ago

Dear authors:

Thanks for submitting the artifact supporting your paper. In the below, please find the results of my efforts in trying to follow your instructions in the install.md file:

As I executed the make, it gave me an error stating: /usr/share/aclocal/ltdl.m4' does not exist.

I searched Google for a solution and found the following link: https://github.com/forthy42/gforth/issues/7

It ended up working by executing: $ sudo apt-get install libltdl-dev

After successfully installing libltdl-dev I executed the make again in order to install the artifact, but I get the following error:

mkdir -p /home/ANON/Escritorio/5-icse20-main-1239-heterorefactor/icse2020-artifacts/heterorefactor/libraries/rose-compiler//build/rose

cd /home/ANON/Escritorio/5-icse20-main-1239-heterorefactor/icse2020-artifacts/heterorefactor/libraries/rose-compiler//build/rose && /home/ANON/Escritorio/5-icse20-main-1239-heterorefactor/icse2020-artifacts/heterorefactor/libraries/rose-compiler//source/configure --enable-static --prefix=/home/ANON/Escritorio/5-icse20-main-1239-heterorefactor/icse2020-artifacts/heterorefactor/libraries/rose-compiler//dist \

--enable-languages=c,c++ --with-boost=/home/ANON/Escritorio/5-icse20-main-1239-heterorefactor/icse2020-artifacts/heterorefactor/libraries/rose-compiler//build/boost --without-java --enable-edg_version=4.12

/bin/sh: 1: /home/ANON/Escritorio/5-icse20-main-1239-heterorefactor/icse2020-artifacts/heterorefactor/libraries/rose-compiler//source/configure: not found

Makefile:14: recipe for target '/home/ANON/Escritorio/5-icse20-main-1239-heterorefactor/icse2020-artifacts/heterorefactor/libraries/rose-compiler//build/rose' failed

make[3]: *** [/home/ANON/Escritorio/5-icse20-main-1239-heterorefactor/icse2020-artifacts/heterorefactor/libraries/rose-compiler//build/rose] Error 127

make[3]: se sale del directorio '/home/ANON/Escritorio/5-icse20-main-1239-heterorefactor/icse2020-artifacts/heterorefactor/libraries/rose-compiler'

Makefile:12: recipe for target '/home/ANON/Escritorio/5-icse20-main-1239-heterorefactor/icse2020-artifacts/heterorefactor/libraries//rose-compiler/dist' failed

make[2]: *** [/home/ANON/Escritorio/5-icse20-main-1239-heterorefactor/icse2020-artifacts/heterorefactor/libraries//rose-compiler/dist] Error 2

make[2]: se sale del directorio '/home/ANON/Escritorio/5-icse20-main-1239-heterorefactor/icse2020-artifacts/heterorefactor/libraries'

Makefile:5: recipe for target 'all' failed

make[1]: *** [all] Error 2

make[1]: se sale del directorio '/home/ANON/Escritorio/5-icse20-main-1239-heterorefactor/icse2020-artifacts/heterorefactor'

Makefile:5: recipe for target 'all' failed

make: *** [all] Error 2

root@ANON:/home/ANON/Escritorio/5-icse20-main-1239-heterorefactor/icse2020-artifacts#

In the attached log3.txt file you will find the entire execution trace.

I tried several times but I was unable to install and use your artifact. I agree with @icse2020review that a ready to use VM image of HeteroRefactor (in addition to the source code) would be helpful.

dotkrnl commented 4 years ago

@andromeda04 Thank you for letting us know this issue! It seems like the Make program somehow skips the target $(mkfile_dir)/source under heterorefactor/libraries/rose-compiler which is used to fetch the source code of ROSE and generate configure file. I cannot reproduce this problem in my environment from a clean clone.

My guess is that the dependency problem of libltdl-dev failed the ./build command of ROSE. After you installed this library, my Makefile did not consider this issue so it thought the ./build succeeded (because the folder is there). Therefore, it tried to use the resulting files configure of it and failed because it was not generated. A possible solution is to completely remove the cloned repository and make again, but you don't have to do it as I just fixed this issue.

I have modified my Makefile to consider the possibility of failing ./build and instruct it to retry when it fails. I have also added the missing dependency to the README.md file. The most recent commit should resolve this issue. You can pull this change and continue your make.

Please let me know if it works. If it still fails, I will build a VM image for you --- does an AWS AMI image work for you?

timm commented 4 years ago

@icse2020review @andromeda04

andromeda04 commented 4 years ago

@timm: The artifact is publicly available at Github (https://github.com/UCLA-VAST/icse2020-artifacts), but the authors did not released it on Zenodo (or similar repo) in order to get a DOI. Therefore, I cannot recommend the “available” badge.

dotkrnl commented 4 years ago

@andromeda04 Can we release it on Zenodo now?

andromeda04 commented 4 years ago

Dear authors,

I tried again to install and use the artifact, but another error in the make ocurred after 1:45h of execution. You can see the reported error in this file: ICSE 2020-main-1239-heterorefactor.pdf

Conclusion: The HeteroRefactor artifact cannot be reproduced, and I cannot recommed the "reusable" badge.

dotkrnl commented 4 years ago

@andromeda04 are you connected to the internet? the building of ROSE compiler requires an internet connection to download a tarball from their website.

dotkrnl commented 4 years ago

@andromeda04 BTW, in my understanding, a DOI is not required for the available badge. Per requirement: placed on a publicly accessible archival repository. A DOI or link to this repository along with a unique identifier for the object is provided., a link is also accepted.

dotkrnl commented 4 years ago

@andromeda04 We registered our release on Zenodo: https://zenodo.org/record/3675842 The DOI is 10.1145/3377811.3380340

minkull commented 4 years ago

It seems this is heading towards available but not reusable.

@andromeda04 @icse2020review please could you confirm?

andromeda04 commented 4 years ago

@dotkrnl: yes, I was connected to the internet. I tried several times to install and use your artifact but I got different errors in the make.

@minkull: Unfortunately I cannot recommend the "reusable" badge, but I am glad to recommend the available badge as the artifact has been released in Zenodo (note that last year the DOI was mandatory to get the "available" badge).

dotkrnl commented 4 years ago

@andromeda04 I suppose this might due to an international internet connection problem. Thanks for trying anyways! :-)

minkull commented 4 years ago

Many thanks!

Available to be awarded by end of 24th Feb AoE unless @icse2020review shouts otherwise.

minkull commented 4 years ago

Available awarded.