ValentinaIvanova
commented
3 years ago Dear Riccardo,
Thank you for your effort and thorough reporting! There is no need to submit the report in easychair.
Thanks again & all the best
Valentina
From: Riccardo Albertoni notifications@github.com Sent: Monday, 19 October 2020 14:02 To: ISWC-Reproducibility-Track/Paper_607 Paper_607@noreply.github.com Cc: Valentina Ivanova valentina.ivanova@ri.se; Mention mention@noreply.github.com Subject: [ISWC-Reproducibility-Track/Paper_607] Summary of reproduction (#4)
@ValentinaIvanovahttps://github.com/ValentinaIvanova @pminervinihttps://github.com/pminervini: you can find in this issue the short paragraph about my experience in reproducing the submission. Let me know if I should submit this experience in another way than by this GitHub issue. I am asking as I have noticed there is a form available on the easychair system for submitting a review, which however seems not much related to the criteria the reproducibility track, should I fill that form too?
COLAB notebook available at [1] https://colab.research.google.com/drive/1XKk_feh4Eb-fC3FGQ9kGwfiP7KieswT3?usp=sharing [2] https://github.com/riccardoAlbertoni/ISWC_reproducibility607/blob/master/Reproducing607.ipynb
Summary: Difficult in reproducing the paper: 2 (see comments below)
Checkpoints building upon the ISWC reproducibility guidelineshttps://repro.semanticweb.org/ISWC2020/reproducibility-assessment-guide/: (results of the checks are in bold, and details can be found in the aforementioned notebook)
System:
- How to obtain the system? (OK)
- How to configure the environment if need be (e.g., environment variables, paths)? (N/A, not critical, just usual classpath, perhaps could it be made explicit using -cp for including the target?)
- How to compile the system? (existing compilation options should be mentioned and why they are needed) (OK Java and Maven)
- How to use the system? (What are the configuration options and parameters to the system and where to set them?) (OK)
- How to make sure that the system is installed correctly? (OK)
Software Reproducibility:
- I ran the server locally on my laptop following the instructions available at https://oba.readthedocs.io/en/latest/server/ .
- Generating OpenAPI for ModelCatalog and DBPEDIA music (OK)
- Unit Tests: see section "Running JUNIT tests" (OK, locally using eclipse and importing the project as a Maven project)
- Integration Test: see section "Testing the server locally on my laptop" in the notebook. The API has been checked
- via browser by accessing the http://localhost:8080/v1.3.0/ui/ (OK)
- by using the automated test via Tox (OK, perhaps some extra explanations could help. See suggestions in section "Useful explanation to add to https://oba.readthedocs.io/en/latest/test/")
Comments: I have successfully reproduced paper 607, which describes the Ontology-Based APIs framework (OBA). The framework supports to automatically create REST APIs from ontologies while following RESTful API best practices. Given an ontology, OBA uses standard technologies familiar to web developers (OpenAPI Specification, JSON) and combines them with W3C standards (OWL, JSON-LD frames, and SPARQL) to create maintainable APIs with documentation, unit tests, automated validation of resources and clients (in Python, Javascript, etc.) for non-semantic Web experts to access the content of a target knowledge graph.
The OBA framework mixes in-house development with existing technology ( e.g., Tox, Docker, Flask). Reusing existing pieces of software instead of reinventing the wheel is an established good practice, but legitimately, it also introduces complexity when trying to reproduce the described software. I was not proficient in all the technologies adopted, and I think that is why interactions with the authors have been necessary. Anyway, the authors have replied to all my comments (see issues #1https://github.com/ISWC-Reproducibility-Track/Paper_607/issues/1, #3https://github.com/ISWC-Reproducibility-Track/Paper_607/issues/3) and integrated the documentation to include suggestions and aspects that help make the reuse of the software easier.
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://github.com/ISWC-Reproducibility-Track/Paper_607/issues/4, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ABDGAIZOGDOKQPDM2RQDW6DSLQTGHANCNFSM4SWBQOBQ.
@ValentinaIvanova @pminervini: you can find in this issue the short paragraph about my experience in reproducing the submission. Let me know if I should submit this experience in another way than by this GitHub issue. I am asking as I have noticed there is a form available on the easychair system for submitting a review, which however seems not much related to the criteria the reproducibility track, should I fill that form too?
COLAB notebook available at [1] https://colab.research.google.com/drive/1XKk_feh4Eb-fC3FGQ9kGwfiP7KieswT3?usp=sharing [2] https://github.com/riccardoAlbertoni/ISWC_reproducibility607/blob/master/Reproducing607.ipynb
Summary: Difficult in reproducing the paper: 2 (see comments below)
Checkpoints building upon the ISWC reproducibility guidelines: (results of the checks are in bold, and details can be found in the aforementioned notebook)
System:
Software Reproducibility:
Comments: I have successfully reproduced paper 607, which describes the Ontology-Based APIs framework (OBA). The framework supports to automatically create REST APIs from ontologies while following RESTful API best practices. Given an ontology, OBA uses standard technologies familiar to web developers (OpenAPI Specification, JSON) and combines them with W3C standards (OWL, JSON-LD frames, and SPARQL) to create maintainable APIs with documentation, unit tests, automated validation of resources and clients (in Python, Javascript, etc.) for non-semantic Web experts to access the content of a target knowledge graph.
The OBA framework mixes in-house development with existing technology ( e.g., Tox, Docker, Flask). Reusing existing pieces of software instead of reinventing the wheel is an established good practice, but legitimately, it also introduces complexity when trying to reproduce the described software. I was not proficient in all the technologies adopted, and I think that is why interactions with the authors have been necessary. Anyway, the authors have replied to all my comments (see issues #1, #3) and integrated the documentation to include suggestions and aspects that help make the reuse of the software easier.