This ticket helps keeping trac of requested changes to the JOSS paper in openjournals/joss-reviews#5040.
Changes to software or docs are handled in separate tickets.
Review by @gaurav
Here are my comments on the GitHub repository so far:
[X] JOSS asks: "Are there clear guidelines for third parties wishing to 1) Contribute to the software 2) Report issues or problems with the software 3) Seek support". While you clearly meet the first requirement in clearly described in CONTRIBUTING.md, I think a few sentences encouraging users to file issues or seek support at https://github.com/OnroerendErfgoed/atramhasis/issues would be nice.
Here are my comments on the manuscript so far:
[X] JOSS wanted me to check whether "the full list of paper authors seem appropriate and complete". I noticed in your repository's contributors list that Maarten Taeymans contributed significantly to this software but isn't credited as a co-author or in the acknowledgements. I will accept your claim if you say that they didn't contribute significantly to the software as it currently exists, but I just wanted to note this in case it was an oversight on your part.
[x] Your first paragraph does a good job of explaining why you built Atramhasis. However, I think a sentence or two on how it compares to other similar tools (such as Skosmos) and what distinguishes it from them would be useful here.
[x] It might be useful to make explicit who your target audience is: do you have a sense of the size of the thesaurus or the number of users that Atramhasis can support? Can a single Atramhasis instance support multiple thesauri? Do you anticipate thesaurus creators running Atramhasis locally on their computer when they want to edit a thesaurus, or do you anticipate an organization setting up an instance that multiple users can use?
[X] "Concept uris" -- I've generally seen "uris" capitalized as "URIs", and I think that makes sense here.
[X] I notice that you reference figures by saying e.g. "edit data in a normal web admin interface \autoref{fig:editingairfields}.". This appears as "... interface Figure 2." in the generated PDF. I would recommend either saying "in a normal web interface, as seen in \autoref{fig:editingairfields}." or "in a normal web interface (\autoref{fig:editingairfields})." to make it a bit more readable.
Review by @SvenLieber
Software paper review
Hi everyone,
thanks for inviting me as a reviewer, the software package seems very interesting! In this comment I would first like to focus on the Software Paper part of the review. Also, because I had issues installing the package. But following the JOSS guidelines I will open a separate issue in the tool's repository shortly and link back to this review-issue.
First of all, I would like to congratulate the authors and all contributors for this software. The tool seems to be actively maintained on GitHub by using different features such as issues and milestones. Additionally, the tool seems to be successfully used in production. I gladly tick the "Substantial Scholarly Effort" box after also have checked the cited peer-reviewed paper from 2017 with contributions of the current authors. That original paper nicely outlines different issues such as efficient algorithms for searching the possibly infinite depth tree structure of thesauri or the different semantics for broader relations and how they are tackled.
The software paper has a clear description of the high-level functionality with given examples. Thus, I would consider it properly described also for non-specialists.
The statement of need highlights that the tool is useful for users without knowledge of SKOS or RDF, it provides a UI abstracting these specifications for two different types of users: users consulting a thesauri and the editors of thesauri. Working in the cultural heritage field myself, I acknowledge that more user-friendly tools are needed. The rest of the section lists several features with some descriptions about why there are useful, e.g. drop-down lists and widgets for end-users or convenient REST services for developers. The sentence about the Linked Data Fragments server could be more descriptive, why was this chosen and what need is fulfilled with it? This would help readers who are not familiar with LDF better understand the decision decision of including it for example compared to a regular SPARQL endpoint.
I mentioned the distinction between users and editors made in the paper, which I find a very interesting point. Here, the paper could outline better who the audiences actually are, currently I find descriptions such as " ..that allows users to create, maintain and consult", "A concept being something a researcher wants to describe", or "editors do not write RDF statements". I think a more clear description of the different types of users and their needs would be beneficial, also with respect to my following point.
There is no dedicated state of the field section. Yet, for SKOS editors there are several existing Open Source solutions with different trade-offs, such as SkoHub, Skosmos or even the more general Protégé, WebProtégé, or VocBench. For example, the paper mentions that "Atramhasis was written to be a lightweight open source SKOS editor", but what is precisely meant with "lightweight"? SkoHub for instance, is basically a static-site generator where all editing flows are realized using Git(Hub). On the one hand this can be considered "lightweight", because not much software or authentication flows need to be implemented. Yet, SkoHub requires knowledge of Git for thesaurus editors which, especially in the cultural heritage field, may represent a hurdle. For a reader and potential user of the presented Atramhasis software, an objective comparison would be helpful (this can be based on very high-level features). Besides the given example of needed knowledge about for example Git vs a dedicated admin user interface for Atramhasis, it might be interesting how Atramhasis differentiates in terms of software license from the other mentioned SKOS editors. A detailed comparison of performance would be out of scope for this submission, but a basic state of the field I find necessary, it also is a review criteria.
Overall, the paper is well written. The references are fine for a paper of this length (besides the fact of the missing state of the field which may require additional references or footnotes to point to other SKOS editors).
I'm happy to answer any questions and help the authors improve their submission. As mentioned, I will provide more comments for the Functionality and Documentation parts of the review.
This ticket helps keeping trac of requested changes to the JOSS paper in openjournals/joss-reviews#5040.
Changes to software or docs are handled in separate tickets.
Review by @gaurav
Review by @SvenLieber