Submission 4 (Research area - Project paper)

[AntonioCerone]

Towards a Web-based Tool for Supporting Research Collaboration in Human-Computer Interaction and Cognitive Science

Abstract Human-computer interaction and cognitive science are interdisciplinary areas in which computer scientists and mathematicians often work together with social scientist, such as psychologists and sociologists, as well as with more focussed practitioners such as usability experts and system analysts. In order to work effectively, interdisciplinary teams need to agree on a common communication language as a compromise between the computer scientists and mathematicians' formal modelling approach and the conceptual models normally used by social scientists for describing their domain-related theories and frameworks. Moreover, even when proper communication is established within a specific research team, the next challenge is the presentation of the result to a heterogeneous international community, to allow for cross-fertilisation, exchanges of ideas, work replication and review. This project paper presents the ongoing development of a web-based tool for the modelling and analysis of human cognition and behaviour as well as interactive systems. The aim of the project is for researchers in human-computer interaction and cognitive science to freely use the tool provided by the web portal in order to run in silico experiments, compare the results of in silico experiments and experiments with human beings, perform simulations, analyse system consisting of computer/physical components and human components, as well as download and upload datasets and models. Domain oriented modelling and visualisation interfaces will ease the modelling and analysis processes by hiding the simulation and formal analysis engines. Finally, the tool will facilitate discussion, review and collaboration. An early prototype of the tool will be available by the end of November 2021.

[paddykrishnan]

Is this related to Open Source Certification? Having said that I can review it.

[kpapanikolaou]

I will review this submission. Kyparissia Papanikolaou

[twasserman]

I can also review this submission.

[AntonioCerone]

The first three reviewers for this paper are:

• Paddy Krishnan
• Kyparissia Papanikolaou
• Anthony Wasserman

Two more reviewers will be added tomorrow.

The PDF of the paper is: OpenCERT_2021_paper_4.pdf

[marte-git]

The final reviewer assignment for this paper is:

• Paddy Krishnan
• Kyparissia Papanikolaou
• Anthony Wasserman
• Peter T. Breuer
• Andreas Meiszner
• John Noll

The PDF of the paper is: OpenCERT_2021_paper_4.pdf

[twasserman]

This paper attempts to explain BRDL and the architecture of a web portal currently under development. In this reviewer's opinion, neither of these efforts is successful. The architecture is just a collection of boxes that suggests a client server architecture system interacting with a database. We don't see what information is passed between the boxes. Also, it appears to be a closed system, with no support for external APIs or third party connections. The different types of entities are identified, but the dataflow (?) among them and the Models does not seem to correspond with the connections described in the body of the paper (Section 2).

Sections 3 and 4 depend on knowledge of topics that are poorly explained here. I found it impossible to comprehend Section 4.1 based on the extremely compact notational description of the various entity types in the previous sections, with the result being that I was unable to learn much from the paper without extensive study of the background topics.

Section 5 makes it clear that the web portal is still under development and that there are many tasks to be completed before it is possible to test the model proposed by the authors. The title of the paper refers to human-computer interaction, but I could see no evidence of related work in the body of the paper.

In short, the ideas are poorly explained for those without deep knowledge of the authors' previous work. The paper will be improved when the authors are able to test their ideas on a real system and demonstrate the value of their models (entities, etc.).

If the authors revise the paper for a future conference or journal, it also needs some English language editing. (Look at the title of Section 3).

[opencert]

Thank you Anthony for your comments. While waiting for comments from the other reviewers, we remind all reviewers that this submission is a Project paper, to describe a new open community project (e.g. on a hosting provider or a dedicated portal) or a new research project, or the status of an ongoing project or the outcomes of a recently completed project.

[opencert]

We would like to clarify that the current interactive phase of the review process is for the reviewers to provide comments that help authors to improve their work. The issue on whether to accept or reject the submission should not be mentioned during this phase. It will be considered, discussed and finalised within a closed meeting during the assessment phase, which will be carried out on EasyChair.

[kpapanikolaou]

I agree with twasserman :) the paper is hard to follow, you need to know the particular work in order to understand the usefulness and innovation of the authors' proposal in the particular context. The paper looks like a part of a project proposal missing the 'proposal objectives and challenges'. It is hard to understand the particular challenges that interdisciplinary teams such as those consisting of cognitive scientists and human-computer interaction experts, face and the way the proposed architecture & the functionalities offered help in dealing with them. Application paradigms could improve the understandability of the paper and the proposal.

[marte-git]

as mentioned in a previous comment, we are currently in an interactive reviewing phase, when the reviewers do comment the current submission, with the intent to help making the submission better.

So we encourage the other reviewers to send their comments, and we also encourage the authors to profit from the comments and work on improvements of their work.

If the authors make changes in the paper, to meet some reviewers' comments, they can attach here (yes, in a comment) the improved version, and maybe explain the changes.

Thanks everybody.

[paddykrishnan]

I agree with the previous comments that the contributions made in the paper are hard to follow. The paper seems to have various ideas that builds on their previous work. Since one cannot assume that readers are familiar with this work, it will be good to have a summary and clearly mark what is the novel contribution.

The authors state "``` The portal will allow researchers to collaborate in modelling, compare each other’s models, replicate in silico experiments and perform reviewing activities."

But I see no real evaluation to justify this claim. So it appears that this is purely aspirational.  While I recognise this is work in progress, the paper needs to have some clear explanation why this goal is likely to be met. I suggest an example be used to show such a tool if open-sourced will help address the collaboration problem.

[AntonioCerone]

Dear Reviewers, Thank you very much for your comments and feedback. We realised that our submission was not well received and we decided to withdraw it. We will resubmit it to a more appropriate venue.

Antonio Cerone