[REVIEW]: NO-DOUBT: A Decision Support Web Platform based on AHP

whedon commented 4 years ago

Submitting author: @gdede-hua (Georgia Dede) Repository: https://github.com/gdede-hua/decision-survey-platform Version: v1.0 Editor: @sbenthall Reviewer: @remram44, @alexssandrofc Archive: Pending

:warning: JOSS reduced service mode :warning:

Due to the challenges of the COVID-19 pandemic, JOSS is currently operating in a "reduced service mode". You can read more about what that means in our blog post.

Status

Status badge code:

HTML: <a href="https://joss.theoj.org/papers/990a6490ba11cc9258d95e33c2c31f46"><img src="https://joss.theoj.org/papers/990a6490ba11cc9258d95e33c2c31f46/status.svg"></a>
Markdown: [![status](https://joss.theoj.org/papers/990a6490ba11cc9258d95e33c2c31f46/status.svg)](https://joss.theoj.org/papers/990a6490ba11cc9258d95e33c2c31f46)

Reviewers and authors:

Please avoid lengthy details of difficulties in the review thread. Instead, please create a new issue in the target repository and link to those issues (especially acceptance-blockers) by leaving comments in the review thread below. (For completists: if the target issue tracker is also on GitHub, linking the review thread in the issue or vice versa will create corresponding breadcrumb trails in the link target.)

Reviewer instructions & questions

@remram44 & @alexssandrofc, please carry out your review in this issue by updating the checklist below. If you cannot edit the checklist please:

Make sure you're logged in to your GitHub account
Be sure to accept the invite at this URL: https://github.com/openjournals/joss-reviews/invitations

The reviewer guidelines are available here: https://joss.readthedocs.io/en/latest/reviewer_guidelines.html. Any questions/concerns please let @sbenthall know.

✨ Please start on your review when you are able, and be sure to complete your review in the next six weeks, at the very latest ✨

Review checklist for @remram44

Conflict of interest

[x] I confirm that I have read the JOSS conflict of interest (COI) policy and that: I have no COIs with reviewing this work or that any perceived COIs have been waived by JOSS for the purpose of this review.

Code of Conduct

[x] I confirm that I read and will adhere to the JOSS code of conduct.

General checks

[x] Repository: Is the source code for this software available at the repository url?
[x] License: Does the repository contain a plain-text LICENSE file with the contents of an OSI approved software license?
[x] Contribution and authorship: Has the submitting author (@gdede-hua) made major contributions to the software? Does the full list of paper authors seem appropriate and complete?
[x] Substantial scholarly effort: Does this submission meet the scope eligibility described in the JOSS guidelines

Functionality

[ ] Installation: Does installation proceed as outlined in the documentation?
[ ] Functionality: Have the functional claims of the software been confirmed?
[x] Performance: If there are any performance claims of the software, have they been confirmed? (If there are no claims, please check off this item.)

Documentation

[x] A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
[x] Installation instructions: Is there a clearly-stated list of dependencies? Ideally these should be handled with an automated package management solution.
[x] Example usage: Do the authors include examples of how to use the software (ideally to solve real-world analysis problems).
[ ] Functionality documentation: Is the core functionality of the software documented to a satisfactory level (e.g., API method documentation)?
[ ] Automated tests: Are there automated tests or manual steps described so that the functionality of the software can be verified?
[ ] Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the software 2) Report issues or problems with the software 3) Seek support

Software paper

[x] Summary: Has a clear description of the high-level functionality and purpose of the software for a diverse, non-specialist audience been provided?
[x] A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
[x] State of the field: Do the authors describe how this software compares to other commonly-used packages?
[ ] Quality of writing: Is the paper well written (i.e., it does not require editing for structure, language, or writing quality)?
[x] References: Is the list of references complete, and is everything cited appropriately that should be cited (e.g., papers, datasets, software)? Do references in the text use the proper citation syntax?

Review checklist for @alexssandrofc

Conflict of interest

[x] I confirm that I have read the JOSS conflict of interest (COI) policy and that: I have no COIs with reviewing this work or that any perceived COIs have been waived by JOSS for the purpose of this review.

Code of Conduct

[x] I confirm that I read and will adhere to the JOSS code of conduct.

General checks

[x] Repository: Is the source code for this software available at the repository url?
[x] License: Does the repository contain a plain-text LICENSE file with the contents of an OSI approved software license?
[x] Contribution and authorship: Has the submitting author (@gdede-hua) made major contributions to the software? Does the full list of paper authors seem appropriate and complete?
[x] Substantial scholarly effort: Does this submission meet the scope eligibility described in the JOSS guidelines

Functionality

[ ] Installation: Does installation proceed as outlined in the documentation?
[ ] Functionality: Have the functional claims of the software been confirmed?
[ ] Performance: If there are any performance claims of the software, have they been confirmed? (If there are no claims, please check off this item.)

Documentation

[x] A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
[x] Installation instructions: Is there a clearly-stated list of dependencies? Ideally these should be handled with an automated package management solution.
[ ] Example usage: Do the authors include examples of how to use the software (ideally to solve real-world analysis problems).
[x] Functionality documentation: Is the core functionality of the software documented to a satisfactory level (e.g., API method documentation)?
[ ] Automated tests: Are there automated tests or manual steps described so that the functionality of the software can be verified?
[ ] Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the software 2) Report issues or problems with the software 3) Seek support

Software paper

[x] Summary: Has a clear description of the high-level functionality and purpose of the software for a diverse, non-specialist audience been provided?
[x] A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
[x] State of the field: Do the authors describe how this software compares to other commonly-used packages?
[ ] Quality of writing: Is the paper well written (i.e., it does not require editing for structure, language, or writing quality)?
[x] References: Is the list of references complete, and is everything cited appropriately that should be cited (e.g., papers, datasets, software)? Do references in the text use the proper citation syntax?

whedon commented 4 years ago

Hello human, I'm @whedon, a robot that can help you with some common editorial tasks. @remram44, @alexssandrofc it looks like you're currently assigned to review this paper :tada:.

:warning: JOSS reduced service mode :warning:

Due to the challenges of the COVID-19 pandemic, JOSS is currently operating in a "reduced service mode". You can read more about what that means in our blog post.

:star: Important :star:

If you haven't already, you should seriously consider unsubscribing from GitHub notifications for this (https://github.com/openjournals/joss-reviews) repository. As a reviewer, you're probably currently watching this repository which means for GitHub's default behaviour you will receive notifications (emails) for all reviews 😿

To fix this do the following two things:

Set yourself as 'Not watching' https://github.com/openjournals/joss-reviews:

watching

You may also like to change your default settings for this watching repositories in your GitHub profile here: https://github.com/settings/notifications

notifications

For a list of things I can do to help you, just type:

@whedon commands

For example, to regenerate the paper pdf after making changes in the paper's md or bib files, type:

@whedon generate pdf

whedon commented 4 years ago

:point_right: Check article proof :page_facing_up: :point_left:

whedon commented 4 years ago

Reference check summary:

OK DOIs

- None

MISSING DOIs

- https://doi.org/10.1007/s11066-010-9051-4 may be missing for title: Evaluation of technological and socio-economic issues affecting the deployment of home networks: Evidence from the ICT-OMEGA project
- https://doi.org/10.1016/s0377-2217(02)00227-8 may be missing for title: Decision-making with the AHP: Why is the principal eigenvector necessary
- https://doi.org/10.1016/j.dss.2005.11.005 may be missing for title: Investigating the effect of website quality on e-business success: An analytic hierarchy process (AHP) approach
- https://doi.org/10.1108/14635770610709031 may be missing for title: An AHP approach in benchmarking logistics performance of the postal industry
- https://doi.org/10.1016/j.ejor.2007.05.001 may be missing for title: The analytic hierarchy process in medical and health care decision making: A literature review
- https://doi.org/10.1016/j.ejor.2014.08.037 may be missing for title: Convergence properties and practical estimation of the probability of rank reversal in pairwise comparisons for multi-criteria decision making problems
- https://doi.org/10.1016/j.ejor.2016.01.059 may be missing for title: Theoretical estimation of the probability of weight rank reversal in pairwise comparisons

INVALID DOIs

- None

remram44 commented 4 years ago

It seems that the paper is missing figures. In addition the references don't render correctly, because the title of most entries are URL, which get capitalized (Https://github.com/airicyu/ahp).

I assume the paper is still being worked on but I thought I'd make a note of it regardless.

sbenthall commented 4 years ago

Does that sound right, @gdede-hua?

gdede-hua commented 4 years ago

@sbenthall https://github.com/sbenthall I have checked the paper and it worked. It also passed the Joss compilation. My apologies. Do you want to revise the paper and resubmit it?

On Fri, Aug 28, 2020 at 2:17 AM Sebastian Benthall notifications@github.com wrote:

Does that sound right, @gdede-hua https://github.com/gdede-hua?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/openjournals/joss-reviews/issues/2605#issuecomment-682238276, or unsubscribe https://github.com/notifications/unsubscribe-auth/AQDL3QLE3FSN2ZQLAGG5GPTSC3SPBANCNFSM4QNKIUGQ .

-- Dr. Georgia Dede Research Associate and Adjunct Lecturer Harokopio University of Athens Department of Informatics and Telematics Tel. +30 210 9549416 email: gdede@hua.gr

remram44 commented 4 years ago

@whedon generate pdf

whedon commented 4 years ago

:point_right: Check article proof :page_facing_up: :point_left:

whedon commented 4 years ago

I'm sorry human, I don't understand that. You can see what commands I support by typing:

@whedon commands

gdede-hua commented 4 years ago

@whedon commands

On Mon, Aug 31, 2020 at 1:34 PM whedon notifications@github.com wrote:

I'm sorry human, I don't understand that. You can see what commands I support by typing:

@whedon commands

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/openjournals/joss-reviews/issues/2605#issuecomment-683699895, or unsubscribe https://github.com/notifications/unsubscribe-auth/AQDL3QJC7ZOLL3S7J2YEC2TSDN4CTANCNFSM4QNKIUGQ .

-- Dr. Georgia Dede Research Associate and Adjunct Lecturer Harokopio University of Athens Department of Informatics and Telematics Tel. +30 210 9549416 email: gdede@hua.gr

whedon commented 4 years ago

Here are some things you can ask me to do:

# List Whedon's capabilities
@whedon commands

# List of editor GitHub usernames
@whedon list editors

# List of reviewers together with programming language preferences and domain expertise
@whedon list reviewers

EDITORIAL TASKS

# Compile the paper
@whedon generate pdf

# Compile the paper from alternative branch
@whedon generate pdf from branch custom-branch-name

# Ask Whedon to check the references for missing DOIs
@whedon check references

# Ask Whedon to check repository statistics for the submitted software
@whedon check repository

gdede-hua commented 4 years ago

@sbenthall I have received the proofs of my paper. May I kindly ask you to inform how I can include the figures in the paper?

On Fri, Aug 28, 2020 at 9:36 AM Georgia Dede gdede@hua.gr wrote:

@sbenthall https://github.com/sbenthall I have checked the paper and it worked. It also passed the Joss compilation. My apologies. Do you want to revise the paper and resubmit it?

On Fri, Aug 28, 2020 at 2:17 AM Sebastian Benthall < notifications@github.com> wrote:

Does that sound right, @gdede-hua https://github.com/gdede-hua?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/openjournals/joss-reviews/issues/2605#issuecomment-682238276, or unsubscribe https://github.com/notifications/unsubscribe-auth/AQDL3QLE3FSN2ZQLAGG5GPTSC3SPBANCNFSM4QNKIUGQ .

-- Dr. Georgia Dede Research Associate and Adjunct Lecturer Harokopio University of Athens Department of Informatics and Telematics Tel. +30 210 9549416 email: gdede@hua.gr

-- Dr. Georgia Dede Research Associate and Adjunct Lecturer Harokopio University of Athens Department of Informatics and Telematics Tel. +30 210 9549416 email: gdede@hua.gr

alexssandrofc commented 4 years ago

Hi @gdede-hua,

I hope to help you, I made available some tips for the development of your paper.

Figures can be included like this: $Caption for example figure.\label{fig:example}$

github_latex_img

As an example available in https://joss.readthedocs.io/en/latest/submitting.html#example-paper-and-bibliography

It is possible to check a preview of your paper at https://whedon.theoj.org/

I also verified that the URLs of the images in your paper.md are incorrect, example of same image i believe:

https://github.com/gdede-hua/decision-survey-platform/blob/master/create_alternatives -> URL in paper.md with error 404. https://github.com/alexandrakisdimi/decision-making-system/blob/master/screenshots/create_alternatives.png -> URL ok in README.md

sbenthall commented 4 years ago

Thank you @alexssandrofc

Are you able to make progress with those instructions, @gdede-hua ?

gdede-hua commented 4 years ago

@alexssandrofc I have made the modifications in my paper according to your instructions, but the whedon paper preview service appears this message:

"Your paper failed to compile with the following errors:

{}"

It seems there is no error but the paper is not compiled. May I kindly ask you to further assist me on this issue?

Thank you.

alexssandrofc commented 4 years ago

@gdede-hua as I checked your paper.md, I believe the error might be on this line:

"}" is missing to close the (\label).

gdede-hua commented 4 years ago

@alexssandrofc Thank you for your help. I have fixed this issue, but it still doesn't work. The message appeared is the same.

alexssandrofc commented 4 years ago

@whedon commands

remram44 commented 4 years ago

The {} part seems like a bug. I'm not sure if it should be reported against whedon or whedon-api, but it certainly makes it hard to fix papers...

alexssandrofc commented 4 years ago

@whedon commands

whedon commented 4 years ago

Here are some things you can ask me to do:

# List Whedon's capabilities
@whedon commands

# List of editor GitHub usernames
@whedon list editors

# List of reviewers together with programming language preferences and domain expertise
@whedon list reviewers

EDITORIAL TASKS

# Compile the paper
@whedon generate pdf

# Compile the paper from alternative branch
@whedon generate pdf from branch custom-branch-name

# Ask Whedon to check the references for missing DOIs
@whedon check references

# Ask Whedon to check repository statistics for the submitted software
@whedon check repository

alexssandrofc commented 4 years ago

@whedon generate pdf

alexssandrofc commented 4 years ago

@whedon check references

whedon commented 4 years ago

I'm sorry human, I don't understand that. You can see what commands I support by typing:

@whedon commands

whedon commented 4 years ago

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

alexssandrofc commented 4 years ago

@gdede-hua i believe the problem was in the API (https://whedon.theoj.org/), when executing the command to generate the PDF, it generated correctly.

gdede-hua commented 4 years ago

@alexssandrofc Thank you very much. I checked the article proof. It's correct.

@sbenthall Thank you for your support. The article proof is correct. So if you wish we are ready to proceed with the review.

sbenthall commented 4 years ago

OK, super @gdede-hua

@remram44 and @alexssandrofc please continue your reviews :)

sbenthall commented 3 years ago

Hello. A gentle reminder to @remram44 and @alexssandrofc to please continue your reviews.

sbenthall commented 3 years ago

Hello. Another reminder to @remram44 and @alexssandrofc to please continue your reviews.

remram44 commented 3 years ago

Apologies for the delay, I am reviewing today.

Some notes before I go into running the software:

The commit history is very short (June 28 - July 9)
- @gdede-hua is the only committer, along with one macro-commit from Dimitrios Alexandrakis 39945023. Hard to tell if the author list is correct from Git data (but it's probably fine if @gdede-hua says it is)
- The commits don't seem to reflect the development history, huge chunks of code were imported in one go. The JOSS guidelines require "not less that three months of work" but I don't think we should judge this from the commit history in this instance
- About 6,000 LoC in total excluding vendored dependencies
Reviewed the paper
- Some minor fixes submitted as a PR: https://github.com/gdede-hua/decision-survey-platform/pull/3 (mostly grammar, punctuation, capitalizing tool and company names)
- The paper claims that many implementations of the algorithm exist, but this is the only AHP tool that offers a web-based interface for both administrators and users, which I think is a sufficient statement of need
- "PWC" should be defined
- One paragraph ends abruptly, it might just be missing a period but maybe that sentence was meant to say more
- "materialized algorithm" seemed odd, but it might just be me

remram44 commented 3 years ago

I am still getting LAPACK etc set up, but going over the code:

There are no tests
Only PHP 5.6 is supported, which is end-of-life since 2018 (PHP 7 was released 2015). This is sure to be a problem going forward
Some content seems to still be in Greek (mostly button labels, some alert() calls)
There is significant code duplication
- en/admin/quest3.php and en/admin/quest_criteria2.php are the same files entirely
- other files share a lot of code, e.g. forms, rendering code. Consider using include or functions to make maintenance possible
- some duplication even within single files, e.g. en/admin/generate_results2.php: lines 144-204 and 265-328 are the same
The flow is also difficult to follow because of naming, e.g. there is a lot of files with similar names quest2.php, quest2db.php, quest3.php, ...
There seems to be lost files like this empty "test" file and some result spreadsheets: Car_buying_decision_Results.xlsx, Choose_the_best_car_for_the_family_Results.xlsx
PHP, HTML, and JS code is mixed in single files, which partly causes the duplication described above. Referencing static JS files from <script> tags and using of HTML templating would make the code more readable and maintainable and would help with duplication (this is considered best practice)

remram44 commented 3 years ago

I managed to build LAPACK, install PHP 5.6, and run the software.

Minor: The name of the system is wrong in the README (decision-making-system)
The apache configuration file is missing (instructions say to "copy the apache/decision-making-system.conf into /etc/apache2/sites-available/" however the file is not in the repository). I created my own and was able to proceed
MD5 is a very weak hashing algorithm and is never appropriate for securing passwords
Most content is in English but some is entirely in Greek, like the "edit research" page
- Screenshot
- This was probably easy to miss because of the code duplication, e.g. when you translated the "create research" page you left the duplicated "edit research" in Greek
I was able to create a research (I did it wrong and had to start over because it seems you can't change criteria and factors from the edit page), however I could not assign a user to it:
"Setting users failed. Error 1"
I was not able to test taking a survey or getting the results

Sorry again for the delay!

sbenthall commented 3 years ago

Thanks for this review, @remram44 !

alexssandrofc commented 3 years ago

@whedon generate pdf

whedon commented 3 years ago

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

alexssandrofc commented 3 years ago

First sorry for the delay in review,

Study with a very interesting topic, as it provides software to help in decision making on an easily accessible platform "WEB", so I would like to leave some contributions.

Below I leave some suggestions regarding the latex document:

-Topic "Statement Of Need", the point is missing at the end of the second paragraph, is there a continuation?

-PWC matrix in "AHP Methodology" has not been defined.

-In the "Sumary", the third paragraph contains the excerpt "materialized algorithms", I recommend changing only for "algorithms".

-Many images in a row without explanation.

alexssandrofc commented 3 years ago

About the technologies used, only the alert when using PHP 5.6 is left, which is out of date and no longer receives official protection updates since January of 2019, making the site vulnerable to cyber attacks.

If possible, I recommend replacing the LAPACK library with another library that meets the needs of AHP, with compatibility with new versions of PHP.

alexssandrofc commented 3 years ago

About the usability of the software:

-In the installation, "decision-making-system.conf" does not exist! -When creating a user logged in as Administrator, the user created does not log in, however when creating a user from the login screen -> register, the registration of the new user occurs correctly. -Recurring error when creating a survey, press the submit button and nothing happens without an error message and does not save the survey. -When editing a survey, the page changes language in the middle of the process, changing from English to Greek. -When completing the search and requesting the result, an "Not Found" error occurs and does not display the "xlsx".

I recommend a new round of tests to detect possible problems on the software .

sbenthall commented 3 years ago

Thank you for this review, @alexssandrofc !

sbenthall commented 3 years ago

@gdede-hua I believe you are allowed to make changes in response to the reviews until the submission passes.

sbenthall commented 3 years ago

Ping @gdede-hua What would you like the next actions to be, with respect to this submission?

gdede-hua commented 3 years ago

@sbenthall we are working on the review of this submission.

On Fri, Dec 18, 2020 at 6:42 PM Sebastian Benthall notifications@github.com wrote:

Ping @gdede-hua https://github.com/gdede-hua What would you like the next actions to be, with respect to this submission?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/openjournals/joss-reviews/issues/2605#issuecomment-748196134, or unsubscribe https://github.com/notifications/unsubscribe-auth/AQDL3QL5QG2HYSM3NPOQKUDSVOA5TANCNFSM4QNKIUGQ .

-- Dr. Georgia Dede Research Associate and Adjunct Lecturer Harokopio University of Athens Department of Informatics and Telematics Tel. +30 210 9549416 email: gdede@hua.gr

sbenthall commented 3 years ago

@gdede-hua How is it going? any updates on this project?

sbenthall commented 3 years ago

@gdede-hua as we've not heard back from you for a while, I'm going to mark this review as paused. If we don't hear back from you in the next month, we will assume you no longer wish to pursue this publication in JOSS and will withdraw this submission on your behalf

arfon commented 3 years ago

@whedon reject

@gdede-hua as we've not heard back from you for a while, I'm going to mark this review as paused. If we don't hear back from you in the next month, we will assume you no longer wish to pursue this publication in JOSS and will withdraw this submission on your behalf

@sbenthall – I've gone ahead and rejected this submission due to the author no longer being responsive for close to 9 months.

@remram44, @alexssandrofc – many thanks for your reviews here. I'm sorry this didn't work out 😞

whedon commented 3 years ago

Paper rejected.

openjournals / joss-reviews