[REVIEW]: BigX: A geographical dataset visualisation tool

[whedon]

Submitting author: @geekysquirrel (Stefanie Wiegand) Repository: https://gitlab.com/geekysquirrel/bigx Version: 2.1.1. Editor: @hugoledoux Reviewer: @liberostelios, @jvdkwast Archive: 10.5281/zenodo.4272105

:warning: JOSS reduced service mode :warning:

Due to the challenges of the COVID-19 pandemic, JOSS is currently operating in a "reduced service mode". You can read more about what that means in our blog post.

Status

Status badge code:

HTML: <a href="https://joss.theoj.org/papers/2cddb2f3d1006c1b2ad48af1d9c77d00"><img src="https://joss.theoj.org/papers/2cddb2f3d1006c1b2ad48af1d9c77d00/status.svg"></a>
Markdown: [![status](https://joss.theoj.org/papers/2cddb2f3d1006c1b2ad48af1d9c77d00/status.svg)](https://joss.theoj.org/papers/2cddb2f3d1006c1b2ad48af1d9c77d00)

Reviewers and authors:

Please avoid lengthy details of difficulties in the review thread. Instead, please create a new issue in the target repository and link to those issues (especially acceptance-blockers) by leaving comments in the review thread below. (For completists: if the target issue tracker is also on GitHub, linking the review thread in the issue or vice versa will create corresponding breadcrumb trails in the link target.)

Reviewer instructions & questions

@liberostelios & @jvdkwast, please carry out your review in this issue by updating the checklist below. If you cannot edit the checklist please:

1. Make sure you're logged in to your GitHub account
2. Be sure to accept the invite at this URL: https://github.com/openjournals/joss-reviews/invitations

The reviewer guidelines are available here: https://joss.readthedocs.io/en/latest/reviewer_guidelines.html. Any questions/concerns please let @hugoledoux know.

✨ Please start on your review when you are able, and be sure to complete your review in the next six weeks, at the very latest ✨

Review checklist for @liberostelios

Conflict of interest

• [x] I confirm that I have read the JOSS conflict of interest (COI) policy and that: I have no COIs with reviewing this work or that any perceived COIs have been waived by JOSS for the purpose of this review.

Code of Conduct

• [x] I confirm that I read and will adhere to the JOSS code of conduct.

General checks

• [x] Repository: Is the source code for this software available at the repository url?
• [x] License: Does the repository contain a plain-text LICENSE file with the contents of an OSI approved software license?
• [x] Contribution and authorship: Has the submitting author (@geekysquirrel) made major contributions to the software? Does the full list of paper authors seem appropriate and complete?
• [x] Substantial scholarly effort: Does this submission meet the scope eligibility described in the JOSS guidelines

Functionality

• [x] Installation: Does installation proceed as outlined in the documentation?
• [x] Functionality: Have the functional claims of the software been confirmed?
• [x] Performance: If there are any performance claims of the software, have they been confirmed? (If there are no claims, please check off this item.)

Documentation

• [x] A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
• [x] Installation instructions: Is there a clearly-stated list of dependencies? Ideally these should be handled with an automated package management solution.
• [x] Example usage: Do the authors include examples of how to use the software (ideally to solve real-world analysis problems).
• [x] Functionality documentation: Is the core functionality of the software documented to a satisfactory level (e.g., API method documentation)?
• [x] Automated tests: Are there automated tests or manual steps described so that the functionality of the software can be verified?
• [x] Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the software 2) Report issues or problems with the software 3) Seek support

Software paper

• [x] Summary: Has a clear description of the high-level functionality and purpose of the software for a diverse, non-specialist audience been provided?
• [x] A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
• [x] State of the field: Do the authors describe how this software compares to other commonly-used packages?
• [x] Quality of writing: Is the paper well written (i.e., it does not require editing for structure, language, or writing quality)?
• [x] References: Is the list of references complete, and is everything cited appropriately that should be cited (e.g., papers, datasets, software)? Do references in the text use the proper citation syntax?

Review checklist for @jvdkwast

Conflict of interest

• [x] I confirm that I have read the JOSS conflict of interest (COI) policy and that: I have no COIs with reviewing this work or that any perceived COIs have been waived by JOSS for the purpose of this review.

Code of Conduct

• [x] I confirm that I read and will adhere to the JOSS code of conduct.

General checks

• [x] Repository: Is the source code for this software available at the repository url?
• [x] License: Does the repository contain a plain-text LICENSE file with the contents of an OSI approved software license?
• [x] Contribution and authorship: Has the submitting author (@geekysquirrel) made major contributions to the software? Does the full list of paper authors seem appropriate and complete?
• [x] Substantial scholarly effort: Does this submission meet the scope eligibility described in the JOSS guidelines

Functionality

• [x] Installation: Does installation proceed as outlined in the documentation?
• [x] Functionality: Have the functional claims of the software been confirmed?
• [x] Performance: If there are any performance claims of the software, have they been confirmed? (If there are no claims, please check off this item.)

Documentation

• [x] A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
• [x] Installation instructions: Is there a clearly-stated list of dependencies? Ideally these should be handled with an automated package management solution.
• [x] Example usage: Do the authors include examples of how to use the software (ideally to solve real-world analysis problems).
• [x] Functionality documentation: Is the core functionality of the software documented to a satisfactory level (e.g., API method documentation)?
• [ ] Automated tests: Are there automated tests or manual steps described so that the functionality of the software can be verified?
• [x] Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the software 2) Report issues or problems with the software 3) Seek support

Software paper

• [x] Summary: Has a clear description of the high-level functionality and purpose of the software for a diverse, non-specialist audience been provided?
• [x] A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
• [x] State of the field: Do the authors describe how this software compares to other commonly-used packages?
• [x] Quality of writing: Is the paper well written (i.e., it does not require editing for structure, language, or writing quality)?
• [x] References: Is the list of references complete, and is everything cited appropriately that should be cited (e.g., papers, datasets, software)? Do references in the text use the proper citation syntax?

[whedon]

Hello human, I'm @whedon, a robot that can help you with some common editorial tasks. @liberostelios, @jvdkwast it looks like you're currently assigned to review this paper :tada:.

:warning: JOSS reduced service mode :warning:

Due to the challenges of the COVID-19 pandemic, JOSS is currently operating in a "reduced service mode". You can read more about what that means in our blog post.

:star: Important :star:

If you haven't already, you should seriously consider unsubscribing from GitHub notifications for this (https://github.com/openjournals/joss-reviews) repository. As a reviewer, you're probably currently watching this repository which means for GitHub's default behaviour you will receive notifications (emails) for all reviews 😿

To fix this do the following two things:

1. Set yourself as 'Not watching' https://github.com/openjournals/joss-reviews:

2. You may also like to change your default settings for this watching repositories in your GitHub profile here: https://github.com/settings/notifications

For a list of things I can do to help you, just type:

@whedon commands

For example, to regenerate the paper pdf after making changes in the paper's md or bib files, type:

@whedon generate pdf

[whedon]

Reference check summary:

OK DOIs

- 10.5258/SOTON/P0014 is OK

MISSING DOIs

- None

INVALID DOIs

- None

[whedon]

:point_right: Check article proof :page_facing_up: :point_left:

[hugoledoux]

👋 @geekysquirrel it took some time (apologies) but now there are two knowledgeable reviewers and the review is starting.

One comment on my part, perhaps you could comment here (or update the paper): I was wondering for which use-cases you see people using your work, and not QGIS for example? I find QGIS good/powerful/etc and it's free, so wondering where does your work fit and what is "simpler" with it.

[hugoledoux]

/ooo August 1 until August 16

[ooo[bot]]

:+1: Marked @hugoledoux as OOO from Saturday, August 1st 2020 to Sunday, August 16th 2020. :calendar:

[geekysquirrel]

Sorry @hugoledoux the notifications did not work for me.

One use case where my team is currently using BigX is to publish preliminary results online for other people on the project to use.

I am aware there's the QGIS cloud but a) it's not clear where exactly the data is stored which is a problem for data that contains personal information (GDPR!) and b) restricting access to QGIS Cloud maps seems a pro feature whereas with BigX you can simply add usernames/passwords (in the simplest instance with a .htaccess file)

I hope that makes sense.

[ooo[bot]]

:wave: Hey @geekysquirrel...

Letting you know, @hugoledoux is currently OOO until Sunday, August 16th 2020. :heart:

[hugoledoux]

I wasn't aware of QGIS Cloud and the pricing option. So yes it does make sense.

Could you add this somehow to the paper? This seems important, and it positions the paper well wrt to others.

[geekysquirrel]

I added a sentence explaining it although I tried to keep it as brief as possible as it still looks ever so slightly too long.

[jvdkwast]

I went through the review checklist. The software runs fine on my laptop and I could play around with the configuration based on the documentation. Here are a few questions/remarks following the order of the checklist.

Documentation:

• I've tried to run the tests as descrobed here. But unit test and lint failed on the default local installation in Electron. Maybe it has to do with my lack of knowledge of this.
• I miss explicit community guidelines for third parties wishing to 1) Contribute to the software 2) Report issues or problems with the software 3) Seek support

Software paper:

• Related to the earlier comments of @hugoledoux: it is not clear enough why the users that you describe would choose for this tool over for example QGIS Desktop. In my opinion the effort of visualising GIS layers in QGIS is much lower than configuring the scripts in BigX, for example for styling the layers. Also for online use of BigX, an open source SDI like GeoNode would be more user friendly.
• The in-text references need a space before the bracket. There also seem some issues with the reference list.

[geekysquirrel]

Oh, I'm surprised! To clarify: the tests are run on Node, not Electron but they should all pass. In the linting stage there are some warnings related to the fact the tool is "normally" run in a browser, not on Node, but there are no errors and the unit tests all pass (not just on my machine locally but you can also see the test results from the Gitlab demo deployment here). Can you tell me which tests fail for you?

I have added some contributing guidelines to the readme.

I'm a bit at a loss as to how much I should go into detail regarding the advantages. Here are a few from the top of my head:

• The main one being QGIS Desktop is (as they name suggests) a Desktop application. You can browse your maps locally or export as image/PDF to share. QGIS online in its free version only supports public maps which may not be suitable for all data. To share private interactive maps you have to pay. The target users I am currently developing for a) don't have the funds to pay, b) would like to share interactive maps within a restricted group of users and c) cannot legally do that in public because of the personal data contained. Deploying a map to share using BigX requires nothing but a secure web space, the access details for which are only shared with a controlled group of users. No signing up for services provided by third parties, no data protection issues, no installation requirements such as databases or scripting languages as it runs entirely in the frontend on the end ("secondary") users' machines.
• To me, QGIS due to its sheer size is very complicated and requires much clicking and selecting and has got many different options and menus that are not necessarily intuitive and can be hard to find for somebody who hasn't used the software before. BigX on the other hand configures layers in one single JSON file, where they can be re-used for other map views. Yes, QGIS does export in a machine-readable, XML like format, but such files, even when stored in a in a CVS, are still hard for humans to read/parse as they don't just contain the "code" but also the "compiled" version, including generated/inferred/default/implicit fields and settings, resulting in a large, obscure file. BigX layers on the other hand contain only explicitly defined options and can easily be modified in a text editor, say in virtualised/containerised situations where there is no graphical OS.
• QGIS has got some quirks (features?) to help users, such as only showing geojson files if the filename extension is .geojson, but hides .json files which are still valid. I've also noticed other issues although I feel it's completely out of scope for me to write about that. I don't want to explain why QGIS is bad (it's not), but BigX is a totally different tool for a totally different circle of users (primary and secondary). Given that the paper is only two pages, do you think I should add these points there (any suggestions what I should remove to facilitate that?) or should I make them somewhere in the actual documentation?

I have added spaces before the references but I'm not sure I can see what the problem is with the reference list. Can you please clarify?

[geekysquirrel]

Oh sorry I forgot to reply to your point about online use. BigX is not supposed to be an integrated collaboration platform. Its purpose is to visualise data in different cases:

• "quick prototyping", where somebody just wants a quick way to combine datasets and explore the data visually on their local machine - all they have to do for that is to create a new layer and add the path to their data as explained in the manual.
• safe/protected presentation to "secondary" users, who are most likely not technical at all and know nothing about data analysis. These users are just presented with a preconfigured map they can explore but not edit. Deployment of these maps does not involve anything technical at all, the map "designers" can simply work on files in a network folder or similar where updates are reflected for users instantly. Code versioning or more sophisticated deployment options can be used optionally.

[liberostelios]

I've installed and used the software. It works fine on my laptop and I was able to play with some of the original data.

The software seems to work as intended. But, same as mentioned by @hugoledoux and @jvdkwast before, I am a bit confused about the motivation. I can see @geekysquirrel arguments about how it compares to QGIS online, but I think there is some relevant software with similar functionality (for instance, qgis2web does something similar). I would like to see a comparison with them.

Based on what I've read in the documentation, the paper and from @geekysquirrel's comments, I personally think that this app has a very niche, but clear use case: it's intended for people who are more comfortable with command-line applications and configuration files, instead of using GUI apps that require local installation. That, I guess, fits perfectly with someone who is working remotely on a server and wants to quickly view the resulting data of a process through their local browser, without having to download them from the server. I think that fits a lot with the modern way of doing things in data science, mostly, using containers and cloud services. Would you agree with that?

The only part I find a bit vague, about this process, is what happens with the data. You mention both in the documentation, as well your comments here, that the data remain in the browser. Does that mean that the data need to be in a location that is already accessible by the end user (e.g. and ftp location) or does the BigX take care of serving the data as well?

Some other minor suggestions:

• The installation instructions are missing an npm install step. I guess it would be useful to add this for some people who are not comfortable with how nodejs works.
• The example demo page is working, but the electron app is missing some data, so most of the example sources are not working. This isn't a problem, per se, but it puzzled me when I was running the initial examples.
• Regarding functionality, when a data source is not available you only get a small "forbidden" icon next to it to indicate that the loading failed. In my opinion, this is a bit confusing for the user. I would suggest that a more clear way (and maybe, also, an error indicating what's wrong) is shown to the user.

Btw, the tests ran fine for me.

[hugoledoux]

okay, sorry everyone for my silence the last week, I was overwhelmed with other things.

re: the novelty, I went back to the editors of JOSS. JOSS point of view is the software does not have to be something significantly new or novel, if it works fine and is useful for research (and all the checkboxes are clicked above!) then we can accept it. I found the first argument of @geekysquirrel about paying with QGIS a valid advantage, and what @liberostelios wrote about sysadmin working with servers also a clear and nice advantage. I suggest this is added to the paper? That being said, the point about qgis2web is also something to consider, and cite I'd say.

For the rest, I see that there are still unchecked boxes above. @jvdkwast : could you check if the tests run for you?

After the scope/paper is updated, I assume that the last checkboxes can be approved.

[geekysquirrel]

Thanks everybody for the feedback. I'm currently on annual leave but will update the paper as soon as I'm back.

[geekysquirrel]

Right I have made some updates to the paper and documentation that should hopefully resolve the remaining issues. Thanks @liberostelios for pointing out the electron issues, it absolutely makes sense to explain this and I have added some words to the manual to that effect.

I could have sworn there was a restriction on length that required papers to be under two pages but I can't find it anywhere now so I guess it's fine to be 3 pages as the extra references blew the paper up quite a bit but the word count for the paper (I assume without the references?) is under 1000 so it should be fine.

Also I'm not entirely sure whether I should replace all the links in the paper with references as some are not really related to my work and I only added the links to make it more usable. Should I leave it as it is, add references instead, or remove the links altogether?

[liberostelios]

@whedon generate pdf

[whedon]

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

[liberostelios]

Personally, I am happy with the improvements made by @geekysquirrel. The motivation is now more clear.

I think the work is very well done, the documentation is thorough and it should be accepted.

[hugoledoux]

@jvdkwast : could you please check if the tests run for you? And confirm that all is good on your side?

[hugoledoux]

@jvdkwast : it would be nice if you could give us a sign if you're still alive. If you don't do so this week I'll have to finish this review without you.

[jvdkwast]

Apologies for the late response. I switched off too many notifications. The updated article as well as the documentation takes into account well the comments. I recommend to accept this submission. I still couldn't run the tests successfully on my machine, but I think that it is because of my lack of knowledge to run these test tools.

[hugoledoux]

@whedon generate pdf

[whedon]

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

[hugoledoux]

@whedon check references

[whedon]

Reference check summary (note 'MISSING' DOIs are suggestions that need verification):

OK DOIs

- 10.5258/SOTON/P0014 is OK

MISSING DOIs

- None

INVALID DOIs

- None

[hugoledoux]

OK, both reviewers recommend acceptance so we're moving towards this.

I read the paper and I have a few comments, see PDF attached here. [Uploading 10.21105.joss.02537 2.pdf…]()

At this point could you:

• [x] Update the paper with my recommendations
• [ ] Make a tagged release of your software, and list the version tag of the archived version here.
• [x] Archive the reviewed software in Zenodo or a similar service (e.g., figshare, an institutional repository)
• [x] Check the archival deposit (e.g., in Zenodo) has the correct metadata. This includes the title (should match the paper title) and author list (make sure the list is correct and people who only made a small fix are not on it). You may also add the authors' ORCID.
• [x] Please list the DOI of the archived version here.

I can then move forward with accepting the submission.

[geekysquirrel]

Thanks @hugoledoux I'll get this done as soon as I can. The Link in your comment seems broken though, it's only sending me back to this issue. I can see the article proof but there don't seem to be any suggestions in it.

@jvdkwast I'd still like to know what exactly didn't work when you were trying to run the tests. I think that'd be a good thing to add to the docs, whatever it is.

[hugoledoux]

sorry, not sure what happened here, another try:

10.21105.joss.02537 3.pdf

[geekysquirrel]

Thanks, that worked.

[geekysquirrel]

Done. DOI is 10.5281/zenodo.4261963

[hugoledoux]

@whedon generate pdf

[whedon]

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

[hugoledoux]

All is good except the version, which should be according to semver.org (<v1.x.x>). I suggest you release 2.1.1? Or change the name of 2.1-joss to 2.1.1?

Then I can formally accept it.

[geekysquirrel]

I thought I shouldn't bump the patch version as I haven't actually made any changes to the code, just the paper, but I've updated it now as you requested.

[hugoledoux]

Cheers.

OK new DOI is 10.5281/zenodo.4272105

[hugoledoux]

@whedon set 10.5281/zenodo.4272105 as archive

[whedon]

OK. 10.5281/zenodo.4272105 is the archive.

[hugoledoux]

@whedon set 2.1.1. as version

[whedon]

OK. 2.1.1. is the version.

[hugoledoux]

@whedon accept

[whedon]

Attempting dry run of processing paper acceptance...

[whedon]

Reference check summary (note 'MISSING' DOIs are suggestions that need verification):

OK DOIs

- 10.5258/SOTON/P0014 is OK

MISSING DOIs

- None

INVALID DOIs

- None

[whedon]

:wave: @openjournals/joss-eics, this paper is ready to be accepted and published.

Check final proof :point_right: https://github.com/openjournals/joss-papers/pull/1914

If the paper PDF and Crossref deposit XML look good in https://github.com/openjournals/joss-papers/pull/1914, then you can now move forward with accepting the submission by compiling again with the flag deposit=true e.g.

@whedon accept deposit=true

[danielskatz]

👋 @geekysquirrel - In final prooreading (I'm the AEiC on duty this week) I've suggested two sets of changes to the paper and references in https://gitlab.com/geekysquirrel/bigx/-/merge_requests/9 and https://gitlab.com/geekysquirrel/bigx/-/merge_requests/8 Can you merge these or let me know what you disagree with in them?

[geekysquirrel]

Thanks for your changes Daniel, you spotted a few things nobody else did. I will pull in your Oxford commas and most of your grammar changes (except for data - it's been much debated but I think it should be singular).

I'd also prefer to leave the punctuation in the bullet points because they are full sentences (including capitalised first words) and some sentences are spread over more than one line so I think they should keep the full stops.

The triple dash was @hugoledoux 's suggestion - I have no strong opinion so I'll leave it as it is.

Let me know if any of this is a showstopper.

[hugoledoux]

The triple dash was @hugoledoux 's suggestion - I have no strong opinion so I'll leave it as it is.

In latex the em-dash is 3, 2 is for a dash between 2 numbers. I believe 3 is right here.

[danielskatz]

I think an endash is more correct here, but I'm not confident enough to argue about it

[danielskatz]

@whedon accept