Closed rafapereirabr closed 3 years ago
The comments from the journal are in. We've received really positive feedback from reviewer # 1, and a puzzling comment from reviewer # 2. See below.
Dear Rafael H. M. Pereira
We have a reached a decision on your submission ‘r5r: rapid realistic routing on multimodal transport networks with R5 in R’ to the journal Findings.
The paper requires significant changes to be made (see comments below) that better address the scientific substance of the research, but the reviewers believe the core analysis remediable. We look forward to seeing your revised manuscript within 1 month.
In addition, please upload
manuscript in .pdf format, manuscript source in .docx or .tex format (the files must be .docx or .tex format), and figures as high resolution .jpeg files (the files must be .jpeg) of any images. a letter describing changes in response to the reviewers. Also ensure all of the article Metadata is correct and complete on the electronic article submission forms.
Sincerely,
David Levinson On behalf of the Findings team.
Congratulations on both a superlative implementation of an R wrapper around the r5 package, and on an extremely well-formulated description of that package’s key functionality. The included code is entirely sufficient to reproduce all key results, and the extension to accessibility analysis critically important for understanding and contextualising the importance of this package and this accompanying manuscript. This is a very important piece of work that absoulutely deserves to be published!
Rating scale questions | Strongly Disagree | Disagree | Neutral | Agree | Strongly Agree |
---|---|---|---|---|---|
The paper makes a significant contribution to scholarship. | ✔ | ||||
The paper is professionally written, easy to read and free from grammatical or spelling errors. | ✔ | ||||
The paper asks a new research question or poses a new hypothesis. | ✔ | ||||
The data is appropriate to the research question and methods. | ✔ | ||||
The paper employs new data. | ✔ | ||||
The research methodology for the study is appropriate and applied properly. | ✔ | ||||
The paper uses a new research methodology. | ✔ | ||||
The paper presents new findings. | ✔ | ||||
The paper corroborates existing results. | ✔ | ||||
The paper refutes previous results. | ✔ |
The paper describes a routing package developed for R that uses Conveyal’s R5 routing algorithm to generate travel time matrices and travel itineraries. While the paper provides a good demonstration of using the package to compute travel time matrices, generate isochrones and apply this to computing access, it is lacking a research question. A potential research direction could be to expand on the package’s efficiency based on seamless parallel computing by comparing its performance (for a large dataset) with other similar tools like opentripplanner (also an R package).
Rating scale questions | Strongly Disagree | Disagree | Neutral | Agree | Strongly Agree |
---|---|---|---|---|---|
The paper makes a significant contribution to scholarship. | ✔ | ||||
The paper is professionally written, easy to read and free from grammatical or spelling errors. | ✔ | ||||
The paper asks a new research question or poses a new hypothesis. | ✔ | ||||
The data is appropriate to the research question and methods. | ✔ | ||||
The paper employs new data. | ✔ | ||||
The research methodology for the study is appropriate and applied properly. | ✔ | ||||
The paper uses a new research methodology. | ✔ | ||||
The paper presents new findings. | ✔ | ||||
The paper corroborates existing results. | ✔ | ||||
The paper refutes previous results. | ✔ |
Here is a gist with code to create a travel time matrix using {opentripplanner}
.
Some comments:
{opentripplanner}
requires the java
command to be bound to Java version 8. Check your Java version with java -version
. I could easily change from 11 to 8 with sudo update-alternatives --config java
in Ubuntu. In Windows you'll have to change the order of paths inside the PATH
environment variable so Java 8's path has higher priority than Java 11. Not sure how to do so with MacOS.clampInitialWait
to 0, but I don't think the parameter is working properly. I remember trying to use this configuration once without success before... Not sure if I'm messing up with something. But I don't think this parameter substantially impacts OTP performance, so I don't think that's a very big of a deal here.Hi all. Here is our response to the reviewers. Please feel free to suggest any changes. Tagging @mattwigway to join the conversation.
We appreciate the reviewer's suggestion. The aim of the paper is to introduce and demonstrate a new computational tool. As such, this manuscript is not intended to articulate a substantial research question, but rather to provide a tool/method that could be used to address a variety of questions that require the calculation of travel matrices or the examination of multimodal transport routes. We changed the last paragraph of the first section of the paper to make this point clearer in the text. The paragraph reads as follows (changes in bold):
Regarding the performance comparison between r5r and the opentripplanner R package. We have conducted a benchmark comparison and we found that r5r is 2054 times faster than opentripplanner (reproducible code can be found here https://gist.github.com/dhersz/14c4b7927a9ef91aeddcc3bf8bd0712a). However, we should note that opentripplanner focuses on point-to-point routing, returning all the spatial geometries of rotes. Because the outputs of the two packages are different, we believe this would not be a fair comparison and so we decided not to include it in the manuscript.
Sorry, just seeing this now - the reviews seem promising and I'm fine with the proposed changes to the manuscript. Just a few thoughts:
clampInitialWait
parameter in OTP should indeed be negligible.Regarding the performance comparison between r5r and the opentripplanner R package. We have conducted a benchmark comparison and we found that r5r is 2054 times faster than opentripplanner on the example dataset used in this paper (reproducible code can be found here https://gist.github.com/dhersz/14c4b7927a9ef91aeddcc3bf8bd0712a). However, we should note that opentripplanner focuses on point-to-point routing, so it runs the routing algorithm for each O-D pair. However, due to the nature of the graph search process used in routing, finding a route from an origin to a destination also finds routes to all places closer to the origin than the destination. R5 takes advantage of this to compute an entire row of the travel time matrix simultaneously, while opentripplanner runs the routing algorithm once per O-D pair. Thus, the computation time for r5r scales near-linearly with the number of origins, regardless of the number of destinations, while opentripplanner computation times scale with the product of the number of origins and destinations. Thus, the ratio of computation times will vary significantly depending on the dataset, but for any substantial travel time matrix r5r
will be significantly faster. Additionally, opentripplanner
provides far more details on each route than r5r
, including their spatial geometries. Because the computational approaches and outputs of the two packages are different, we believe this would not be a fair comparison and so we decided not to include it in the manuscript.
Not part of my proposed changes, but a caveat to the above:
I suppose to satisfy Reviewer 2 we could add something similar to what I wrote above to the manuscript when we talk about how this is different from what came before. We could add a paragraph comparing the computational approaches of r5r and opentripplanner/dodgr/stplanr. I can draft something.
Hi @mattwigway . Thanks for the detailed answer. I'm glad to incorporated the changes you suggested in our response to reviewer 2.
I would be glad to include a paragraph comparing the computational approaches of different packages. However, I don't know it would be possible to make an informative and yet short paragraph. We are already over the word-count limit. What do you think?
I drafted a paragraph in #151. I'm fine though if you don't want to include it, it is a bit of a complex point.
Ok, that's very succinct! Thanks. I think we should included it, for sure.
Great! I'm good with final submission at this point.
On Wed, Feb 24, 2021, 7:21 PM Rafael H M Pereira notifications@github.com wrote:
Ok, that's very succinct! Thanks. I think we should included it, for sure.
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/ipeaGIT/r5r/issues/108#issuecomment-785482433, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAEKNLU3V7MPVU6KZRVVASLTAWJXRANCNFSM4QT45RCA .
Paper resubmitted.
Awesome, thanks @rafapereirabr
Hi all. The editor has sent an email with the good news that our paper has been accepted!
Good news. Our paper is now published online. Thank you all for the contributions! I will be include the reference to this paper as the Citation
of our package.
Paper submitted to Transport Findings. See files in commit 11a6d2d5cd441c278dad9cc52a2ff46d522157d1