Robinlovelace
commented
6 years ago Now #178 is closed we can work on this - suggest we only fix the unambiguous and smaller issues and wait for further reviewers input before working on issues that need to be discussed and decided first.
reviewer one
Chapter 9:
graphicsandgridpackages. Ggplot2 has entire books dedicated to it. Both are referenced in the vis chapter. We think deliberately adding new content and documenting tmap beyond (the admitedly very good) documentation of the package itself is a further strong reason, beyond those outlined above. The approach is to link to existing resources and guide people to good packages as much as possible while minimizing duplication of material covered in existing books.nzis at this stage. This and some associated issues are addressed in https://github.com/Robinlovelace/geocompr/commit/888c7d9d6a1b431355ee29042b3e260790cd88cd making the section easier to understand for people who start on this chapter.Chapter 10:
Chapter 11:
funarguments is used. Secondly, it is not distinctly quicker - the extraction process is the most time consuming one. The only actual advantage (that I can see) of usingfunis it requires less code to write.Chapter 13:
[x] 1.21 [JM] Section 13.2 – To make the landslide dataset "balanced", in the given example the non- landslide points are sampled to give similar positive and negative sample sizes (175 vs. 175). Intuitively this seems wrong: the sample should either be completely random (e.g. sampling random points inside the study area, then checking them for landslides) or comprising the entire population (completely mapped study area). Why not divide the study area into a grid (i.e. raster), where samples are grid cells and have the response variable specify whether the cell contains (at least one) landslide, assuming complete mapping? In other words – it is unclear to me what is the rationale behind keeping "balance" between sample size of the two categories, at the expense of randomness. Few years ago I did a study on forest mortality and made the above-mentioned decision. The dataset was similar in structure: there are tree mortality locations scattered around the forest. Similarly to the landslides example, there were quite few mortality sites and a lot of non- mortality sites. However, I thought that instead of making the data balanced, the entire range of conditions in the forest needs to be characterized. So I ended up dividing the forest into a regular grid, and having each grid cell sampled as TRUE/FALSE, i.e. whether the grid cell had at least one mortality location inside. The data indeed were unbalanced, with just ~3% of cells being TRUE. (The data then went to a Mixed-Effects GLM considering spatial autocorrelation). Similarly, in presence/absence modelling of species distributions: I would think that entire valid dispersion range (e.g. a continent) needs to be considered as the "sample", with all cells given TRUE/FALSE values according to presence/absence of the species in that cell, rather than choose N observed locations and N absent locations. I believe that spatial logistic regression models (e.g. package 'spatstat') use a similar rationale: the spatial window is considered completely mapped, then the point pattern of observed events is modeled as function of background (e.g. raster) predictor also available for the entire area. Admittedly, I have no theoretical justification, it just intuitively seems more correct. If I'm missing something and the authors have any comments on this I will be very happy to learn. -Thank you for sharing your experience and the feedback! And I see, why our approach might be a bit confusing. Let me give you a bit more detail on the background. In the original study (https://doi.org/10.1016/j.geomorph.2011.10.029), we mapped all landslides in the study area. Secondly, we used the mid-point of the landslide scar as the response variable. Thirdly, we randomly sampled points which were located outside of the landslide polygons. We selected as many non-landlides points as there were landslide points since logistic regression using a log-link roughly expects a balanced response (see e.g., http://highstat.com/index.php/beginner-s-guide-to-glm-and-glmm). However, the same dataset has been also used in classes and summer schools to show how different modeling algorithms can handle the uneven distribution of the response variable and the related increase in spatial autocorrelation. This is why 1360 non-landslide points where randomly selected but again this was done by randomly selecting these outside of landslide polygons. To clear this up, we have added this explanation to a footnote. To answer your question on dataset completeness. We do not need the entire population but just a representative sample of it for statistical modeling. In fact, using the whole population would make modeling redundant because you already have the information for each pixel. But you are right that some spatial point pattern algorithms expect a full inventory, e.g. all trees (see the spatial point pattern analysis chapter of the ASDAR book.
[x] 1.22 [JM] Section 13.5.1 – In the first code section, the expression 'coords = lsl[, c("x", "y")]' is duplicated. -Changed! Thanks for spotting!
[x] 1.23 [JM] Could be helpful to comment on model interpretation in GLM vs. SVM. For example, how can we interpret the coefficients in GLM? How can we calculate variable importance in machine learning methods, such as SVM? -As emphasized in the introduction, this chapter is on spatial predictions using machine learning. Still, you are right that one can compute variable importance of ml algorithms but we better refrain from it here since otherwise we would have to add another section on the difference of statistical inference and variable importance in machine learning.
Additional questions:
[x] 1.24 [RL] The idea of an "Algorithms" chapter is great. From the chapter draft it seems the example (centroid) is relatively complex though. Also seems like part of the solution will be using an external package 'decido'. I would suggest replacing or supplementing this example with a simpler one, where the solution only requires functions covered elsewhere in the book, such as functions from the 'sf' package only. Just one of many possible suggestions – a function that accepts a polygonal layer and returns a new line layer containing separate line segments (as separate features) along with the azimuth where the exterior of each line segment is facing. This can be useful in urban planning, e.g. to understand how building facades are oriented. Such a function needs several steps: first splitting the polygons to line segments, then extracting each segment coordinates (x1, y1, x2, y2), and finally applying some trigonometry and math to find the azimuth. This does not require anything outside of 'sf' plus base R. Here is an illustration of a sample layer and the resulting azimuth classification - [FIGURE IN THE PDF]
- The chapter was at an early stage so we're very grateful for the high level comments and the specific suggestions. Based partly on these suggestions, the chapter has evolved substantially. We are confident it is much better and addresses each of the issues raised. See https://geocompr.robinlovelace.net/algorithms.html . Regarding the centroid example, we agree the description was a little long-winded. Based on this comment we've re-written the algorithms section. It no longer depends on decido (or purrr for that matter). It's shorter. And hopefully provides more boom for your buck! - Regarding the decision to use centroid, we defend that decision. It's something everyone can related to. It's now visually very clear what's going on with the new 3 facet plot - see https://geocompr.robinlovelace.net/algorithms.html#fig:polycent - that should be satisfying. More importantly it provides a reason to link to source code of industrial-grade implementations such as GEOS. That was always an intention: to make the reader aware that hard work has gone in to mean they don't have to write all their algs from scratch and that now comes through in the prose also hopefully. Any further feedback on this very much appreciated - we have a version that works even for non-convex polygons for example - worth mentioning? - We agree with your suggestion for a function to 'explode' linestrings representing building walls. We'll work on it.
reviewer two
Chapter 12:
Chapter 9:
[x] 2.2 [RL] As tmap is based on ggplot, I deem it necessary to introduce the underlying philosophy of ggplot first. - We now mention the Grammar of Graphics on which both tmap and ggplot are based - see https://github.com/Robinlovelace/geocompr/commit/7ec9456191e8f5b2701eaac18c4ed81683ed05f6 . I am not an expert in this field of visualization so any follow-on comments or suggestions welcome here (RL)
[x] 2.3 [JN] 9.2.2: There is no dataset nz_elev - The
nz_elevdataset is in the development version of the spDataLarge package.[x] 2.4 [JN] 9.2.3: There is no variable Land_area. The plot command therefore does not work. Replace with existing variable name LAND_AREA_SQ_KM. - This variable exists in the development version of the spData package.
[x] 2.5 [JN] There is no variable Population nor Median_income. As, in this case, there are no such variables in the data set, none of the code in 9.2.4 works. - This variable exists in the development version of the spData package.
[x] 2.6 [JN, RL] Section 9.2.5 has to be re-written to accommodate the way that tm_style works now. None of the +tm_style commands work. This includes the concluding pointer to the style_catalog function - We present how the tm_style function works starting from tmap version 2.0. It should be on CRAN soon.
[x] 2.7 [JN] In section 9.2.7, the call to the grid() library seems innocuous. However, the subsequent call of the viewport function depends on it. This is hard to understand for the uninitiated. The last paragraph in section 9.2.7 is meant to be a transition to section 9.3 but what is confusing is that the paragraph actually refers to all of 9.2 rather than just the last three pages. - We state that a viewport from the grid package will be used in the text. What else do you recommend? The last sentence of section 9.2.7 is meant to show readers that the examples in this section could be improved (e.g. with a better style) and the method presented can be applied not only to combine maps, but also to combine maps and plots.
[x] 2.8 [JN] Section 9.3 relies on an older version of tmap. The animation function is now called animation_tmap and assumes the installation of a software package external to R (ImageMagick). That is already problematic. With the change in the function call, I also get a "convert -version" error. - Actually, this section relied on the newer version of tmap. Some of the function names were standardized, including the change from
animation_tmaptotmap_animation.[x] 2.9 [RL] The world coffee code does not work either: the layer definition for facets requires a true/false value. Mapview, leaflet, and shinyApp, however, work just fine.
Chapter 10:
data("cycle_hire", package = "spData"); points = cycle_hire[1:25, ].Chapter 11:
Chapter 13:
reviewer three
Chapter 12:
Chapter 9:
[x] 3.2 [RL] This is a very good overview, it’s not only very practical in showing how to do things with these packages, it also gets across what a diverse and dynamic space the R spatial landscape is. I’m generally just reading it and learning, which is terrific! It’s a long chapter but for good reason, lots of strong ideas and good useable examples. I find this does lack a little bit of the big picture though, why is the space so fragmented, why are there so many mapping packages, and sooo many animal tracking packages? Is it because R is very flexible and dynamic and a lot of great ideas are happening fast, is it because it’s just how it is in geospatial? Are there prospects for tighter unity or should we expect more and more packages? How does the R community compare to the QGIS community for mapping? Why is QGIS and ArcGIS more about Python than R? A few thoughts on some of these questions might be useful.
[x] 3.3 [JN] Section 9.1 typo “are often be the best way”
[x] 3.4 [JN] Section 9.2 typo “and the grid providing” " is making static with tmap."
[x] 3.5 [JN] Section 9.2.3 typo “The purpose this section is to show how”
[x] 3.6 [JN] Section 9.2.3 typo " for to create superscript text)"
[x] 3.7 [JN] Section 9.2.4 typo “functions it in facts originates” - Thank you - https://github.com/Robinlovelace/geocompr/commit/5c4d03bf87bce93fec73b78975389906246001cf.
[x] 3.8 [RL] Section 9.2.6 Is “small multiples” defined in the lit somewhere? A ref? I see the term but it’s possibly an ESRI thing? I’ve tended to use “conditioned on” (a grouping value) in the past, but can’t remember where I picked that up - faceted is a very good term.
[x] 3.9 [RL, JN] Section 9.3, 9.4 This is excellent, very good description of the need for various modes of faceting and animation. Might mention the lack of an overarching framework for continuously varying data in R and animations, and while magick, animation, moveVis, and gganimate and others provide frame-based animation there is a longer-term effort to make continuous transitions more generally available (tweenr, new gganimate).
animation_tmap().[x] 3.10 [RL] Section 9.5 For absolutely up-to-date-ness it might be worth mentioning the new “async” facilities for shiny, and how that will help make shiny more acccessible and useable because of better responsiveness and scaleability.
Chapter 10:
Chapter 11:
Chapter 13:
reviewer four
General remark:
[x] 4.1 [RL, JN, JM] Minor general remarks: I notice that you use the "=" assignment throughout the book. You may be aware that within the R community the general consensus is to use the "<-" operator. Personally, I do not care, but for the vast majority of R users who are used to "<-", it might be confusing.
We all use
=in our own work and have decided to use it for ease of teaching. We have discussed changing it and would be open to doing so if a strong case can be made. It is certainly worth mentioning, however.Chapter 9:
[x] 4.2 [RL] Section 9.2 "... and the grid providing functions for low-level control of graphical outputs" Do you mean the "grid package"? If so, you may note that base-R graphics differs from grid-based graphics. Base-R graphics can be used for charts and maps (i.e. the plot function), but the grid graphics only offer graphical building blocks (like grid.rect and grid.text), which are not very useful for the end-user.
plot()offering dozens of arguments. Another low-level approach is the grid* package, which provides functions for low-level control of graphical outputs, --- see R Graphics* [@murrell_r_2016], especially Chapter 14."[x] 4.3 [RL] Section 9.2 Note that the vignettes will be reorganized in version 2.0 (and also renamed), so these links may not work after releasing tmap 2.0. You can also refer to "the vignettes listed in https://cran.r-project.org/web/packages/tmap"
[x] 4.4 [RL] (
? tmap-element--> ?'tmap-element' (Also watch the difference between ` and ', otherwise it won't work when copy-pasting)help("tmap-element")for a full list)."[x] 4.5 [RL] Small detail on "adding a border on top of the fill layer.". Normally, the order of layers represents the plotting order, but polygons are drawn only once. The different layers tm_fill and tm_borders are just to let users be able to specify borders and fill separately. In other words tm_borders() + tm_fill() gives the same result.
tm_borders()call aftertm_fill() +in the previous code chunk." - agreed but that is too subtle to be worth mentioning - the order of layers is the main point.[x] 4.6 [JN] 9.2.4 Last paragraph. Small note: in tmap 2.0, the viridis palettes can be specified directly in tmap, e.g. palette = "magma" - Thanks - https://github.com/Robinlovelace/geocompr/commit/512d1769914db0acdcee31757958d5f73824c8a3.
[x] 4.7 [JN, RL] Figure 9.13. In order to prevent occlusion, you could increase the figure a little, and try to add white borders. For that, you'll need tm_symbols rather than tm_dots, since it has a different shape (symbol 21 instead of 16). data(World) data(metro) qtm(World, projection = "longlat") + tm_shape(metro) + tm_symbols(col = "black", size = "pop2020", border.col = "white", scale = 3) - Thank you for this suggestion. We have implemented it.
[x] 4.8 [RL] As of version 2.0, using tm_view(basemaps = basemap) will be deprecated and replaced by tm_basemap(server = basemap). Users can specify the default basemaps with tmap_options.
[x] 4.9 [RL] Third paragraph: shiny offers a solution to the first and third limitation, but not the second one (scalability of large datasets).
[x] 4.10 [RL] Inset paragraph on shiny: note that copy-pasting runApp(“coffeeApp”) will not work, since the double quotes are different than ".
block2.Chapter 10:
find_algorithms()for the first time, we also write that when using it without specifying anything, it will return all available QGIS geoalgorithms including a short description. Still, we have added a link to the online documentation of all QGIS geoalgorithms since this provides a better arranged overview as you correctly point out. Thanks!Chapter 11:
Chapter 13:
resamplingtoperf_levelin the GLM section as well and say in the SVM section that the resampling is identical to the one used in the GLM section.tune_levelobject in the code chunk below; see Figure 11.6) for a visual representation). To find the optimal hyperparameter combination we here fit 50 models (ctrlobject in the code chunk below) in each of these subfolds with randomly selected values for the hyperparameters C and Sigma. The random selection of values C and Sigma is additionally restricted to a predefined tuning space (psobject). The range of the tuning space was chosen with values recommended in the literature (Schratz et al., 2018). To make the performance estimation processing chain even clearer, let us write down the commands given to the computer: 1 Performance level (upper left part of Figure 11.6): split the dataset into five spatially disjoint (outer) subfolds. 2 Tuning level (lower left part of Figure 11.6): For each of these folds, run the hyperparameter tuning, i.e. spatially split the performance fold again into five (inner) subfolds. Use the 50 randomly selected hyperparameters in each of these inner subfolds, i.e. fit 250 models. 3 Performance estimation: Use the best hyperparameter combination from the previous step (tuning level) in the performance level to estimate the performance (AUROC). 4 Do all of the steps described above for the remaining four outer folds. 5 Repeat a 100 times all the steps from above.Chapter 12:
Additional comments
Chapter 12:
mapview::mapview(desire_carshort$geom_car)Chapter 13:
mapply()which we replaced byforin accordance with a reviewer’s comment. Now the corresponding text is also adjusted accordingly, thanks for noting!Chapter 5:
Misc:
[x] 5.20 [RL, JN, JM] (...) That said, the current division is perfectly fine in my view; perhaps its rationale can be given more emphasis in the text as follows - • Chapter 4 deals with operations where the spatial location is being used, but is not being modified with any spatial algorithms as part of the operations – just reordered, filtered, merged, rearranged, etc. For vector layers this includes spatial subsetting, checking spatial relations, spatial joins, aggregation and calculating distances. One thing to consider – aggregation does modify the geometry, though only by dissolving borders or combining single-geometries to multi-geometries, so perhaps it can be moved to Chapter 5 and combined with the geometry unions into Section 5.2.6. For raster data, this chapter includes subsetting, masking, map algebra, local, focal and global operators and merging. • Chapter 5 deals with operations where the spatial location is being processed in complex ways that are beyond simple rearrangement. With vector layers this includes simplification, calculating centroids, calculating buffers, shifting, clipping and unions, and casting to other types. With raster data this includes intersection, changing the extent and origin, aggregation and disaggregation. Second thing to consider – the "intersection" part is perhaps more fitting to Chapter 4, along with Section 4.3.1, as it is another example of subsetting, without modifying the spatial arrangement of pixels, analogous to Section 4.2.1 for vector layers. The raster-vector interaction section (5.4) is a good fit here because all of the presented operators transform the geometry of the raster or vector inputs. • Chapter 6 deals with reprojection, which can be described as a specific type of geometric operation where the data are transferred "as is" into a different CRS. General comments -
We have made the structure of the book clearer and updated chapter names partly in response to these suggestions. Any further comments welcome