creating maps for location, home range

[xhdong-umd]

I'm working on maps for animal locations, home range/occurrence polygons.

@chfleming , the SpatialPolygonsDataFrame.UD returned the polygon coordinates as relative x,y values (Two-Point Equidistant projection?), not the absolute coordinates. There are reference coordinates in standard parallels of proj4string, so supposedly this should work, indeed the shapefiles(which have same structure) can be plotted in right location.

However I'm having trouble plotting SpatialPolygonsDataFrame in right location.

Should I reproject the data?

[xhdong-umd]

OK, I found I can reproject the data with spt_p <- spTransform(spt, CRS("+proj=longlat +datum=WGS84")). Though I need to make sure this is correct usage since these projections, datums, coordinate systems are quite complex.

[chfleming]

That looks right for long-lat. Generally, you want to get things into the same projection as the background map/raster. I have never found a way in R to plot rasters in a non-native projection, or even just to rotate a raster.

[xhdong-umd]

@chfleming @jmcalabrese I tried two datasets with Carto.com for various options. You can click each link to view them.

I think in general the Carto maps have better performance, and the Here satellite base map is also better.

• buffalo, I didn't take time to put every individual as separate layer yet.
• gulls with animation layer
  • by default the animation is shown. You need to wait for some time before it start. You can pause the animation, select a range in the time bar. This animation is based on the first individual.
  • You can switch to the point layer. The zoom in/out on this 50k points map is still quite smooth.
• gulls with heatmap layer, all individuals are not separated here. This heatmap is much more consistent when zoomed in/out compare to the leaflet heatmap. With leaflet the shape changed a lot when you zoom in/out.

It's still possible to use Carto maps in app, with these limitations:

• we need to register for an account. They have free account with 350M storage for non-commercial use, but all the data your uploaded are public, i.e. no privacy here.
  • The app can also provide a box for user to input his/hers API key if user registered an account. This can make sure the map generated will be kept in their account.
• I made the maps in the website UI. It's possible to recreate every step in code because they have this map API, but I'm not sure if every step can be automated.
• The total turn over time for a map to be ready will definitely longer than leaflet, though it should be still manageable.

[xhdong-umd]

I tried to add labels to home range polygons, but only the biggest range label is shown, i.e. all 3 ranges are showing same label. Simulated polygon data don't have this problem, and I didn't find any meaningful difference. I checked the hole status, row names of data slot etc.

I'll just skip the label for now, since it's relative straightforward based on context.

[chfleming]

I will look into it. I just finished fixing some contour labels/bugs and may have done something unintended.

[xhdong-umd]

@chfleming You don't need to spend extra time on this. This is probably not problem in ctmm side. I didn't update to the latest ctmm before running the code.

I can try to generate a minimal example to reproduce the problem, and submit bug report to leaflet if it's a bug.

[xhdong-umd]

I have submitted a issue to leaflet.

If there are some intersections between two individuals home range, what label should we show when mouse hover to that area? I think there is no perfect solution for this case.

There is a option to highlight polygons with mouse hovering, if we can highlight all polygons under mouse that should serve the purpose. However that option only highlight one polygon. Leaflet didn't consider these overlapping polygons case everywhere.

[xhdong-umd]

I'm not sure if we can plot the occurrence on the map. The occurrence data have a complex structure and I didn't find a method to draw them with leaflet.

[xhdong-umd]

We can work around the polygon label problem if we change the polygons order as bigger first, smaller second. However I think the label will not be perfect in general with lots of individuals overlapping. I'm inclined to just skip the label. User should have no problem to recognize which polygon is which without label.

[xhdong-umd]

I have updated the app with points map

• I registered a free account with Here maps so we can use the better satellite map. I think the free plan should be enough for our usage.
• There is no label for home range polygons, because I felt the overlapping and intersections of different polygons make it rather difficult to do it properly.
• At first I drew the home range polygons with same color of individual. Then I realized the home range list came from current selected models, which could be different from the current selected individuals. The models could only cover a subset of individuals, or even have duplicated individuals with different models. Right now the home range color is a little bit darker than individual colors when they have 1:1 match, but there is no guarantee that home range color will be similar to individual color.

[chfleming]

When the colors are the same, I don't think extra labeling is necessary. Also, I think choosing one movement model will be, by far, the largest use case. So I would tune the user experience more for that purpose.

[xhdong-umd]

I can make the home range color same with the individual color, but two models for one individual will same color, is this OK?

[chfleming]

Probably not. Is it not possible to do something like the following: col=rainbow(n) for n unique models. Then partition col by individual animal and average each individual animal's partition of colors into one mean color for their data.

[xhdong-umd]

I have been using same color palette for animals through plots and summary tables. I think it's better to keep the animal location color consistent with other pages.

Maybe for multiple models of same individual, we can make each model color to be a small variation of brightness of location color. If there is only one model for one individual, it should be same color.

I'll try this and see how it works.

[chfleming]

For n individuals with colors col=rainbow(n), if an individual has m models, then you could make a dummy array of colors rainbow(n*m) and partition that dummy array into n dummy individuals with m model colors each. Then pull out the one individual's m length partition of model colors. Repeat for all individuals. This will keep the individual animal colors evenly spaced among all animals and the model colors evenly spaced within individuals.

[xhdong-umd]

The individual animal colors are used in about 10 plots and 2 tables. I think it's better to keep them consistent.

[chfleming]

Yes, that's what I am saying.

[jmcalabrese]

@xhdong-umd mapping doesn't work on my Linux system when I install the webapp with your script:

source("https://raw.githubusercontent.com/ctmm-initiative/ctmm-webapp/master/run.R")

I get: Error: could not find function "spTransform"

I thought the problem was that your installation script for the webapp was missing a dependency for the sp package. However, even after installing sp manually and restarting, I still get the same error.

[xhdong-umd]

Yes it's a dependency problem. sp has spTransform but it was actually came from rgdal. Though rgdal should be installed by ctmm. I will test again.

[xhdong-umd]

@jmcalabrese It should work now. I added sp and rgdal and tested with linux VM.

[xhdong-umd]

This is the color palette for 6 individuals (the plot usually have them with some transparency)

I call these color as base color. Now I need to create some variations from base color but still resemble the base color, so I'm not going to work in rgb space directly. Two alternatives are hcl and hsv.

hcl values don't always have [0,1] range depend on value combination, so it's not safe to interpolate in hcl space. I used hsv space instead.

The hsv value of first base color is as follows

h 0.01079137
s 0.56048387
v 0.97254902

I think it's best to only change v. v < 0.5 will be too dark, so I vary v of the base color in [0.5, 1.0] range depend on how many variations I need for one individual.

This is 6 variations from first base color plotted, the first box is the original base color.

One limit is the original base color have a v value of 0.97 so the variations will not be equal distance from the base color, like the case of 2 variations

I didn't find a simple way to overcome this. Though I think it's OK as long as the variations are separable but similar colors, since the original color is used in points, and the variations are used in home range polygons.

[xhdong-umd]

Here are the base colors of 13 models, duplicated base color means different models for same individual.

And these are variation colors for these models. Each duplicated color get some variations.

[xhdong-umd]

I have updated the repo app with new color theme.

I decided not to add legend for home range colors, since it could be too busy and difficult to separate for subtle differences. It's easier to find which is which by switching the layers on and off.

[xhdong-umd]

I have added heatmap and clustered map to repo.

• I don't think the clustered map have much advantages compare to heatmap. It didn't show the distribution as well as heatmap. I set the map to show points under lower zoom level, but the color can be confusing: the cluster circle color is determined by cluster size, bigger numbers are red etc, and the points color are individual color as the legend shown.

  I'm only including it for you to have a look. I'm inclined to remove it in app later.

• Both the heatmap and clustered map cannot separate each individual with color.
• Previously we talked about that switching heatmap to points at lower zoom level. I'll try this and the syncing the bounding box and zoom level in next step.

[xhdong-umd]

For the syncing between heatmap and points map, my plan is

• If the data set is big, open heatmap by default when user moved to map page, so the map will render quickly and show a good overview. After this, user can still switch to points map if he/she want.
• Previously I was thinking to always sync two map's bounds and zoom, but now I think it only make sense to use heatmap for overview, zoom in then switch to points map. There doesn't seem to be a need to use points map bounds for heatmap. Two way sync is doable but I didn't see the need for now.

[chfleming]

I don't know if you have done it, but I would make the lowest AICc model the same color as the data and trend the variation such that the highest AICc model is the most dissimilar color.

[xhdong-umd]

The model color is only used in home range polygon color, not in the model summary table. In the map it's very difficult to separate the subtle difference because

• the colors have some transparency
• different models for same individual have very similar polygons thus overlap each other
• the satellite background is strong

There are 4 models for Toni here but only 2 set polygons visible, with 2 models overlapped on each. So you are seeing the combined color instead of actual single color of polygons.

I switched off two models to show 2 set of polygons without overlapping. Now you can see the real color of each model.

Plus you said it's not a usual case to display multiple models of same individual in map, do we really expect user to catch the subtle color difference and map it to value? I think it will be quite challenging.

It will make more sense to use that color theme in the model summary table, however we are already using color grades in the 3 rows of one model, assign each model as a different shade of same color will make all the rows look very confusing.

[NoonanM]

I would suggest have a different line style for the 95% CIs on the contours.

[xhdong-umd]

The default contours have labels 95% high, 95% ML, 95% low. Which one do you want to use different line style? The 95% ML one? I think that one is always the middle sized polygon, right?

[chfleming]

The middle, point estimate should stand out the most, while the high and low estimates should be less emphasized. In ctmm's plot function I make them thinner and less opaque.

[xhdong-umd]

Yes I saw that. Do you want similar treatment, i.e. high and low thinner and less opaque? I think make it consistent with home range plot is a good idea.

This should apply to all levels, not just 95%, right? For example 95, 20, each should have the middle stand out.

[NoonanM]

Yes, in all cases the middle (ML) contour should stand out.

[xhdong-umd]

The satellite background can be difficult for home range polygons

These are the 95%, 20% contour in terrain background

I updated the repo app and you can have a test.

[xhdong-umd]

It seemed that Shiny doesn't support switching tabbed box in program. I can make tabbed panel like this and have the switching enabled, but it's not very good looking / consistent with other parts.

Another option is just put heatmap in first tab as default, so the map should always render quickly and give an overview. Then user can decide whether to switch to points map.

Which option do you prefer?

[xhdong-umd]

@chfleming Previously I was hesitating on that color theme because the v value could be 0.64 ~ 1, so variation color could be brighter or darker. Thus if we assign the best model with base color, the other models may have color brighter or darker, it will be difficult to identify which one is same with base color in a series of similar shades.

How about we always use the brightest color for best model? The model color may not be same with base color, but brighter color models are always better than darker color models. I think it's much easier to identify which one is brighter, compare to which one is same with base color.

[chfleming]

That sounds fine to me.

[xhdong-umd]

@chfleming My model summary table was built on individual summary on each model so there is no AICc information. Though I think I can assume the models are always sorted with AICc in the model list, right?

On the other hand, should we include the AICc column in the model summary table?

[chfleming]

Yes and yes.

[jmcalabrese]

@xhdong-umd install from the script works now on my linux system. Not sure what the problem was, though. I definitely had rgdal installed. Thanks for fixing that.

[jmcalabrese]

@xhdong-umd I agree with the idea to remove the cluster map and instead focus on the point and heat maps. I've never liked the cluster map and agree that it is too busy and doesn't give a good sense of an individual's full distribution when zoomed out.

[jmcalabrese]

I agree with @NoonanM and @chfleming about the middle contour for the home ranges standing out more than the CI's.

[jmcalabrese]

@xhdong-umd I like the terrain background and agree that it is much easier to see what is going on relative to the satellite background. I would recommend making the terrain background the default with satellite background as an optional alternative.

[jmcalabrese]

@xhdong-umd I'm not clear on how your suggestion to have the heat map as the default tab would work. I the user zoomed to a preferred view with the heat map, would switching to the point map then preserve that zoom level?

And what about the case of smaller datasets with the performance of the point map would still be fine. Would it make sense then to always default to the heat map?

[xhdong-umd]

At first I couldn't make the switching heatmap tab work so I was thinking make it default. Now I've make it work so this suggestion can be ignored.

[xhdong-umd]

I finished on these changes:

• removed the clustered map
• move terrain map to default
• heatmap will be selected by default if current selected data set > 25,000 rows
• dAICc column was added to model summary table

Next I will make the home range color brightness map to the dAICc value, and sync the heatmap and points map bounds/zoom.

[NoonanM]

Maps currently look very good. I had a few thoughts that might appeal to users

• Is it possible to add a feature that will export the current plot?

• Is it possible to annotate the map with axes and/or a scale bar? (Axes are probably more useful when exporting the plot)

• Is it possible to add a feature for changing the width of the x-axis to restrict the window to an area closer to the data/contours? (Again, probably more useful when exporting the plot)

[xhdong-umd]

It took me several hours to make the home range colors right.

• I need to create the color palette on the full model summary table, which requires some jumping around some deeply tangled code converting model summary into data frames. In the end only several lines were added, but it was very time consuming to add them right.
• There is a overlooked parameter that didn't keep character vector order when converting them into factors. It's very difficult to find this cause.

I have updated the repo and now the home range with smaller dAICc are brighter.

@NoonanM

• Which plot do you mean? The map? It's on my list, user should be able to save the map as an html file.
• I'll look at the scale bar. But I'm not sure about axes since I seldom saw axes in maps. Do you have an example that show what you want?
• I think the current default initial view is good that included a little context around the data. You can resize the app window to change map width, adjust map height etc. I'll add a button later to recenter the map back to initial view so you can use it after some pan and zooming.

[NoonanM]

My thinking is that these plots are generally high quality, and informative. Many users might want to use the app to produce figures for publication (e.g., something like that attached figure). Being able to adjust the axes, and export the map as e.g. a png or pdf would enable some of that functionality.

[xhdong-umd]

I did some simple search and didn't find support of axes in leaflet. The axes are often seen in the static map plot, but leaflet as a interactive live map seldom have this need from user.

There are ggmap which can generate static maps like the example above, but we didn't go that route because an interactive map should be much more useful.

User should be able to save the map itself and share it, which can be much more useful than a static plot. However I'm not aware of an option to export plot like above.

~Actually user will need to make a screenshot by himself/herself if a picture is needed. There is a leaflet.print plugin to save image from it, but that need another dependency of complex print modules, so I think we will need to rely on screenshot.~ There seemed to be a tool to save screenshot, I need to test it later. Update: it was very slow and require more dependencies, I think we will just let user to take screenshot.

[xhdong-umd]

Scale bar is added, also the repo and hosted app are updated.