Overlap

[NoonanM]

Are there any plans to implement the overlap() function? My guess is a lot of users would want to use this function for subsequent analyses and it would be good to provide the option within the app.

[xhdong-umd]

@chfleming @jmcalabrese What's the work flow you have in mind?

I saw overlap can be applied to A list of ctmm fit, telemetry, or aligned UD objects. So it can be

• telemetry overlap for current selected individuals
• overlap for fitted models
• aligned home range
• aligned occurrence

There could be a button in each page, or one button to generate all available overlaps for current selected individuals/models/home ranges etc.

The result will be just some tables. How do you want to use the result for subsequent analyses?

[chfleming]

You want the UD overlap only for now, after home range-analysis.

If you calculate all of the UDs on the same grid by feeding them all into akde() at once, then you can more quickly calculate UD overlap from the existing UD objects (+ fit object argument).

[NoonanM]

Most appropriate would probably be an 'Overlap' tab since it's a tangential analysis that people don't have to do, but many will probably want to. But I leave that choice to @chfleming and @jmcalabrese.

My guess is users wanting to estimate overlap will generally use all individuals included in the Home Range tab.

Output is three matrices. I've found visualizing them via the attached ggplot script to be a useful way of quickly seeing what's happening. BC is an object output from overlap()

`dis <- as.matrix(BC[,,2]) #This one plots the point estimate [,,1] and [,,3] are the CIs

dis[upper.tri(dis, diag = FALSE)] <- NA

dis <- data.table::melt(dis, na.rm = TRUE) dis$value <- log(dis$value) #Usually have to log scale to make the colors stand out more

ggplot(dis, aes(Var1,Var2, fill=value)) + geom_tile(color = "black") + theme_bw()+ theme(panel.grid.minor = element_blank(), panel.border = element_blank(), axis.text.y = element_text(size=6, hjust=1, vjust = 0, family = "serif"), axis.title.y = element_blank(), axis.ticks.y = element_blank(), axis.title.x = element_blank(), axis.text.x = element_text(angle=90, size=6, hjust=1, vjust = 0, family = "serif"), legend.background = element_blank(), legend.text = element_text(size=4), legend.key.size = unit(0.4, "cm"), legend.title=element_blank()) + viridis::scale_fill_viridis(na.value = '#440154FF')`

Distances.pdf

If you could have a set up where the user can click on a tile that looks interesting, and they would see the two (or more?) UDs plotted together my guess is many users would find that useful.

[xhdong-umd]

I'm working on maps now, probably will revisit this after maps are finished.

[NoonanM]

@xhdong-umd That's fine. We are currently working on the paper that will accompany the overlap() function. Ideally we would have the two come out around the same time (app functionality and paper), but the paper is still a few months away from publication so there is plenty of time.

[xhdong-umd]

@NoonanM @chfleming I haven't read the full paper yet. Based on the help of overlap and discussions above

• we can have overlap for gaussian distribution of models, or home ranges.

• we can use the selected models in the model summary table, and the selected home ranges in home range table as the basis of overlap. By default it will select the best model for each animal. The home range table have no selection now but it can be added.

  This way user also have the flexibility to select any combination.

• In the model selection page
  • If we add a tab in model summary table box, user will not be able to see selected models and overlap result at the same time.
  • If we add a tab in variogram box, I felt it's kind of weird as the overlap is only available after models are generated and selected, thus should not be in same "level" of variogram. Though this is still an option.
  • We can also add an individual box of Overlap(we can also use a pop up window, but again, an individual box can keep the result better). And I can make the content in the box respond to the selections in model summary table dynamically. The performance will not be a problem since the overlap calculation for gaussian distribution is very fast. The negative of this option is user need to scroll a long page.
• For the home range page, we can either add an overlap box or add a overlap tab to home range plots. The overlap of 3 home ranges of buffalo took 15 seconds in my machine, so it's better to make it an option only activated by user.
• Thus the better option seemed to be adding a tab in variogram box and home range plot box. Once user clicked that tab, overlap of current selected models/home ranges will be calculated.
• Because the overlap of home range take longer time to calculate, maybe we should not select all home ranges by default? User can select some home ranges and click tab to check overlap. There can be a button to select all, just we don't put it as default.

For the representation of overlap results

• I think a table of numerical values is always needed. I can add a table, also save it in the work report.

• We definitely need some plot to visualize the result. The color scale doesn't seem to be very intuitive as a guide of 0-1 values to me. Maybe a grayscale or alpha scale of same color is better. And we may also only draw the triangle half of the matrix to reduce duplications.

  Or something like this

I'll also explore some variations and compare the results.

• one option is pie chart, we can put a pie chart in each grid cell. I'm not sure it's doable with ggplot though.
• another option is a bar with two segments to show the overlap percentage.
• we can also just draw a scatter plot of x: animal a, animal b, animal c, y: 0 ~ 1, color (shape) of point: animal a, animal b, animal c.

[NoonanM]

When plotting the matrix, in addition to the concentric circle style that @chfleming suggested, we can also make use of the two halves of the matrix to include more information as in this example:

(This uses the corrplot.mixed() function from the corrplot package)

[chfleming]

Here's a mock-up of what I was thinking, where the inner most circle radius would be the lower confidence interval, the middle circle radius would be the point estimate, and the outer circle radius would be the upper confidence interval.

[xhdong-umd]

I'm thinking to use color background in the numeric table, which can serve like the overview plot. And use 1 dimension plot instead of a 2-d grid, because the grid gives a pattern which does not exist.

I'll try a arranged bar chart and wrapped facet. 1-d plot is also easy to compare values.

[xhdong-umd]

This is the overview table. I plan to remove the duplicate half, use color background scaled to the value in the app.

And a plot with error bar for all unique combinations

point range version

[xhdong-umd]

I made these changes to the table, try to make it easier to read:

• the low/high rows can be hidden with a checkbox
• the duplicate half can be removed with a checkbox
• the low/high rows have a grayscale background, just like the model summary table
• the value color is grayscale by the value.

Currently I'm using the animal name - model type because they are home ranges and are model specific. This make the names quite long but I can't find better alternatives.

[xhdong-umd]

@jmcalabrese @chfleming @NoonanM The basic parts of the overlap page are ready now.

The different views of the summary table

The plot of the value range for each combination

• the value label can be removed if needed
• I rotated the plot so more combinations will increase plot height instead of width, which should be easier to handle
• Note the y axis start from bottom, so the first values of the table appeared in bottom of the plot.

The location tab will show the animal locations plot.

I'm trying to add interactivity among the table and plots for now:

• clicking cells in the table will highlight the corresponding part in the value range plot, and the animal locations (draw other animals in grey background)
• clicking the value range plot should highlight the cells in the table

[xhdong-umd]

I found this version of summary table is better than the 2d matrix table above:

• there is no wasted space. You don't need to hide CI columns or remove the duplicate half
• It's easy to sort by animal name or value

I updated the app and included both versions of summary table so you can play with it and compare them.

[xhdong-umd]

@jmcalabrese @chfleming If we simplify the model name to just animal name when there is no conflict, should we also change the variogram/home range plot title from the animal - model type full name to just animal name?

[xhdong-umd]

I found both the overlap result matrix names and the home range plot titles come from the names of the home range list, they need to be consistent.

I have made them to be animal names only when there is no duplicates, and change to full names when there are multiple models from same animal.

In the other hand, the modeled variogram plot title is still full names with model part now. It can be changed, but I'm wondering maybe it make sense to include model name in this plot?

[xhdong-umd]

I also need to change the map part, because there are layers of animal locations and home ranges. Previously the location layer name is animal name, home range layer name is id - model type full name.

Now there will be conflicts if we just use animal name for home ranges when there is no duplicate.

Option 1: change the home range layer name to

• something like Cilla - range or Gabs - home range format when there is no duplicates.
• Cilla - OU anisotropic format when there is duplicates.

Option 2 is still use the previous setup in maps:

• animal name for location layer
• Cilla - OU anisotropic pattern for home range layer no matter what.

[xhdong-umd]

I took option 1 as it is easier to read in simple cases.

[xhdong-umd]

Sorting table and plot at the same time is also done.

When you input a word to filter the table, the plot also updates.

[xhdong-umd]

Previously I added a location plot tab in overlap page. Now I think maybe it's better to just use the map tab for this purpose.

The only benefit from a separate location plot tab is that we can also make the location plot respond to selected overlaps thus highlight them. Though the map page also have layer selectors to check a subset only.

Should we just remove this tab?

[chfleming]

I think what most of us want on the location tab is that after selecting one row from the table, the location tab will show a plot of the two animal's locations and home ranges.

If multiple rows are selected, I suppose you could show a grid of plots.

[xhdong-umd]

Previously I planned to add some interactivity between the table and the value range plot:

• click on table to highlight part of plot
• click plot to highlight table

Now with the sorting and filtering working, I think this is not really needed anymore. It's very easy to map from table to plot.

Sorted

Filtering

[xhdong-umd]

I agree a group of plots of overlap pairs will be most helpful. Though we will need 15 plots for 6 animals, (n**2 - n)/2 for more animals.

Is this acceptable?

[chfleming]

I would only plot the selected pairs.

[xhdong-umd]

And my previous plan of two direction interactivity can cause a loop:

• selecting rows in table highlight plot parts
• selection plot parts will select rows in table, which could cause update again because row selection in table itself trigger changes in plot.

It should still doable, maybe with some additional refreshes.

It'll be much simpler if we only do one direction:

• selecting rows in table highlight plot parts
• the location plot also update for selected rows.

Is this enough?

[xhdong-umd]

For the variogram plot of models, should we use animal name only or the animal - model type as plot title?

[chfleming]

I don't think you would want to have one big plot, but zoomed in plots of the pairs. In many cases, individuals will be very spread out.

[chfleming]

I would only include the model name in cases where models are being compared.

[xhdong-umd]

I agree. So we will keep the location plot tab

• draw overview plot by default (we need to have something in the beginning when nothing is selected)
• with rows in table selected, switch to group plot of pairs

[xhdong-umd]

I'm referring to this plot:

It can be changed to similar arrangement of home range plot, i.e.

• use animal name only when there is no multiple models from same animal
• use animal name - model type name when there is multiple models.

But I feel the current arrangement is somewhat useful to always show the model detail in this place.

[chfleming]

I think that's probably fine as is.

[xhdong-umd]

OK. I will work on the paired location plots.

Is one direction interactivity from table to plot enough? i.e. no "clicking on plot to highlight rows in table".

[chfleming]

I don't have a preference for the reverse direction. @jmcalabrese @NoonanM

[xhdong-umd]

Selecting table rows will change color of corresponding values in plot.

It's difficult to have more specific non-default color mappings in ggplot.

I wanted to just keep non-selected values in blue point and red error bar, and change selected ones to other color. That means two separate color palette for point and error bar, which is quite difficult.

Hopefully the current state is good enough.

[xhdong-umd]

For the overlap summary table, do you want to sort by ML value descendingly by default?

[chfleming]

That makes sense to me.

[xhdong-umd]

I have finished the grouped location plot of overlap pairs, made the summary table sorted by default.

There is a brief error message in certain cases #48, I have to leave it as is now since I have spent several hours on it without success.

Previously we talked about "jump to home range page if user clicked overlap page directly". That turned out to be difficult because it's hard to determine whether the home range is available. It could be updating and not NULL. The only way to do that seemed to be maintain a page click history and check the history, then there could be multiple corner cases of different click combinations.

I think most user will not jump to overlap directly like what I did, so I'm leaving this as is now too.

Other than this two points, all discussed features above have been implemented. I have updated the app and you can try it out.

[xhdong-umd]

I managed to silent the warning and error messages for the special case. That should be enough because it only happened briefly then update to correct data automatically.

[xhdong-umd]

I have finished the help page for overlap, and is looking at the parallel functions for integration into ctmm now.

[xhdong-umd]

I'm experimenting with home range plots.

Here is location points and home range contours

plot(tele_list_sub3, UD = hrange_list_sub3, col = c("red", "blue"), labels = names(tele_list_sub3))

Without location points

plot(hrange_list_sub3, col.level = c("red", "blue"), labels = names(tele_list_sub3))

I'm not familiar with base plot, and I'm not sure how to customize these issues:

1. The home range contour label is a little bit small and not obvious. A bigger font or bold should be better.

Here is another plot (there are 3 animals in plot while we only need plot of pairs, but I want to use this plot to show that when two animals are far away and stretched plot to bigger scale, it's even more difficult to read the labels.

2. Maybe we should only draw one contour for each home range by default? Of course we can add an input to customize levels, just like the other pages.

3. Should we remove the grid and density function to make the overlapped pair easy to read?

[NoonanM]

I would remove the grid for sure. I don't think it adds any additional information at this stage. I think it would also be good to change the color of the UDs for each individual.

I would also consider removing the labels on the confidence interval contours.

[xhdong-umd]

By color of the UDs do you mean the blue density in plot? I'm not sure if mixed colors in overlapped area will work well.

If we removed the labels on contours, we will need a color legend or some other label to show which is which.

[NoonanM]

Yes that's what I meant. I personally find the different colours more informative. The information on the overlap estimate comes from the density estimate, so it's good to see where the high/low density regions for each animal are.

[xhdong-umd]

OK, actually we can make them all as options. By default all will be shown, user can toggle off some elements if they are too busy for a specific plot.

• location points - maybe turn this off by default?
• density
• contours

[NoonanM]

Yes, that makes sense. I think locations and labels off by default is the cleanest baseline option?

[xhdong-umd]

If we decide to use a color legend in case label is off, then maybe we can disable label all together, since the current label font is too small/fade anyway.

[NoonanM]

I think the legend option is a better idea.

[xhdong-umd]

I think we can use home range plot as default and put the current ggplot2 points plot into another tab. It could be a useful option, especially since the code is already there.

[xhdong-umd]

The legend for base plot is placed inside the plot region by default. If I draw both location points and home range contours, then add legend for them, the legend could take too much space and overlap with plot.

(the plot above don't have points drawn yet)

It's possible to move the legend outside but the parameter often need manual tweaking depend on legend name lengths.

I'm inclined to just disable location points in this plot, which make the home range contour and density easier to read, and we have the ggplot point plot tab if user need to check the location points.

[xhdong-umd]

So I'm planning to just draw contour and densities, with a color legend.

There is no need for controls of layers since we only have contours and densities, and they don't interfere with each other.

[xhdong-umd]

How should I remove the grids? I only saw col.grid, which is not really an option to turn it off unless I make some transparent color?