ctmm-initiative / ctmmweb

Web app for analyzing animal tracking data, built upon ctmm R package
http://biology.umd.edu/movement.html
GNU General Public License v3.0
30 stars 21 forks source link

Option to calculate individualized home ranges #98

Closed chfleming closed 4 years ago

chfleming commented 5 years ago

For people with spread out individuals, where overlap is not interesting, calculating the home ranges on one single grid takes too much RAM. It would be nice to have an option to calculate the home ranges individually (ruling out overlap).

markbidwell commented 5 years ago

Thanks for posting this request Chris.

Xianghui, I have 22 individuals with small home ranges that are spread out over a large spatial extent. Currently, I can do model selection on all 22 individuals at the same time, but I can't calculate HRs for them simultaneously (run out of RAM) so I have to do them one by one which is time consuming. Thanks for considering adding this feature to ctmmweb.

xhdong-umd commented 5 years ago

It's not hard to add the feature but this may need to change how app works.

Right now the app start to calculate home range automatically once switched to the page. The option to turn off same grid need to happen before this.

We can put this option in model page but it is a little bit counter intuitive. The other approach is to let user start home range calculations manually by clicking a button, and user have chance to change settings if needed.

Which one do you prefer?

markbidwell commented 5 years ago

I would prefer the second option, I.e. click button on HR page after selecting the option for same/different grid. Thanks!

On Jun 23, 2019, at 3:24 PM, xianghui dong notifications@github.com wrote:

It's not hard to add the feature but this may need to change how app works.

Right now the app start to calculate home range automatically once switched to the page. The option to turn off same grid need to happen before this.

We can put this option in model page is a little bit counter intuitive. The other approach is to let user start home range calculations manually by clicking a button, and user have chance to change settings if needed.

Which one do you prefer?

— You are receiving this because you commented. Reply to this email directly, view it on GitHub, or mute the thread.

xhdong-umd commented 4 years ago

@chfleming If we don't use same grid, how would this impact the overlap page analysis?

chfleming commented 4 years ago

I would disable that page if individualized estimation is toggled. That would be the easiest solution, at least.

xhdong-umd commented 4 years ago

Instead of adding a button and user have to manually click it, maybe the better approach is to pop up a dialog in this page, ask user to confirm before continue. This way user can either just use default option, or have the chance change option before going forward.

chfleming commented 4 years ago

Can we have a toggle with joint analysis by default and then catch the OOM error on the home range estimation and popup only after OOM to change the toggle to individualized analysis or ask the user to only analyze nearby individuals for this.

xhdong-umd commented 4 years ago

I am not sure if the R process can continue to work normally after a out of memory error. I think there used to be some sample code reproduce the out of memory error.

chfleming commented 4 years ago

Then maybe we could do a sanity check for the size of the grid compared to the amount of RAM on hand?

chfleming commented 4 years ago

The sample code was just taking one buffalo and adding a large number to either the x or y column.

xhdong-umd commented 4 years ago

I'm not sure an estimate of memory to be used is reliable, because how much memory will be used is highly depend on platform and other factors like Garbage Collection, and it's not easy to get an accurate report of usable RAM which is controlled by OS and others.

I think it will be trying too much to do all of this in app end. Maybe the simple and more reliable method is just to let user select the option before proceed. If the memory was not enough, choose a different option in next run. User can save the progress before running home range to save some time.

xhdong-umd commented 4 years ago

To calculate home range separately, we just supply telemetry data and ctmm object one by one instead of in a list, right?

chfleming commented 4 years ago

Yes.

xhdong-umd commented 4 years ago

This change is quite different from previous behavior since the home range estimation need to be triggered manually. One thing worth to note is:

Thus I think once user made change to model selection, it's better to clear the home range page. Then user will know that it need to be estimated again.

markbidwell commented 4 years ago

Hello, Is there an updated version of ctmmweb that allows me to turn same grid off for home range estimation? I am running into the same problem I reported earlier, estimating home ranges for even a small number of individuals doesn’t work, I think because they’re on the same grid. Thanks for any ideas, Mark

On Nov 5, 2019, at 6:29 PM, xianghui dong notifications@github.com wrote:

 This change is quite different from previous behavior since the home range estimation need to be triggered manually. One thing worth to note is:

Previously when the model selection changed, once user move to home range page it will update automatically. Now the update has to be triggered by user again. This might be a little bit confusing when user changed selection then moved to home range page and still seeing home range based on previous model selection. Thus I think once user made change to model selection, it's better to clear the home range page. Then user will know that it need to be estimated again.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub, or unsubscribe.

xhdong-umd commented 4 years ago

I am working on this and about 50% finished. I probably can update it before next Tuesday.

chfleming commented 4 years ago

I'm not sure that I follow, but I will test it after you have it working.

xhdong-umd commented 4 years ago

@markyxe , I just updated the master branch with the feature to calculate home range individually. You can try if this will solve your problem. Thanks!

jmcalabrese commented 4 years ago

@xhdong-umd Sorry to be late to the party here... I was just rerunning the jaguar data for the ctmmweb paper and noticed the need to manually start home range estimation. This is a pretty big departure from the way the other analyses work in the app (automatically upon click the page or tab).

Would it be possible to have estimation triggered automatically upon clicking on the home range page, as it was before, but with a pop-up box with radio buttons to choose between common grid or individual grid? It seems like you made a similar suggestion above, but didn't implement it. In any case, I think that would be more intuitive and more consistent with the way the rest of the app works.

xhdong-umd commented 4 years ago

I don't remember why I didn't go with that route, or if there was some tech difficulties. I'll try this, thanks!

xhdong-umd commented 4 years ago

It might be not easy to use a radio button (it will only exist when the dialog is present, so it may be gone after dialog is closed). Alternatively we can pop up a dialog with 2 buttons for common grid and individual grid respectively.

I'm not sure if we should have a cancel button. If cancelled, home range will not update, then how should user initialize the option again? So maybe we don't put a cancel button (it's like before, if you click home range when model updated, it will run anyway).

jmcalabrese commented 4 years ago

Doesn't have to be radio buttons, just some way to give the user the choice between grids. I agree that there shouldn't be a cancel button.

xhdong-umd commented 4 years ago

To revert to previous style I need to undo a lot of things... I really wished I tried harder in the pop up dialog route. It's not simple to implement so I didn't go ahead...

xhdong-umd commented 4 years ago

Turned out I don't have to undo the changes I made, as the new change is based on the changes. However it's really not straightforward to make it work.

Now finally it's working.

jmcalabrese commented 4 years ago

Great, thanks for your efforts @xhdong-umd. Looking forward to checking it out in the next update.

xhdong-umd commented 4 years ago

Master is updated with this and the start page rename. I haven't update the cloud hosted version though.

jmcalabrese commented 4 years ago

@xhdong-umd Just checked out the updates on Master. That works really nicely!

jmcalabrese commented 4 years ago

Wait a minute...just noticed two problems with the updated home range page:

jmcalabrese commented 4 years ago

Hang on again...re-estimating the HRs after choosing different models does actually work, but there is a 30 second or so lag after clicking "home range" in the sidebar before I get the popup with grid choices and the estimate button. That lag does not happen on the first time through, when just taking the selected models. @xhdong-umd can you reproduce that behavior?

xhdong-umd commented 4 years ago

Sorry, I should test it more. The logic to implement this is quite tricky.

xhdong-umd commented 4 years ago

Master updated with the fix on home range summary table. I didn't see the delay in selecting different models in my side though.