Saving progress

[NoonanM]

I've had a request to add save buttons to each of the tabs in the app so that work can be easily saved as users progress through the app. Would it be possible to add a save feature so users can safeguard against crashes?

[xhdong-umd]

Everything happened were recorded by default, with data, plots, user selection etc. User can save the work report and app data in work report page any time. I don't see the point of a save button in each page, actually we can only save data, changed data, data changing operations. We cannot "save" everything user did in app, especially interactions didn't change the data, just exploring the data.

Unless the app will crash so bad that a report cannot be exported, I don't see the need to "save" from time to time. If user do meet some fatal crash, we can check if that can be contained so the work can still be saved.

[NoonanM]

The user in question spent a day cleaning outliers, then tried to save data on the work report page but the app crashed before being able to save the report. They felt it would be useful to be able to save after cleaning each animal directly from the filtering page. Would it not be possible to duplicate the 'Save Data' button?

[xhdong-umd]

Did they save the error message? Can the error be reproduced? I should look at this kind of bad crashes and try to prevent them, instead of relying on save data.

Do you mean to place a button in every page that function like the save data button? Maybe that's possible. Although they can just click the work report page and save data, then come back to the page continue the work. The only advantage of that duplicate button is to save two clicks.

[NoonanM]

No, unfortunately, they didn't save the error message and I tried to reproduce the error, but wasn't able to. They also couldn't remember what the message said.

Yeah, I did mean adding the save data button to every page, but maybe it only needs to be added to pages where the user actually manipulates data? And yeah, I know the advantage is mostly to save two clicks, but it might also encourage people to save periodically?

[NoonanM]

Also, I'm not sure if the crash was related to the app, or to their computer (which might have struggled with RAM issues).

[chfleming]

In addition to a save button on every page (with the iconic 3.5" floppy disc icon), I think an auto-save feature would also be good.

[xhdong-umd]

Auto-save will post a little bit too much burden on performance.

The current work report is saving every plot after every update, which slow down the app visibly already but I don't see any way to reduce it other than an option to turn off the log.

To save the updated data frequently, especially when the input data is not small, could be too much.

I still think crashes should be detected and contained, instead of relying on saving, which is not really a solution if the app still crash randomly.

An alternative of putting save button on every page is to put the button on the left side bar, though it may look a little bit weird.

[xhdong-umd]

The button in side bar look like this. Then I can remove the work report page and move the preview/download report button to first page app option box.

[xhdong-umd]

I also found that putting same button in multiple places will totally mess up the app. That will not work.

[xhdong-umd]

I modified the style of the save button to make it match side bar style better, also rearranged the other buttons in work report page to the App option box.

[NoonanM]

I like the new save feature.

So I know you mentioned that saving the workspace would be too much. I'm wondering if it would be possible to save the modeling results as an rda?

[xhdong-umd]

The saved data included everything can be saved, see the help text under preview report button:

• Save Data button in side bar will save these data as a zip:
  • combined_data_table.csv: the current combined telemetry data.table. It can be imported into app later as csv.
  • data.rds: a list that can be imported into R later with readRDS
  • input_tele_list: the original input telemetry object list
  • tele_list: the current telemetry object list, for example some outliers can be removed, new time subset may be added.
  • merged: list of
    • data: the combined data.table of all animals
    • info: the summary table
  • all_removed_outliers: all rows that were removed as outliers.
  • cache.zip: cached calculation results. Exactly same calculation can be finished instantly
  • report.html: work report.
  • plot.zip: the plots saved as png or pdf files.

For modeling results, it's a list of complex structure and highly depend on current selected subset of animals. So it will be useless unless your current subset is exactly same with the saved model result. We have all the modeling process cached, and that means any modeling operation will be finished immediately if you are using exactly same input. Loading saved data will also restore cache, so you just need to click the try models button again and it will complete immediately(you still need to click that button because the app need to take the current selected subset, so we cannot just restore last modeling result which could be wrong).

[NoonanM]

Ah okay great! I didn't realise the cache was also saved.

In that case, it might be worth changing the name of the button from Save Data, to Save Progress?

[xhdong-umd]

Save Progress is a better name. Though we have a Load Data button to import the saved zip. Is Load Progress sound OK? Maybe Restore Progress?

[NoonanM]

Restore Progress sounds good.

[NoonanM]

Ok, last question. I know you said the user has to click on try models to get their results back, but would it be possible to automatically apply the try models feature if the user is restoring progress?

[xhdong-umd]

It make senses to apply it automatically since the data was just restored and the subset is same. Though it's a little bit cumbersome to wrap the code so I can call it again.

Alternatively, how about we remove the try models button, just run the code when the modeled tab is selected?

• Previously there was a button because all modes are in same page, and I don't want to start fitting unnecessarily.
• Now there are 3 separate tabs, if user clicked the modeled tab, he/she most likely want to start fitting.
• In the restoring progress, user need to open model selection page and select the modeled page anyway to see the result, so I think the automate fit should work the same and save one click.

[NoonanM]

I agree, removing the try models button is probably the best option. Like you said, for the majority of users try models is going to be the next action after clicking on the modeled tab.

[xhdong-umd]

I forgot how complex the try models was. There was a complex order of events and possible infinite loop if not careful.

Finally I made it work. Note the process need data from 1st tab of empirical variogram, so if user selected the 3rd tab, switch back to visualization page to change the current subset, switch back to model selection page, the app will clear previous models and jump to 1st tab automatically. After that user can click 3rd tab to try models.

[xhdong-umd]

I made another change so that when current subset changed, the tab will be switched back to 1st tab automatically. This should ensure the data consistency better.