Geographic analysis is eating up all my VM disk + my host disk

[nicolasray]

When launching the geographic analysis at 120min on the datasets of 5 states of Sudan, after only 6-7 facilites (out of 80) the VM is full, and its size on my disk is close to 40 GB. However, AccessMod still tells me that I'm still using only 7.5 GB out of the max. 40 Gb: . In another previous run on the same data set (but with no live monitoring form myself), I let my computer continues to run the analysis. The VM reaches 40GB, but more than 100 GB were eaten on my disk, which brings it to a critically low disk space situation and it shut down automatically. After restarts, this 100 GB were not freed immediately, but eventually become free after some 20-40 min of computer uses (which points into the direction of some cache on my disk).

Logs are attached: AccessModLogs-2018-11-12@00_44_38.xlsx

[nicolasray]

Closing this issue for the moment, as it seems it was a peculiar behaviour in earlier version. Will reopen if it occurs again.

[nicolasray]

I just have the same behaviour than I got in this old issue, so I'm reopening. This is high priority.

This happens with the project Ethiopia (I put it in the Dropbox under the BUGS folder). When I do a referral analysis, it's eating up all my VM disk (40 GB), then continue eating my disk (I stopped the VM when it had eaten more than 25 GB in addition to the 40 GB), just after a few iteration is seems. I used a VM extended to 6 GB RAM, with 3 CPUs. To reproduce, the referral should be launched with the following parameters:

• merge land cover: landcover merge
• scenario table: scenario
• facilities: facilities ADJUSTED
• facility ID field: facilityid
• facility name field: faciname
• anisotropic, limit the analysis to the closest destination point in time
• max travel time: 300
• check "Optimize dynamically computation according to the scenario"
• FROM facilities: unselect all, facilityty == "Health Center" --> select
• TO facilities: unselect all, facilityty != "Health Center" --> select

[fxi]

Which version of AccessMod are you using ?

The data provided had remaining mapsets that should have been removed. This could explain that the size of the vm keep growing : each worker need around 4.6 GB of temporary data in your case. after 10 iterations, the virtual disk could have been filled.

This does not seems to happen in 5.6.26 (testing branch).

Meanwhile, I will continue to investigate.

[nicolasray]

I used the current version, not 5.6.26.

-- Sent on the move ->

Le 10 janv. 2020 à 13:13, F.Moser notifications@github.com a écrit :



Which version of AccessMod are you using ?

The data provided had remaining mapsets that should have been removed. This could explain that the size of the vm keep growing : each worker need around 4.6 GB of temporary data in your case. after 10 iterations, the virtual disk could have been filled.

This does not seems to happen in 5.6.26 (testing branch).

Meanwhile, I will continue to investigate.

— You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHubhttps://github.com/fxi/AccessMod_shiny/issues/214?email_source=notifications&email_token=ACTLTG3KVSPW4QJMWWPF5BDQ5BQ65A5CNFSM4GDC2JG2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEITXR6Q#issuecomment-573012218, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ACTLTG4A4E25DXZFGLNXMUDQ5BQ65ANCNFSM4GDC2JGQ.

[fxi]

Ok. I've ran a 5 facilities analysis, for 273 destination (1365) referral. Done in 27 minutes.

VM internal disk (1K-block, used, available)
before
/dev/sda1                       41251136   3080620  36434604
after
/dev/sda1                       41251136   3080852  36434372  

Data volume (1K-block, used, available)
before
srv_shiny-server_data_grass    976797816 435800392 540997424  
after
srv_shiny-server_data_grass    976797816 442532180 534265636

So, before/after : 6.42 MB .. That is also too much. It should be around 70kB max..

I will continue