search

microsoft / RTVS

R Tools for Visual Studio.
MIT License
390 stars 118 forks source link

cpu & memory usage climbs the longer i have R source & interactive windows open #3565

Closed myusrn closed 7 years ago

myusrn commented 7 years ago

Please briefly describe what you were doing that led to the issue, if applicable:

If i use vs17 open folder solution view mode and start working on R sources in that folder i'm finding that as time goes by the longer i have R source and interactive windows open the more my cpu and memory usage creeps up until the point where i have to stop, save everything, close and reopen vs17 in order to get back to low cpu and memory numbers.

I've done empirical tests with my workspace set to Microsoft R Client (3.3.2.0) and R 3.3.2 and seem to get the same result with both.

If i carry out the same activities for similar durations in RStudio, configured to use R 3.3.2, i don't find my cpu and memory usage climb based on fact the cpu usage for it in task manager never reports more than 1% and i never hear my laptop cooling fan come on after using it for a 10mins+ like it does after using vs17 rtools for 10mins+.

If you have any screenshots demonstrating the issue, please include them as well to help us diagnose it better.

vs17rtoolscpumemoryusagetimeline Additional information:

OS Information Version: Microsoft Windows NT 10.0.15063.0

RTVS Information: Assembly: Microsoft.VisualStudio.R.Package, Version=1.1.30413.947, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a

MikhailArkhipov commented 7 years ago

Memory is measured in the R host process (Microsoft.R.Host.exe), not in VS. We run R out of process. 252 MB for VS is quite OK :-) However, CPU consumption is indeed high.

Now, could you please, when it happens, do the following

  1. Run another instance of VS
  2. Debug | Attach to Process
  3. Locate devenv.exe
  4. In the dialog Click Select... and check Managed 4.6 and Native (see below)
  5. Close dialog
  6. Debug | Break All
  7. Debug Save Dump...
  8. Share the file somewhere (it is big).

Thanks!

image

myusrn commented 7 years ago

Thanks for the details on how to provide relevant followup information.

I captured three devenv.dmp files associated with times when cpu usage being reported in vs17 had reached the 30% range causing laptop fans to spin up.

In the second case this arose coincidentally ??? with an ide hang right after executing script statements "install.packages("data.table"); library(data.table)", using ctrl+enter, which never produced expected output in R interactive window.

In the third case it successfully completed the install.packages() and library() function calls above producing expected results in R interactive window . . . but never the less the cpu usage was reporting 30% range afterwards.

For all three cases i took screen grab of vs17 ide machine utilization details and task manager processes sorted by cpu to show what they were reporting just before i used 2nd instance of vs17 to debug attach to first instance and break in order to generate dump file.

Files can be downloaded from the following urls . . . http://mysawestus.blob.core.windows.net/public/devenv1.dmp http://mysawestus.blob.core.windows.net/public/devenv1.png http://mysawestus.blob.core.windows.net/public/devenv2.dmp http://mysawestus.blob.core.windows.net/public/devenv2.png http://mysawestus.blob.core.windows.net/public/devenv3.dmp http://mysawestus.blob.core.windows.net/public/devenv3.png http://mysawestus.blob.core.windows.net/public/devenv4.dmp http://mysawestus.blob.core.windows.net/public/devenv4.png

MikhailArkhipov commented 7 years ago

Most probably https://github.com/Microsoft/RTVS/issues/3424

karthiknadig commented 7 years ago

Looks like about 17% of the heap blocks are taken by object allocated in wisp.dll. To actually get more details we need to enable gflags /i devenv.exe +ust

0:000> !heap -s

************************************************************************************************************************
                                              NT HEAP STATS BELOW
************************************************************************************************************************
LFH Key                   : 0x11eb51aa
Termination on corruption : ENABLED
  Heap     Flags   Reserv  Commit  Virt   Free  List   UCR  Virt  Lock  Fast 
                    (k)     (k)    (k)     (k) length      blocks cont. heap 
-----------------------------------------------------------------------------
00b80000 00000002   48796  34484  48740   1814   704     7    2      3   LFH
00a60000 00001002   15460   4892  15404    651   144    16    0      1   LFH
    External fragmentation  13 % (144 free blocks)
00b50000 00001002     116     64     60     15     3     1    0      0   LFH
037b0000 00001002    1136     88   1080     18     7     2    0      0   LFH
02e30000 00001002    1136    144   1080     12    11     2    0      0   LFH
037a0000 00041002      60      4     60      2     1     1    0      0      
05dd0000 00041002    1136    608   1080     44     5     2    0      0   LFH
063e0000 00001002    7272   2972   7216    951    68     4    0      0   LFH
    External fragmentation  32 % (68 free blocks)
0aa10000 00001002    1136    412   1080    152     6     2    0      0   LFH
0ff60000 00001002    1136    172   1080      3     2     2    0      0   LFH
0ff30000 00001002      60      4     60      2     1     1    0      0      
-----------------------------------------------------------------------------
0:000> !heap -stat -h 00b80000 
 heap @ 00b80000
group-by: TOTSIZE max-display: 20
    size     #blocks     total     ( %) (percent of total busy bytes)
    78 148b - 9a128  (3.65)
    18 62aa - 93ff0  (3.51)
    114 839 - 8dd74  (3.36)
    54 180b - 7e39c  (2.99)
    fdc 72 - 70ff8  (2.68)
    7c e4e - 6edc8  (2.63)
    40 1a36 - 68d80  (2.48)
    390 1a2 - 5d120  (2.20)
    20 29fb - 53f60  (1.99)
    10 4668 - 46680  (1.67)
    849 6e - 38f5e  (1.35)
    35850 1 - 35850  (1.27)
    70 774 - 342c0  (1.24)
    3c d13 - 31074  (1.16)
    a0 4cc - 2ff80  (1.14)
    60 79d - 2dae0  (1.08)
    3bc c2 - 2d478  (1.07)
    f8 2d1 - 2ba78  (1.03)
    90 4d5 - 2b7d0  (1.03)
    26674 1 - 26674  (0.91)

0:000> !heap -stat -h 00a60000 
 heap @ 00a60000
group-by: TOTSIZE max-display: 20
    size     #blocks     total     ( %) (percent of total busy bytes)
    11c9 2a - 2eafa  (17.07)
    1cc9 b - 13ca3  (7.24)
    19c9 b - 11ba3  (6.48)
    1c10 7 - c470  (4.49)
    9ff8 1 - 9ff8  (3.66)
    8610 1 - 8610  (3.06)
    8000 1 - 8000  (2.93)
    158 4d - 6778  (2.36)
    240 2d - 6540  (2.31)
    3180 2 - 6300  (2.26)
    204 2f - 5ebc  (2.16)
    2780 2 - 4f00  (1.81)
    214 26 - 4ef8  (1.80)
    26d8 2 - 4db0  (1.78)
    cc0 6 - 4c80  (1.75)
    1178 4 - 45e0  (1.60)
    2c0 18 - 4200  (1.51)
    98 6c - 4020  (1.47)
    cc9 5 - 3fed  (1.46)
    3f54 1 - 3f54  (1.45)

0:000> !heap -flt s 11c9
    _HEAP @ b80000
    _HEAP @ a60000
      HEAP_ENTRY Size Prev Flags    UserPtr UserSize - state
        0648d038 0241 0000  [00]   0648d040    011c9 - (busy)
          wisp!ATL::CComObject<CMTContext>::`vftable'
        0648e240 0241 0241  [00]   0648e248    011c9 - (busy)
myusrn commented 7 years ago

Am i to run "gflags /i devenv.exe +ust" from administrator command prompt before repro'ng this matter in vs17 rtools session and then attach debugger | break | create dump file again or am i entering this into debugger session before after i break to capture dump file?

karthiknadig commented 7 years ago

Actually it will be better if you can give some steps that you did. It is easier to analyze this with a live debugger attached.

myusrn commented 7 years ago

Not really doing anything special that i can see. Simply starting vs17 and opening folder with .R scripts in it, and then start using ctrl+enter to execute specific lines or chunks of those scripts as well as using R Help and global.env Variable Inspection pane views.

The R code is what i expect to be basic stuff covered as part of the aka.ms/dsJhu curriculum, e.g. reading in csv & text files using read.* and fread functions followed by data manipulation and cleanup. No plotting or statistical analysis function calls being used at this time.

The cpu rampup seems to only start happening after some install.packages + library() calls followed by execution of functions provided by packages. If i can get time i'll try and test to confirm that opening vs17 against folder with R scripts in it and loading one of them and just execute only base packages provided functions leads to this cpu climb or not. All the same activities in RStudio do not cause cpu rampup.

I'm open to a w10 remote assistance (msra.exe) or skype desktop sharing session to facilitate oversight and additional debug data collection while i repro the issue. Usually happens in 10mins. send mailto:myusrn@outlook.com if you want to try that.

If it weren't for this issue i'd be using vs17 rtools support as i find it easier to flip between scripts, interactive, plots and variable views, and popout tabs for full screen views, in that environment than in rstudio environment.

MikhailArkhipov commented 7 years ago

Update to 15.2 will probably be release next week

myusrn commented 7 years ago

status update

I have been running vs17/dev15.2 (26430.12) r tools (1.1.30523.1233) update since yesterday and so far this devenv.exe process cpu usage gradual run up, and eventual ide hang, issue has not repro'd having made no other changes to workstation . . . at least none that i can tell. Now after >10mins of use, e.g. 1hr+ i find vs17 ide machine utilization for cpu reading floats between appx 5% and 15%.

Thanks for whatever you did to make this go away and the vs17 rtools usable in lieu of rstudio for the work i'm doing. The window placement, sizing and ctrl+dblclick popout/backin flexibility feels more useful to me than what i get in the rstudio experience. Especially so in ability to place script and interactive views down center of ide while having insight into plot and help windows taking up left and right sides of screen real estate.

Also note that i'm now testing operating my vs17/dev15.2 setup with data science workload configured to have just the following components installed as i don't foresee using any of the python related components of that workload.

//sCmd += " --add Microsoft.VisualStudio.Workload.DataScience"; // data science and analytical applications sCmd += " --add Microsoft.VisualStudio.Component.R.Open"; // microsoft r client (3.3.2) sCmd += " --add Microsoft.VisualStudio.Component.RHost"; // runtime support for r development tools sCmd += " --add Microsoft.VisualStudio.Component.RTools"; // r language support

MikhailArkhipov commented 7 years ago

OK, glad it is working for you, it was #3424 then :-)