search

posit-dev / positron

Positron, a next-generation data science IDE
Other
2.26k stars 68 forks source link

Large dataset will not load inside R 4.4.1 console on Linux #4019

Open jonpeake opened 1 month ago

jonpeake commented 1 month ago

Positron Version:

Positron Version: 2024.07.0 build 49 Code - OSS Version: 1.90.0 Commit: 677a0d0f0a4f5f9389507c079c8b9a67699d52fd Date: 2024-07-15T03:09:23.962Z Electron: 29.4.0 Chromium: 122.0.6261.156 Node.js: 20.9.0 V8: 12.2.281.27-electron.0 OS: Linux x64 6.8.0-38-generic

Steps to reproduce the issue:

  1. Attempt to load large .RData file (22.1 GB on disk) containing a coda MCMC object
  2. Wait for the console to finish

What did you expect to happen?

The console should finish loading the dataset and show it in the Variables list.

Were there any error messages in the output or Developer Tools console?

No error messages were reported in the output or the Developer Tools console. The Console shows the green "running" highlight while the dataset is being loaded (tracked in RAM and disk access). As soon as the full dataset is finished "loading" from the disk, the console green highlight goes away, the "Interrupt Execution" button goes away, but the ark system process hangs at 3.1% and the Variables tab does not show the variable in question. Additionally, the console will not accept any further commands (any attempt to execute a command results in the command remaining in the > line. I have not reached a point where the console rights itself again, after waiting over 15 minutes for the console to regain response.

The same process on RStudio takes only a few seconds to show the dataset after it has finished loading. The system used has 64GB of RAM available, so I highly doubt this has anything to do with the system using swap-space.

jonpeake commented 1 month ago

Quick update, it does appear that the dataset was finally loaded into Positron, but it took approximately 25 minutes for the console to regain response. This seems like it may also be related to https://github.com/posit-dev/positron/issues/4008.

juliasilge commented 1 month ago

This could be related to R being locked while we generate the update message for the Variables pane, as in #3628.