Holy RAM

[oojacoboo]

I guess this is why it's so slow. Why is this service taking up so much memory?!

[jakubmisek]

Thank you for reporting this. I know it's happening but we couldn't find out why.

May I ask you for more details? I'd be happy to fix this asap.

• what is the size of your workspace
• maybe used packages?
• for how long is it running, how long is the VSCode workspace opened
• or any steps to reproduce?

I'd appreciate any info.

[oojacoboo]

Sure. The entire workspace is nearly 20GB in total. But, that's all files and not just PHP files, which the language server should only be processing. I'm using the following command which is giving me roughly 1GB for all the PHP files in the workspace.

find . -name "*.php" -exec du -ch {} \; | awk '{total+=$1}END{print total}'

The editor has been running for maybe a day - not even sure. VSCode update and restarts itself a lot and I've had to restart it a lot recently with this plugin. That shouldn't be a huge consideration or issue. It's not like it's been running beyond what would be considered normal.

No idea on reproducing. This issue doesn't occur with Intelliphense.

[d8vjork]

@oojacoboo Sure that doesn't count vendor folder or anything?

[oojacoboo]

It does count the vendor directories, as it should. They must be indexed and used in auto-complete.

[jakubmisek]

Thank you! Yes, vendor should be included ... @d8vjork may have found one possible bug (https://github.com/DEVSENSE/phptools-docs/issues/285#issuecomment-1438273371) but it would not cause 6gigs of RAM used.

We'll do more testing, I'd be very interested in reproducing this issue on our machines; we'd perform complete memory profiling ...

[oojacoboo]

It gets worse. Quitting VSCode isn't killing this node server or whatever is handling the indexing. Of course, VSCode was force-quit since it was bringing my system to it's knees by draining available memory and causing system paging.

It ended up corrupting memory on dozens of other apps in the process. Going to have to move back to Intelliphense until this issue is resolved.

[jakubmisek]

I would really love to be able to reproduce this issue :-)

There is probably an infinite loop happening .. we'll keep trying.

[ggolda]

@jakubmisek have the same issue, maximum I had was 8GB of RAM after executing lots of composer commands related to install/update. Maybe extension doesn't free memory if file is no longer in use?

[jakubmisek]

@ggolda thank you for letting me know - I'll try composer commands subsequently. Is it something specific? Installing packages? Removing packages? Updating?

[ggolda]

@jakubmisek just tried it again, was doing a global laravel update via composer update several times, followed by composer remove {package-name} and composer require {package-name} and my RAM consumption went from 1GB idle to 5GB after executing these commands. Also VSCode reported that it is trying to index 46k files after each command (I guess its fine in this case).

RAM consumption haven't gone down after leaving it in this state for a couple minutes without doing anything.

[jakubmisek]

@ggolda I have a suspicion there is a memory leak in parsing .phar files .. I'll try updating Laravel a few times on various OS's

[jakubmisek]

alright, every time I update/copy a new phpstan.phar file, it consumes about 300MB more RAM and never return it back

[jakubmisek]

I've fixed the memory leak happening when updating a .phar files.

We'll prepare a pre-release within a day.

There are other memory-related improvements for future updates, but this one is fixing the issue at least for me (it happened on most composer update actions)

[jakubmisek]

please update to pre-release 1.32.12898

[Songworks]

Hi there, been having similar issues. The devsense.php.ls process gobbles up memory like there's no tomorrow. Saw ~9GB yesteryday evening after couple hours working on a Shopware 6 project (which is PHP & Symfony based).

When I start vscode and the language server starts indexing files or some such? That uses up 50% of my CPU, increasing memory usage up to ~4.5GB once it's finished.

After that, it keeps increasing it's memory usage the longer I use vscode. Which for me is generally just working files, php and otherwise. No composer installs or vendor changes or some such.

edit: Just realized, Shopware 6 (or more precisely Symfony?) caches alot of PHP, including twig-templates as PHP code. When I clear/warmup the cache the devsense.php.ls process uses 8% cpu for a few seconds and memory usage moves up and down. I guess the language server tries to reindex the deletes/inserts/updates of the cached PHP files?

Anyway, am trying the prerelease v1.32.12898, and will report back. Starting behaviour and the ~4.5GB usage of it is still the same though.

System Information:

vscode: (Extension first installed on 1.76.1, just updated vscode to check if it helps) Version: 1.76.2 Commit: ee2b180d582a7f601fa6ecfdad8d9fd269ab1884 Date: 2023-03-14T17:57:21.103Z Electron: 19.1.11 Chromium: 102.0.5005.196 Node.js: 16.14.2 V8: 10.2.154.26-electron.0 OS: Linux x64 5.15.0-67-generic Sandboxed: No

OS: Linux Ubuntu 22.04.2 LTS

Beyond this issue (had to increase swap, thank god for nvme ssds :D), the extension is great though! Thanks!

[Songworks]

Report: After couple of hours working with prerelease v1.32.12898 the ram usage of the devsense.php.ls went down over time. From the ~4.5GB after vscode startup, to ~3.4GB now.

Although I then went back to stable release and while it started out with ~5.0GB, it went down to ~3.8GB also.

So yeah. I guess today I can't reproduce the excessive ram usage today...

However, I can confirm, the language server does try to index cached PHP files. Is there a way to exclude files/directories from being indexed? Probably would reduce the memory usage for me:

[...] (A whole lot more of these)
> Removed path 'file:///project/path/source-code/var/cache/dev_h404a97683f5f1de1a7d8afd72c86db43/profiler/ef' ...
> Removed path 'file:///project/path/source-code/var/cache/dev_h404a97683f5f1de1a7d8afd72c86db43/profiler/f0' ...
> Removed path 'file:///project/path/source-code/var/cache/dev_h404a97683f5f1de1a7d8afd72c86db43/profiler/f2' ...
> Removed path 'file:///project/path/source-code/var/cache/dev_h404a97683f5f1de1a7d8afd72c86db43/profiler/f5' ...
> Removed path 'file:///project/path/source-code/var/cache/dev_h404a97683f5f1de1a7d8afd72c86db43/profiler/f7' ...
> Removed path 'file:///project/path/source-code/var/cache/dev_h404a97683f5f1de1a7d8afd72c86db43/profiler/f8' ...
> Removed path 'file:///project/path/source-code/var/cache/dev_h404a97683f5f1de1a7d8afd72c86db43/profiler/f9' ...
> Removed path 'file:///project/path/source-code/var/cache/dev_h404a97683f5f1de1a7d8afd72c86db43/profiler/fa' ...
> Removed path 'file:///project/path/source-code/var/cache/dev_h404a97683f5f1de1a7d8afd72c86db43/profiler/fd' ...
> Removed path 'file:///project/path/source-code/var/cache/dev_h404a97683f5f1de1a7d8afd72c86db43/profiler/fe' ...
> Removed path 'file:///project/path/source-code/var/cache/dev_h404a97683f5f1de1a7d8afd72c86db43/profiler/ff' ...
> Removed path 'file:///project/path/source-code/var/cache/dev_h404a97683f5f1de1a7d8afd72c86db43/pools/system' ...
> Removed path 'file:///project/path/source-code/var/cache/dev_h404a97683f5f1de1a7d8afd72c86db43/Symfony/Config' ...
[...] (A whole lot more of these)
> Indexing directory 'file:///project/path/source-code/var/cache/dev_h404a97683f5f1de1a7d8afd72c86db43/profiler' ...
> Indexing directory 'file:///project/path/source-code/var/cache/dev_h404a97683f5f1de1a7d8afd72c86db43/profiler/69' ...

[jakubmisek]

@Songworks thank you for all the information!

Please exclude the "cache" directory using the Visual Studio Code setting "files.exclude", i.e.:

"files.exclude": ["/cache/"]

I hope that will help.

If you'd have anything unexpected in the output again, please let me know. We're trying to improve the RAM usage within every new update.

[Songworks]

@jakubmisek Thank you! That helped at lot. On startup the devsense.php.ls is now down to ~2.7GB! :)

However, the language server still seems to detect and index changes to the cache (I guess while a local server with php-fpm is running, it is making some changes to the cache):

PHP Tools server started.
    PID: 1094816
    Processing files: *.php; *.phtml; *.html5

> Parsing Phar file '/project/path/source-code/ci/vendor/phpstan/phpstan/phpstan.phar' ...
> Parsing Phar file '/project/path/source-code/ci/vendor/phpstan/phpstan/phpstan.phar' ...
> Parsing Phar file '/project/path/source-code/ci/vendor/bin/phpstan.phar' ...
> Parsing Phar file '/project/path/source-code/ci/vendor/bin/phpstan.phar' ...
> 1 file system change(s) ...
> Indexing directory 'file:///project/path/source-code/var/cache/dev_h404a97683f5f1de1a7d8afd72c86db43/profiler/e6' ...
> 1 file system change(s) ...
> Indexing directory 'file:///project/path/source-code/var/cache/dev_h404a97683f5f1de1a7d8afd72c86db43/profiler/1c' ...
[...] And so on

settings.json:

{
    "files.exclude": {
        "node_modules": true,
        "var/cache": true
    },
}

[jakubmisek]

@Songworks ... However, the language server still seems to detect and index changes to the cache

I believe it's only logging the information to the output window, but then it's not indexing it. I'll update the log so it's less confusing) Also, I'll make sure it's really working ...

[oojacoboo]

@Songworks That's great, but even 2.7GB is way too high IMO. Right now, intelephense, on my system, is running at 1GB - at least I assume that's the plugin using 1GB, since it's the highest. Our codebase is quite large as well. So, it seems to be corresponding with the total size of all PHP files in our codebase, which makes sense.

It's also worth mentioning that we don't have any "cache" files, unless we're considering Composer autoload files as cache.

We do regenerate our models/entities, which will result in new files, quite a lot of them. I would assume this plugin is watching file pointers and dropping these from it's internal index.

[d8vjork]

I think RAM as a "cache" system is good for performance, generally speaking disks are slower even the faster SSD doesn't have anything to do with a module of RAM.

2.7GB is nothing compared to what other stuff like Docker consumes as minimum requirement

[jakubmisek]

@oojacoboo thank you for the information. We'll keep working on it. I think the 50% decrease in RAM usage is achievable.

And yes, our PHP extension is maintaining the internal index indeed, keeping only files included in the workspace. The issue was in leaking .phar files.

[jakubmisek]

@d8vjork thank you, Ruben. I think so as well!

[marinaglancy]

My memory usage just got to 100% and I found this in the system log. This is AFTER I closed vscode, btw. Why is there so many processes, why is the memory usage so high and why do they keep running when vscode is closed?

[jakubmisek]

Hello Marina,

this is a bug. There should be exactly one devsense.php.ls instance for one code window.

[Songworks]

I've noticed that too yesterday, but figured it was a problem on my end as vscode(-insider) crashed several times.

Anyway, checked it just now:

• Opened vscode: devsense.php.ls gets started
• Waiting for indexing to stop, then closing vscode: devsense.php.ls is still running (doing not much other than using up memory)
• Opening vscode again: Another devsense.php.ls gets started

devsense Version: v1.34.13313

Version: 1.80.0-insider Commit: f3aa9a201b2455e6556797e8ffb7145d77adb792 Date: 2023-06-14T09:02:36.066Z Electron: 22.5.7 Chromium: 108.0.5359.215 Node.js: 16.17.1 V8: 10.8.168.25-electron.0 OS: Linux x64 5.15.0-73-generic

Cheers!

[jakubmisek]

@Songworks closing code should stop devsense.php.ls immediately. Sadly this issue is not happening on our test machines.

[marinaglancy]

We just had four people in our team test it. For three of us (Ubuntu, ubuntu, mac) the processes did not stop, for one person (Mac) it did.

[jakubmisek]

@marinaglancy thank you for the quick response. We did some fixes and we'll be doing more testing today. I expect to release a pre-release in the evening.

Weirdly, this is still not happening on our test machines.

[marinaglancy]

If it helps, I just reverted to v1.34.13120 and now the processes stop when I close vscode, so it's some recent regression

[jakubmisek]

@marinaglancy thank you, we have reverted a few related changes and testing. Preparing the update within 1-2 hours.

[jakubmisek]

There is release v1.35.13322, we'd appreciate any feedback.

It should fix the devsense.php.ls not being closed, and improves the memory usage.

[marinaglancy]

I'm sorry to report that it did not fix the issue for me. I tested many times, I double checked that I'm on the latest version on the extension, I even restarted my computer. I can still see all the processes accumulating and I also noticed that one more process is being added every time I change the workspace. I do not see a decrease in memory usage either.

[jakubmisek]

@marinaglancy thank you! we've found a case when it happens and the fix is on the way.

We'll keep working on the memory.

Version 1.35.13327 fixes closing devsense.php.ls processes.

[Songworks]

v1.35.13327 fixed it on my end. Thank you. The devsense.php.ls processes are terminated again when I close vscode(-insider).

vscode: Version: 1.80.0-insider Commit: f3aa9a201b2455e6556797e8ffb7145d77adb792 Date: 2023-06-14T09:02:36.066Z Electron: 22.5.7 Chromium: 108.0.5359.215 Node.js: 16.17.1 V8: 10.8.168.25-electron.0 OS: Linux x64 5.15.0-73-generic

[marinaglancy]

Thank you very much guys, v1.35.13327 fixed it for me too

[jakubmisek]

Thank you all for the quick feedback, and sorry for the inconvenience.

The bug was there for years, in the previous update it just started to happen more frequently.

Next, we'll be improving the RAM usage.

[jakubmisek]

Found the real memory leak, preparing an update today.

[jakubmisek]

Update - we have released an update which fixes the RAM usage in a few more edge cases.

Please try the latest version and let me know. From our testing it seems the problem has been resolved.

Also - we're preparing another update which cuts the RAM usage to absolute minimum (we have Laravel 10 in 39MB of RAM)

[oojacoboo]

@jakubmisek keep us posted on the next update that'll decrease RAM usage overall - will give this another go. Glad to see some progress here.

[Songworks]

Small update from my end: Memory usage of devsense.php.ls after vscode init went from several GB down to ~180MB currently!

No clue if or what else may have changed (workspace size didn't), but I'm happy about it.

Great success! Cheers!

[jakubmisek]

@Songworks thank you very much for letting us know!

Yes, we have implemented caching of /vendor/ folder, which now does not use much RAM. I'm happy the difference is so significant.

[jakubmisek]

I'm closing this issue as we have improved and fixed many related issues.

• we're caching composer packages now in a way that they consume just a few MBs
• memory leaks that were reported were fixed
• still recommending to exclude caches and unnecessary workspace folders using "files.exclude" setting - significant improvement!
• devsense.php.ls gets closed correctly now when closing/reloading VSCode.

Thank you all for your help!