bug: High memory and CPU usage in editor in bun with some specific imports

[jimjimovich]

Describe the bug

I upgraded to the latest build yesterday from version CE v1.272.0-9-gf17e8bc15 to the latest and again today to v1.280.0. After upgrading, it's impossible to work in the web interface because it instantly uses over 5GB of RAM and maxes out at least 3 cores. If I try to work in it, it'll consume all 32 GB of system RAM and 32 GB of swap space and crash the computer.

To reproduce

1. Install the latest version via Docker Compose.
2. Open the site in your browser.

Expected behavior

Much better performance and no memory leaks

Screenshots

Before opening Windmill in the browser, it was using almost no CPU and less than 5GB of RAM. After just a few clicks in Windmill, this was the system resources.

Browser information

Firefox 123.0, Brave v 1.63.162 on Fedora

Application version

v1.280.0

Additional Context

No response

[rubenfiszel]

Are you absolutely certain this is coming from the Windmill interface ? Can you reproduce on app.windmill.dev (which is on latest). Does it happen on any page?

[rubenfiszel]

I cannot reproduce myself, and I have many many windmill tabs open :) Would appreciate if you can give more precise reproduction steps. I have doubt that there is any memory increase in between 1.272 and 1.280 on the client side but you never know

[jimjimovich]

Yes, I'm absolutely sure that it's the Windmill browser tab. When I kill the offending process, only that tab is killed. It happens within seconds with only one Windmill tab open.

I'm also getting a lot of this kind of thing in the console.

The problem starts with 1.279.0, 1.278.5 is working fine.

[rubenfiszel]

Those request failures are normal, just our frontend editor trying to download types for the editor assistant

The only thing changing between those 2 versions are the wasm used for parsing types. Would suggest clearing your cache just in case.

[jimjimovich]

Okay, you are correct about the versions. It's something specific to the script I'm working on. I'll try to figure out exactly what it is that is setting off the memory and CPU issues. It seems to be related to importing a library in Bun.

[rubenfiszel]

If you can share your imports, it's not impossible that the bun frontend assistant is causing this by trying to import too many deps which I can easily reproduce by having the same imports.

It's a known possible issue that we need to fix.

[jimjimovich]

I've narrowed it down to these two imports

import { OpenAIEmbeddings } from "@langchain/openai";
import { PGVectorStore } from "@langchain/community/vectorstores/pgvector";

The second I paste them into the Windmill editor, my memory usage starts going up by about 100MB/s.

They are from this documentation. https://js.langchain.com/docs/integrations/vectorstores/pgvector

It does mention installing @langchain/openai @langchain/community

So far, everything I've done has been auto imported without issues. Maybe there's something I need to add to clarify the packages that need imported? I suppose it's possible that these are not even compatible with Bun.

[rubenfiszel]

It's not related to Bun, we are probably among the very few project to attempt fetching types directly from browser to feed monaco frontend typescript assistant. The way we do it is a bit naive and even though there are some limits to our auto-discovery, there are some edge-cases that can end up like this. This will help us adjusting the limits of our type discovery and we should be able to fix this soon.

[rubenfiszel]

Should be greatly improved in latest release

[jimjimovich]

I just tested v1.281.3 and it's working so much better! Thank you so much!