sst dev not working: too many open files in system

[iOSonntag]

I managed to migrate my mega project from sst v2 to v3.

First of all I want to say thanks, that this actually worked lol. prod, stage, and dev environments are up and running working with the new v3 version containing multiple apps, websites, apis and more.

But there is a catch, I cant live dev on local stage anymore. The problem is that the sst dev command crashes immediately with the following error:

...
level=INFO msg="done mosaic" err="\"/Users/XXX/development/Repositories/XXX/apps/XXX/ios/Pods/FirebaseInstallations\": too many open files in system"
level=ERROR msg="exited with error" err="\"/Users/XXX/development/Repositories/XXX/apps/XXX/ios/Pods/FirebaseInstallations\": too many open files in system"

I have no clue what is going on here but I guess the watch process that responds to file changes watches every single file in my VSCode project and therefore also watching files in the ios folder of a flutter project (lol) which have nothing to do with the local sst dev build (even no static site build commands or so).

This is as I said a mega project which used to work fine with sst v2 - all I did was migrating the constructs and some api code.

Help appreciated

[iOSonntag]

I think I found the problematic part in the sst framework.

In the file ion/cmd/mosaic/watcher/watcher.go there is this part:

func Start(ctx context.Context, root string) error {
    defer slog.Info("watcher done")
    slog.Info("starting watcher", "root", root)
    watcher, err := fsnotify.NewWatcher()
    if err != nil {
        return err
    }
    err = watcher.AddWith(root)
    if err != nil {
        return err
    }
    ignoreSubstrings := []string{"node_modules"}
    ...

I have no clue about go lang but I guess there should be some more options then to not only exclude node_modules here, maybe via some sst config?

[iOSonntag]

For anyone facing this issue while SST looks for a fix. You can prefix the folders with a period "." in order for them to be ignored by the sst file watcher so we have:

apps/
  .flutterAppA
  .flutterAppB
  someWebsite
  apiA
  apiB

Its not perfect but it works

[iOSonntag]

Update: seems like this is not a valid solution with the period because the dart analysis server fails to analyze those folders with no way of configurating this behaviour.

I guess we will have to wait for a fix from SST

[iOSonntag]

Update we finally found a fix but it involves a lot of restructuring of the project. We have now the following project structure:

flutter/
  appA/
  appB/
sst/
  apps/
  packages/
  infra/
  package.json
  pnpm-lock.yaml
  pnpm-workspace.yaml
  sst.congig.ts

So we shifted the entire project under sst and all none node projects into the root of the repo.

Note that this involves a lot of changes in the ci/cd scripts, also make sure to add the following in the launch configuration:

{
  "name": "Debug SST Live",
  "cwd": "${workspaceFolder}/sst", // this one
  ...
}

[jayair]

Oh this is good debugging. Thanks.