Ubuntu 20.04 and 22.04 Defaults lead to repeat of #9120

13steinj commented 1 year ago

`brew doctor` output

└─[$] <> brew doctor
Your system is ready to brew.

Verification

[X] My "brew doctor output" above says Your system is ready to brew. and am still able to reproduce my issue.
[ ] I ran brew update twice and am still able to reproduce my issue.
[X] This issue's title and/or description do not reference a single formula e.g. brew install wget. If they do, open an issue at https://github.com/Homebrew/homebrew-core/issues/new/choose instead.

`brew config` output

└─[$] <> brew config
HOMEBREW_VERSION: 4.1.3
ORIGIN: https://github.com/Homebrew/brew
HEAD: 128df14e0afd0525564051a7665708c2973a3cab
Last commit: 30 hours ago
Core tap JSON: 01 Aug 19:38 UTC
HOMEBREW_PREFIX: /home/linuxbrew/.linuxbrew
HOMEBREW_CASK_OPTS: []
HOMEBREW_DISPLAY: :100
HOMEBREW_EDITOR: nano
HOMEBREW_MAKE_JOBS: 8
Homebrew Ruby: 2.6.10 => /home/linuxbrew/.linuxbrew/Homebrew/Library/Homebrew/vendor/portable-ruby/2.6.10_1/bin/ruby
CPU: octa-core 64-bit skylake
Clang: N/A
Git: 2.41.0 => /home/linuxbrew/.linuxbrew/bin/git
Curl: 7.68.0 => /bin/curl
Kernel: Linux 5.15.0-76-generic x86_64 GNU/Linux
OS: Ubuntu 20.04.6 LTS (focal)
Host glibc: 2.31
/usr/bin/gcc: 9.4.0
/usr/bin/ruby: 2.7.0
glibc: N/A
gcc@11: N/A
gcc: N/A
xorg: N/A

What were you trying to do (and why)?

Update my brew packages, and then cleanup I use OMZ, so I have an alias.

└─[$] <> als bubc
[brew]
    bubc = brew upgrade && brew cleanup

What happened (include all command output)?

I can't do this now, because I've lost that terminal.

However, same thing as in #9120.

What did you expect to happen?

Not to get an error.

See #9120

cc @MikeMcQuaid

1024 might be low but note, this is default for any VM in AWS (and other VM's IIRC), so is likely to impact quite a few people (plus, it's always nice to lower resource usage either way).

Will reopen this if we see it affect other people.

...Well, I can't comment on that issue since the bot limited conversation one month later. So I can't even ask for a re-open without making this new issue.

1024 is the default on new boxes from Dell (Precision 3460) with whatever their custom ubuntu default install is (they appear to add some custom software for recovery purposes as well as drivers out of the box).

I can understand the inclination to say "just increase the limit", but you'd be asking people to change defaults they have because (presumably) homebrew is doing something funky (I don't have this issue with any other application).

Frankly, I don't think it's too much to ask for brew to close file handles once done with the file...presumably, the issue is (psuedo code):

def operation(filenames, subop_for_files):
    files = [open(fn, "w") for fn in filenames]
    for f in files:
        subop_for_files[f.name](f)
    [_.close() for _ in files]

Rather than opening and closing the files more aggressively (right before and after the relevant operation).

Note: I'm lying when I checked off "I ran brew update twice and was still able to reproduce", this doesn't happen on a second-run unless I wait a few days and then have a list of packages that would be upgraded again.

Step-by-step reproduction instructions (by running `brew` commands)

brew upgrade && brew cleanup

I don't know which of the two commands is the culprit, unfortunately, as brew cleanup has no output on my system, and won't be able to tell you until a week or two passes and enough formulae get updates.

Bo98 commented 1 year ago

I can perhaps see if I can see anything obvious, but more information would be useful, such as narrowing down the specific command and ideally posting the complete output of the command (so we know, for example, how many packages being upgraded at once are we talking about and whether the dep trees of them make a difference).

We're not intentionally keep files open so it's not an easy case of "just close them please" - it's potentially a bug that will need tracking down, but it's hard to know where exactly to look from this.

13steinj commented 1 year ago

I will update this issue the next time this happens, and refrain from updating for a week or two to make sure it re-occurs. I can narrow down the command (cleanup vs upgrade). Apologies for the lack of info, I accidentally closed the wrong terminal when I had too many open.

If you'd like I can also strace it for file-based events, but that might pollute the log.

We're not intentionally keep files open so it's not an easy case of "just close them please" - it's potentially a bug that will need tracking down, but it's hard to know where exactly to look from this.

Please forgive me if my commentary in the original issue came off as pedantic / condescending; that was not my intent. I fully understand that this kind of thing would need to be tracked down and isn't easy, as the original reporter of #9120 did not make a PR AFAIK (and I don't know the codebase, let alone have recent experience with Ruby).

MikeMcQuaid commented 1 year ago

I can't do this now, because I've lost that terminal.

Please supply this next time it happens, thanks.

Frankly, I don't think it's too much to ask for brew to close file handles once done with the file...presumably, the issue is (psuedo code):

Diplomatically, if it were that easy to solve the problem: it would have been done long ago. We specifically lint for and use the Ruby structure that avoids leaving dangling file handles like this. It's very hard to figure out whether this is Homebrew itself, something Homebrew calls on your system, something Homebrew is installing, etc.

did not make a PR AFAIK (and I don't know the codebase, let alone have recent experience with Ruby).

Almost no Homebrew contributors or maintainers knew Ruby before they first contributed. Ultimately, this is open-source software maintained by volunteers and supported by a community of users. If an issue affects a small number of people infrequently: the most likely way it's going to get fixed is for someone affected by the problem (e.g. you) to fix it.

I can understand the inclination to say "just increase the limit", but you'd be asking people to change defaults they have because (presumably) homebrew is doing something funky (I don't have this issue with any other application).

Yes. Regardless, I'm afraid this is not reproducible in any sense as-is so: closing this out. Saying "increase the limit" is not because we think it's the best solution but, in the absence of (much) more information and reports to help us resolve this: it's a workaround that will work for you today when the likelihood of this being fixed any time soon is effectively zero (hence: I'm closing this issue out but we can continue to talk here).

Homebrew / brew