Git branch previewer acts like a fork bomb (causes out of memory)

[jekhor]

Description

When walking through branch list, git_branch_log previewer tries to show commit log for ENTIRE branch history. In a big project (e.g. Linux kernel) this spawns a lot of git processes consuming all your available memory and IO. I have triggered OOM killer on system with 96GiB of RAM just by walking on linux kernel remote branches :)

Git options that cause this can be found here: https://github.com/nvim-telescope/telescope.nvim/blob/b744cf59752aaa01561afb4223006de26f3836fd/lua/telescope/previewers/buffer_previewer.lua#L800

It may make sense to limit history depth in this previewer to avoid such situations.

Neovim version

NVIM v0.9.5
Build type: Release
LuaJIT 2.1.1703358377

Operating system and version

Debian sid

Telescope version / branch / rev

telescope 0.1.5

checkhealth telescope

3 telescope: require("telescope.health").check()                                   
   4                                                                                  
   5 Checking for required plugins                                                    
   6 - OK plenary installed.                                                          
   7 - OK nvim-treesitter installed.                                                  
   8                                                                                  
   9 Checking external dependencies                                                   
  10 - OK rg: found ripgrep 14.1.0                                                    
  11 - WARNING fd: not found. Install [sharkdp/fd](https://github.com/sharkdp/fd) for extended capabilities
  12                                                                                  
  13 ===== Installed extensions =====                                                 
  14                                                                                  
  15 Telescope Extension: `aerial`                                                    
  16 - No healthcheck provided                                                        
  17                                                                                  
  18 Telescope Extension: `fzf`                                                       
  19 - OK lib working as expected                                                     
  20 - OK file_sorter correctly configured                                            
  21 - OK generic_sorter correctly configured                                         
  22                                                                                  
  23 Telescope Extension: `notify`                                                    
  24 - No healthcheck provided                                                        
  25

Steps to reproduce

1. Clone the linux kernel (https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git) repo
2. Run :Telescope git_branches command
3. Quickly walk through branches list by 'Down' key

Expected behavior

Cusor moves, previewer shows log fragment.

Actual behavior

Many git processes consume all available RAM

Minimal config

vim.cmd [[set runtimepath=$VIMRUNTIME]]
vim.cmd [[set packpath=/tmp/nvim/site]]
local package_root = '/tmp/nvim/site/pack'
local install_path = package_root .. '/packer/start/packer.nvim'
local function load_plugins()
  require('packer').startup {
    {
      'wbthomason/packer.nvim',
      {
        'nvim-telescope/telescope.nvim',
        requires = {
          'nvim-lua/plenary.nvim',
          { 'nvim-telescope/telescope-fzf-native.nvim', run = 'make' },
        },
      },
      -- ADD PLUGINS THAT ARE _NECESSARY_ FOR REPRODUCING THE ISSUE
    },
    config = {
      package_root = package_root,
      compile_path = install_path .. '/plugin/packer_compiled.lua',
      display = { non_interactive = true },
    },
  }
end
_G.load_config = function()
  require('telescope').setup()
  require('telescope').load_extension('fzf')
  -- ADD INIT.LUA SETTINGS THAT ARE _NECESSARY_ FOR REPRODUCING THE ISSUE
end
if vim.fn.isdirectory(install_path) == 0 then
  print("Installing Telescope and dependencies.")
  vim.fn.system { 'git', 'clone', '--depth=1', 'https://github.com/wbthomason/packer.nvim', install_path }
end
load_plugins()
require('packer').sync()
vim.cmd [[autocmd User PackerComplete ++once echo "Ready!" | lua load_config()]]

[jamestrew]

I think for large code bases like this, having a custom previewer is probably best.

require("telescope").setup({
  pickers = {
    git_branches = {
      previewer = require("telescope.previewers").new_termopen_previewer({
        get_command = function(entry)
          return {
            "git",
            "log",
            -- "--graph", can be waaayy too expensive
            "--pretty=format:%Cred%h%Creset -%C(yellow)%d%Creset %s %Cgreen(%cr)%Creset",
            "--abbrev-commit",
            "--date=relative",
            entry.value,
          }
        end,
      }),
    },
  },
})

This lets us use a pager and avoid the --graph option while retaining some highlighting. The other approach of using something like --max-count=<number> doesn't seem to work with --graph.

I hope this helps and is satisfactory. I don't want to dig out and change the default previewer too much. I think generally, this isn't issue for most people. I'll keep an eye on this and maybe consider changing the default in the future.

[jekhor]

Hmm, using of pager will cause spawning of a new git log process every time when next branch is selected. They will wait until quit from the pager or until branch list is closed. Disabling of --graph and limiting of commit number is enough for me now.

btw, the default mapping for scrolling preview window down is overlapped by git_delete_branch here. Should this be reported as bug?

In any case, scrolling in the git branch log previewer doesn't work for me even after key remapping, with pager or without...

[jamestrew]

btw, the default mapping for scrolling preview window down is overlapped by git_delete_branch here. Should this be reported as bug?

I think this was just a design choice. Overriding it in your personal config seems like a reasonable choice.

As for scrolling, using new_termopen_previewer as in my snippet, I believe is intended for use with a pager. And scrolling is handled by the pager. And this works for me if I do bind the previewer_scrolling_down action.

But you can pass a custom scroll_fn to the previewer to achieve this. I think something like this should be close to checking all your requirements

git_branches = {
  mappings = {
    i = {
      ["<C-d>"] = "preview_scrolling_down",
    },
  },
  previewer = require("telescope.previewers").new_termopen_previewer({
    get_command = function(entry)
      return {
        "git",
        "--no-pager",
        "log",
        "--max-count",
        "150",
        "--pretty=format:%Cred%h%Creset -%C(yellow)%d%Creset %s %Cgreen(%cr)%Creset",
        "--abbrev-commit",
        "--date=relative",
        entry.value,
      }
    end,
    scroll_fn = function(self, direction)
      if not self.state then return end

      local input = direction > 0 and [[]] or [[]]
      local count = math.abs(direction)

      vim.api.nvim_buf_call(
        self.state.termopen_bufnr,
        function() vim.cmd([[normal! ]] .. count .. input) end
      )
    end,
  }),
},

[jekhor]

No, it doesn't work for me with pager enabled as well with pager disabled (and scroll function added). Default previewer works with scrolling. What are the strange symbols in line local input = direction > 0 and [[�]] or [[�]] in the your snippet?

[jamestrew]

Oh sorry those symbols are termcodes for <C-e> and <C-y> respectively. Seems to not render well in github.

You can get those insert <C-v><C-e> in insert mode. Or replace that line with

local input = vim.api.nvim_replace_termcodes(direction > 0 and "<C-e>" or "<C-y>", true, false, true)

[bradpols]

I've run into this issue on a massive repository (3.6 million commits and rising) just by opening the git branch previewer. Even without scrolling through the options, and even if I close the previewer, the git log process for the initial selected branch runs and runs and runs....and nvim's memory usage just grows and grows and grows. It was using over 17 GB of memory by the time I realized something was amiss.

It might be useful to have a configuration setting for a maximum number of commits or a maximum time frame for that git log call.

[jamestrew]

The other approach of using something like --max-count= doesn't seem to work with --graph.

I don't know how I originally came to this conclusion. Trying this now and I seems to work so I'll probably implement this.