search

twosigma / git-meta

Repository for the git-meta project -- build your own monorepo using Git submodules
http://twosigma.github.io/git-meta
BSD 3-Clause "New" or "Revised" License
218 stars 50 forks source link

`git meta merge --no-ff` is very very slow #691

Open jhedwardyang opened 5 years ago

jhedwardyang commented 5 years ago

Repro steps:

git clone https://github.com/bpeabody/big-mono.git 
cd big-mono
git checkout f132442efc3a326188f1c6afc3547a3fc0512a19
git meta merge 720f26ec535e80aec6ab1392037cf7e86eb984eb --no-ff -m 'merge'

Expected:

Actual:

shijinglu commented 5 years ago

it takes about the same time with or without half-open enabled:

without half-open module:

$ time git meta merge 720f26ec535e80aec6ab1392037cf7e86eb984eb --no-ff -m 'merge'

Merging meta-repo commit 720f26ec535e80aec6ab1392037cf7e86eb984eb.

real 0m3.939s user 0m2.133s sys 0m0.875s

with half-open module:

$ time git meta merge 720f26ec535e80aec6ab1392037cf7e86eb984eb --no-ff -m 'merge'

Merging meta-repo commits f132442efc3a326188f1c6afc3547a3fc0512a19 and 720f26ec535e80aec6ab1392037cf7e86eb984eb Merge commit created at 2f876012e0c856430f8cfc8855b49da298a1ddc2.

real 0m2.212s user 0m1.657s sys 0m0.599s

The reason is that it takes seconds to merge is because this repo has >6000 submodules and git-meta process iterate over all submodules twice during the merge. One for simple changes like adding, renaming or deleting submodules and the other iteration is for complicated merge conflicts if there are any.

motlin commented 5 years ago

Does it iterate over unchanged modules? If it doesn't already contain the optimization, it seems like merge ought to skip submodules where the submodule sha is the same in both meta commits.

On Thu, Feb 14, 2019 at 7:13 PM shijing notifications@github.com wrote:

it takes about the same time with or without half-open enabled: without half-open module:

$ time git meta merge 720f26ec535e80aec6ab1392037cf7e86eb984eb --no-ff -m 'merge'

Merging meta-repo commit 720f26ec535e80aec6ab1392037cf7e86eb984eb.

real 0m3.939s user 0m2.133s sys 0m0.875s

with half-open module:

$ time git meta merge 720f26ec535e80aec6ab1392037cf7e86eb984eb --no-ff -m 'merge'

Merging meta-repo commits f132442efc3a326188f1c6afc3547a3fc0512a19 and 720f26ec535e80aec6ab1392037cf7e86eb984eb Merge commit created at 2f876012e0c856430f8cfc8855b49da298a1ddc2.

real 0m2.212s user 0m1.657s sys 0m0.599s

The reason is that it takes seconds to merge is because this repo has

6000 submodules and git-meta process iterate over all submodules twice during the merge. One for simple changes like adding, renaming or deleting submodules and the other iteration is for complicated merge conflicts if there are any.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/twosigma/git-meta/issues/691#issuecomment-463855294, or mute the thread https://github.com/notifications/unsubscribe-auth/AAO6IulUQLdn-McHIfQrpNJOxrptvgbyks5vNfuYgaJpZM4a8CtH .