init / split missing?

[ml31415]

I want to split a given repo into several subrepos. This means, I don't want to start with cloning another external repo into my repo, but I want to initially create a subrepo from a subdir, and push that to some new bare repo.

With git stree this was a common way to create a repo and looked like git stree split subreponame -P subdir code@remotehost:subreponame. How can I do the same with subrepo? I haven't seen any command to init a repo in such way.

[jawshooah]

Doesn't look like git-subrepo currently has a built-in way to do this, unfortunately. But you can use this StackOverflow answer as a guide, and just substitute git subrepo clone for git submodule add at the end. It would be nice to have a command to do this automatically, though.

[ml31415]

Thanks for the quick answer. This actually looks quite like what stree is doing, so I'll probably use that for initial setup.

[ml31415]

Is there any plan to integrate this in the near future? Looks like this is the only missing piece to replace stree completely, which is somewhat buggy and seems to be not really developed further anymore.

[ingydotnet]

Hi. Sorry, just seeing this. I'm not familiar with git-stree. Can you give me a url to that? Is it this: https://github.com/tdd/git-stree ?

[jawshooah]

@ingydotnet Yes, that's the one, and I believe that this is the command @ml31415 would like to see implemented here.

[ingydotnet]

I can see how that would be a nice addition.

You want to leave your repo structure the same but turn subdirs into sharable subrepos.

I would probably call this subcommand init since it is starting a new repo. All the current subcommands have git command counterparts.

Does that make sense?

[ingydotnet]

Ha. I failed to see that "init" was already in the issue title. :\

[ml31415]

init, split, I'm fine with anything, as long as it's there :) I agree, that init is probably the better name choice than split. I hadn't seen another thread, so sorry if I missed out something there. Besides that, having init working would make my day!

[ml31415]

To add some incentives to get this fixed, I'd sponsor a bugbounty of 150$ for this.

[ingydotnet]

@ml31415 OK I'll bite. :)

[ml31415]

Awesome! Paypal or elance for payment?

[ingydotnet]

@ml31415 We can figure that out later.

See this commit: https://github.com/ingydotnet/git-subrepo/commit/774a09d7dfc0

And read the new doc section: http://git.io/vqkDR

Let me know if this seems like the right plan so far.

[ml31415]

Looks great!

[jawshooah]

Awesome! Glad to see this is getting implemented :+1:

On Fri, Jul 3, 2015 at 11:46 AM Michael notifications@github.com wrote:

Looks great!

— Reply to this email directly or view it on GitHub https://github.com/ingydotnet/git-subrepo/issues/79#issuecomment-118378585 .

[ingydotnet]

OK. Proceeding :)

I actually need to relearn this codebase a bit. After that it should be fairly straightforward.

[ingydotnet]

The init branch is now complete and ready for testing.

Please hammer on it. I'm sure there's a bug or 2 in there.

[ingydotnet]

Pushed a couple more commits to fix --remote and --branch usage on an init command.

[ml31415]

I'll try it out tomorrow!

[ml31415]

$ git subrepo init sources/ -r code@rhode:sources.git
Subrepo created from 'sources' with remote 'code@rhode:sources.git' (master).
$ git subrepo push sources
git-subrepo: Command failed: 'git fetch --quiet code@rhode:sources.git master'.

Shouldn't init alreay push some stuff to the remote? At least, the bare repo remained empty after both commands.

[ingydotnet]

Hmm. I'll play with how the init dance works with pushing content.

I lean towards not having init do anything with the upstream except set the url if provided. But if you have created an empty repo to push to then subrepo push should certainly work.

Stay tuned. (And thanks for testing).

[ml31415]

Can I help with debugging somehow?

[ingydotnet]

@ml31415 I probably won't get back to this for a day or so. Feel free to start a discussion on how you would like it to work in terms of commands entered and what happens. We should also decide what kind of errors can happen and what should be done.

I don't want init to be too magical. But it can be informative in telling the user what to do next, etc.

[ml31415]

I'm fine with init both ways, as long as the doc is clear. Just the first push should work without problems.

[ingydotnet]

The latest push to the init branch makes things work as far as I can tell. The changes were all to the push command. It needs to deal with pushing to a remote or branch that is not yet populated. This was probably needed anyway for pushing to a new branch of an existing repo.

I wrote a script to test this that can be run over and over: https://gist.github.com/1c0f1199097d0c1de830

The output of a run is here: https://gist.github.com/c92772afd74d5850ccf4

The script uses ingydotnet/git-hub command, which is handy for scripting this stuff. (Making and deleting repos on the fly).

@ml31415 please try it out and try some possible common mistakes, and report back.

[ml31415]

Hi ingy! Thanks for the continued efforts! The only thing I stumbled upon so far is this, when I tried to update the remotes of a subrepo. Besides that, everything seems to work so far. I'll do some more testing later today.

$ git subrepo push sources --remote code@rhode:subrepo/sources.git --update
git-subrepo: Can't use '--update' without '--branch' or '--remote'.

[ml31415]

Some more issues. Now I have some modifications that I want to push. Rebase fails for some reason:

$ git subrepo push sources
The 'git rebase' command failed during your push.
You will need to finish the push by hand. Follow these steps:

  # Resolve the rebase conflicts
  git checkout master
  git subrepo push sources subrepo-push/sources

To abort the push and reset back to where you started:

  git rebase --abort
  git checkout ORIG_HEAD
  git subrepo clean sources

See 'git help subrepo' for more help.

So I resolve the conflicts, readd the files, rebase --continue, checkout master, but the push still fails. Is there a way, to remove any traces of subrepo completely, so I can be sure, that I have a fresh start?

Update: Removing .gitrepo and reinitializing the subrepo helped.

[ingydotnet]

@ml31415,

re: the --update bug, I have recreated, but want to get regression tests in place before I push a fix. There's a lot of moving parts and I want to make sure we keep everything intact.

re pull conflict resolution, if you could send me a replication strategy on a public repo, I could detail how to do it for you. Conflict resolution is one of the harder parts of using git, and then doing it within subrepo adds a level of mystery to the new user. I'd like to get out some solid docs on how to do it. Even I have to put on my thinking cap when I do it.

One thing to use is the -v and -d flags that show you exactly what is happening under the hood. And -x if you want it line for line.

The basic parts of a push are this:

• Fetch the remote history
• Make a branch called subrepo/<subdir> (for the subrepo called <subdir>)
  • Contains only the commits to the subdir since last push
• git rebase FETCH_HEAD subrepo/<subdir>
  • Now you will have the subrepo/<subdir> branch checked out
  • That's how rebase with 2 args works
• git checkout master # or wheverever you started
• Push the subrepo/<subdir> branch to the subrepo's remote

When the rebase fails you are left inside that subrepo branch, to resolve conflicts.

Anyway, please send me the url of a repo that you can't resolve and I'll see if I can do it, then tell you.

[ml31415]

Unfortunately I can't reproduce the error myself anymore right now. Reinitializing the subrepo worked. This is the diff, but I'm afraid it won't be of any help:

diff --git c/sources/.gitrepo w/sources/.gitrepo
index 4779349..718561f 100644
--- c/sources/.gitrepo
+++ w/sources/.gitrepo
@@ -6,6 +6,6 @@
 [subrepo]
        remote = code@vpnrhode:sources.git
        branch = master
-       commit = a2a6543c94f364b88ff05dfd36976b48eda53be7
-       parent = fde13b484374e146635675342bab71d570bd982b
+       commit = 3a5ed7079b5b659893b91dfd5681723fcbca79b9
+       parent = c65f758259a78fa6d525f9e3309ad23e650b2741
        cmdver = 0.2.2

Unfortunately it's some closed stuff I can't give out. I hope in case I run into it next time, I can create a reproducer!

[ingydotnet]

Just pushed to init branch. Added tests for pushing to an empty repo/branch and also for push --update. It turns out that --update wasn't even being triggered on push. :\

@ml31415, can you test once more. I can't think of anything more that needs done here. I'll re-read the docs, and hopefully we can merge this to master and make a release tag soon.

[ml31415]

Sure, give me two days. Back to work on monday again.

[ingydotnet]

@ml31415 I'm going to merge the init branch back into master now. I'll leave the issue open until you sign off, then I'll make a release tag.

[ml31415]

For me everything is allright. No issues so far anymore. Thanks again for the quick implementation of this! This is for me the first well working subrepo / submodule / stree workflow with git!

[ingydotnet]

@ml31415 great! Thanks for being the catalyst. My attention seems to wax and wane between a lot of projects. Maybe now that subrepo is loaded into my RAM, it's a good time to rifle through the issues that have built up.

One thing I should probably do is write up a full comparison between submodule, subtree, subrepo and stree. I'll add an issue to do that.

[ml31415]

Well, let's sum it up: All other implementations got enough issues to switch over to subrepo ;)