REQ: Workflow best practices

[xxxserxxx]

There are a couple of ways in which I keep biting myself in the butt with jj in what seem irrecoverable ways, and I wonder if it's just a mental block that I'm never going to get around, or if there are things I can do to fix it when I get stuck, or if there are ways of configuring jj so that I prevent getting into these situations.

Number 1 is autocommits. Work work work, describe, push! Go off and do something else, come back, make some more changes, and ... oops! Now those changes are mingled with my other, pushed commit. With auto-magic commits, this is so easy, and it's such a common mistake that I make, and the number one cause of "take a diff, blow away the repo, and check it out fresh." It's most painful when I haven't yet pushed the changes upstream. I'm thinking I also need to alias "push" to "push && jj new". If there were some micro-history I could access to shelve and undo the past N minutes of changes. I know I can be stupid, but this seems like such a common "gotcha" that I feel like I'm missing some obvious correction. Split is a work-around if you happen to have cleanly separated changes, or haven't donet too much, or haven't modified already-modified code.

Number 2 is Meld as a tool. It's OK, but it's also really easy to accidentally tell jj you're done with your split when you aren't, and now you have a mangled split that you can hopefully back out and retry.

Number 3 is conflicts. I suspect this one is just a conceptual one, conflicts seem unnecessarily difficult to resolve. More than once I've felt like I was in a game of whack-a-mole, where I'd fix a conflict only to have in squeezed into a different commit.... which I'd then chase around the history for a while befole throwing up my hands, blowing the repo away, re-cloning in git and just doing whatever it was there. I am genuinely terrified of merges in jj, and haven't had much luck with rebases, either.

Number 4 is ... how do you fork from a local jj repo? I need to try something that I know is going to go horribly awry, leaving me with a mangled repository; in git or hg, I'd just make a clone, mess around in that, and delete it when I'm done. But jj has no clone command, and jj git clone doesn't work on jj repositories. Is it possible? Why is it so hard?

Most of these things are a form of recognition that jj feels like a foot-gun. It iis very easy to make mistakes, and requires experience with jj (and, frequently, with deep understanding of the underlying .git repo) to undo the mistakes. Some of these hurdles I attribute to my being a bear of little brain; after all, there are a couple thousand stars on the repo, so people must be being productive with it. And some of it's just that the project is relatively young; heck, it doesn't have tagging yet!

Still I must be missing some basic things. I know that I will forget "new" every time after I'm done -- how do I prevent my publishable commit from being polluted accidentally? That's got to be something people do accidentally all the time, right?

[xelxebar]

1. Ah, so you're using push as a way to sync your local working copy? What about committing the changes first, e.g. jj commit -m 'WIP: ...'? Then you can jj squash any additional changes you need. And if you end up working on a different issue, it's easy enough to rebase your working copy onto a different branch or whatever.

2. Are you aware of jj workspace? It's a lighter weight alternative to full-blown local checkouts.

[martinvonz]

Thanks for the feedback!

what seem irrecoverable ways

A common theme in your report is that you seem to have not found jj undo :) That undoes the most recent operation (or an earlier operation). There's also jj op restore to restore the whole repo to how it looked earlier. Use jj op log to figure out which operation you want to restore to.

Number 1 is autocommits. Work work work, describe, push! Go off and do something else, come back, make some more changes, and ... oops!

Yes, I completely agree! I typically either create a new commit on top (jj new) or check out the main branch (jj co main) when I'm done actively working on a change.

"take a diff, blow away the repo, and check it out fresh."

jj undo (or jj op restore) also includes snapshotting of the working copy, so you can undo that snapshotting. Maybe we should also have an easy way of separating out the recent changes somehow.

I'm thinking I also need to alias "push" to "push && jj new"

FYI, you can also do jj new && jj git push; jj git push will push the parent branch if there isn't a branch pointing to the working-copy commit.

Number 2 is Meld as a tool. It's OK, but it's also really easy to accidentally tell jj you're done with your split when you aren't, and now you have a mangled split that you can hopefully back out and retry.

Again, the "back out" bit is jj undo. What would you prefer over Meld? We have #48 about adding a TUI to use instead of Meld. There's still going to be a risk of exiting before you're done, of course. If you do, you can use move/squash -i/unsquash -i to adjust.

More than once I've felt like I was in a game of whack-a-mole, where I'd fix a conflict only to have in squeezed into a different commit....

Was it you I talked to in some other thread about the order to resolve conflicts? Make sure you resolve conflicts in parent commits before resolving them in children.

which I'd then chase around the history for a while befole throwing up my hands, blowing the repo away, re-cloning in git and just doing whatever it was there.

jj op restore should have worked here too.

I am genuinely terrified of merges in jj, and haven't had much luck with rebases, either.

😞 Please file a bug report or start a discussion when you have a concrete example so we can understand better and/or explain better.

Number 4 is ... how do you fork from a local jj repo?

That's not supported yet, but as @xelxebar said, consider using jj workspace (which is like git worktree).

I need to try something that I know is going to go horribly awry, leaving me with a mangled repository

Again, jj op restore. However, that assumes that you haven't made other changes that you do want to keep in the meantime. If you need a clone for that reason, you can of course simply cp -r the whole repo.

requires experience with jj (and, frequently, with deep understanding of the underlying .git repo) to undo the mistakes

I probably don't need to say it anymore than I already have... 😄

[xxxserxxx]

Thank you for responding. This is helpful. As I've said before, at the moment most of my use of jj is because I'm providing patches to other people's projects. Because these are in github, I have to use git, and consequently, I'm trying to use jj. But I think this is one of the more difficult cases because there's a lot of: creating branches off main for patches, PRs, merging when the patches are merged, and using remotes. I understand how to do this with git (and moreso with hg), but I keep stumbling with jj.

@xelxebar

1. Ah, so you're using push as a way to sync your local working copy? What about committing the changes first, e.g. jj commit -m 'WIP: ...'? Then you can jj squash any additional changes you need. And if you end up working on a different issue, it's easy enough to rebase your working copy onto a different branch or whatever.

Is there a difference between new and commit? The look the same.

I do think I was taking auto-commits for granted, and thinking about commits like hg or git -- making specific checkpoints -- conceptually easier for me. "Commit" implies I'm more or less done with something -- it's done at the end. "New" is starting something new, and is done at the beginning. Even if they effectively do the same thing, the former is far more intuitive to me than the latter, and I suspect trying to force myself into thinking like the latter was much of the issue.

2. Are you aware of `jj workspace`? It's a lighter weight alternative to full-blown local checkouts.

Peripherally; I don't know how to work with them, nor have I tried. I was trying to get used to the base case. Lightweight how? The basic commands for committing etc. are the same, right?

@martinvonz

Thanks for the feedback! Yeah, I keep posting these "help me!" tickets. I'm sorry.

A common theme in your report is that you seem to have not found jj undo :)

Oh. My. God. That doesn't do what I thought it did, and -- yes! It would address my most common mistake: doing something before remembering that I have to (a) do it on a branch, so trying to branch retroactively, or (b) making changes before starting a new commit. Well, it might not help with the branching issue, but certainly the second thing.

That undoes the most recent operation (or an earlier operation). There's also jj op restore to restore the whole repo to how it looked earlier. Use jj op log to figure out which operation you want to restore to.

Yeah, that will be a big help, too. Being able to see the snapshots is what I was missing.

jj undo (or jj op restore) also includes snapshotting of the working copy, so you can undo that snapshotting. Maybe we should also have an easy way of separating out the recent changes somehow.

I wasn't aware of jj op. That's exactly what I was looking for, I just didn't know it.

I may publish some PRs for the tutorial, which covers the differences from git and the "best case" workflow, and could use some fleshing out in the "what to do when you screw up" area. I'm very good at screwing up my commit history.

FYI, you can also do jj new && jj git push; jj git push will push the parent branch if there isn't a branch pointing to the working-copy commit.

I suspect that "new" was tripping me up. It's a prepatory action, whereas a completion action is more intuitive. There may be little difference between commit and new, but it dramatically changes the conceptual model (for me). commit isn't in the normal jj help, so I didn't realize it exists!

Number 2 is Meld as a tool. It's OK, but it's also really easy to accidentally tell jj you're done with your split when you aren't, and now you have a mangled split that you can hopefully back out and retry.

Again, the "back out" bit is jj undo. What would you prefer over Meld? We have #48 about adding a TUI to use instead of Meld. There's still going to be a risk of exiting before you're done, of course. If you do, you can use move/squash -i/unsquash -i to adjust.

I am not arguing to change the merge tool. I think meld is a good choice, and probably the best option; IIRC kdiff3 was a bit better, but meld is a fine tool. Having a terminal-friendly option would be fantastic, though. I'm very fond of vim's dirdiff, mainly because vim is everywhere and it's remote-terminal-friendly.

I really like jj taking inspiration from darcs. I really, really missed darcs' cherry-picking; somehow, that interface was both primitive and entirely intuitive, and I recall it being almost painful to give it up when, well, the world moved on.

More than once I've felt like I was in a game of whack-a-mole, where I'd fix a conflict only to have in squeezed into a different commit....

Was it you I talked to in some other thread about the order to resolve conflicts? Make sure you resolve conflicts in parent commits before resolving them in children.

Yeah, I am in a loop. Normally, I'd make some repos, play with jj until I was comfortable, and then use it. Or, maybe use it on one of my own projects. But as I said at the top, I'm jumping right into trying to use it for collaborating, and the hardest activity in VCS is branching and merging. So I recognize I'm learning to swim by jumping off the cruise ship; I just keep feeling like I'm missing some core concept that is making working with jj harder for me, but I still try to use it to provide patches to these other projects.

jj op restore should have worked here too.

Yup, it would have. Now that I know about it, I think fixing things will be much easier.

I am genuinely terrified of merges in jj, and haven't had much luck with rebases, either.

disappointed Please file a bug report or start a discussion when you have a concrete example so we can understand better and/or explain better.

I'm mainly concerned about the behavior I've seen a couple of times where I'm unable to resolve conflicts. That is, I believe I've addressed the conflicts in the files and have commit the changes, but the conflict gets pushed back to some earlier commit. I actually had one time where I merged a branch with one marked conflict at the head; I fixed the conflicts and committed the change, and then every commit on the merged branch showed up as having a conflict marker. Most frustratingly, even though the HEAD was clean, jj refused to push because conflicts remained. I had no idea how or where to start to resolve that.

I feel, though, as if this is a result of me not using the right commands or workflow to resolve conflicts. I may try to replicate it, because having documentation about how to address non-trivial merge conflicts would be helpful.

Number 4 is ... how do you fork from a local jj repo?

That's not supported yet, but as @xelxebar said, consider using jj workspace (which is like git worktree).

Is there any documentation beyond the embedded help text? I'm not sure where to start with the command, and I don't see it in the tutorial.

If you need a clone for that reason, you can of course simply cp -r the whole repo.

Is this the recommended way of forking? I wanted to get my (committed) work onto another machine, and without a clone I resorted to an rsync. Was this the expected way?

[martinvonz]

(a) do it on a branch, so trying to branch retroactively

Do you mean that you forgot to create a branch before you started working? You can run jj branch create at any time, so that shouldn't be a problem (unlike Git, commits don't need to have branches pointing to them to prevent GC). Or do you mean that you started working off of the wrong commit? Use jj rebase to fix that.

I may publish some PRs for the tutorial

Thanks! I know the documentation needs a lot of work. I sent #1380 because I noticed how extremely little documentation there was of jj op commands as I was replying to this thread. More documentation is very much appreciated.

Is there any documentation beyond the embedded help text? I'm not sure where to start with the command, and I don't see it in the tutorial.

There's a bit at https://github.com/martinvonz/jj/blob/main/docs/working-copy.md#workspaces. We should at least link there from the embedded help text.

If you need a clone for that reason, you can of course simply cp -r the whole repo.

Is this the recommended way of forking? I wanted to get my (committed) work onto another machine, and without a clone I resorted to an rsync. Was this the expected way?

Yes, that's what I'd recommend for now.