Forge with incorrect user/project name capitalization

[jdtsmith]

I could not get forge to create a PR or list topics, etc. at a repository owned by an organization of which I'm the member. No matter what I tried, forge commands set to the `upstream' repository lead to the error:

Cannot use ‘forge-list-topics’ in "/Users/jdsmith/code/python/pahfit/" yet.
Use `M-x forge-add-repository' before trying again.

No amount of (re-)adding, forge-pull etc. fixed this. Eventually I figured it out: the repository name is all upper case, but my remote had defaulted to a lower case "user". The tricky bit: it seemed to work fine, e.g. for pulling, (and even forge-pulling), etc. because GitHub (/git urls?) doesn't care about the case of the user/organization name.

But apparently the forge related API stuff does care about case! I re-added the repository using the correct all-caps org name and that fixed it. Perhaps there's some way to detect this and save someone struggle (or at least document somewhere)...

[jdtsmith]

Actually I just came across this again. Even a single letter with changed case in the name leads to the same error. Since at the time of adding a remote in magit this isn't possible to detect, perhaps there's a way for forge to notice the capitalization mismatch.

[tarsius]

Yeah, while the web interface and git do not mind it if capitalization is wrong, the api does. Theoretically the git server could print informative messages when the client uses the wrong capitalization, which we could then detect. But it doesn't and so I think there is nothing we can do about this.

[pkryger]

I think github is returning a capitalized names. At least in some cases. I.e., when I did, following the case from OP:

(ghub-fetch-repository "pahfit"
                       "pahfit"
                       (lambda (data)
                         (let-alist data
                           (message "pahfit/pahfit: %s" .nameWithOwner)))
                       '((issues-until . "2021-03-19T00:00:00")
                         (pullRequests-until . "2021-03-19T00:00:00"))
                       :host "api.github.com"
                       :auth 'forge)

It produced:

pahfit/pahfit: PAHFIT/pahfit

[tarsius]

Hm maybe I remembered that wrong. Or this changed. Or graphql can deal with it but not rest.

So probably we could implement some self-healing method, but I don't think I will find time for that anytime soon.

[jdtsmith]

I don't think healing is necessary. Even a tiny warning for working git URLs which fail with forge to double-check their capitalization would be effective.

[gonsie]

I have been hitting this error for months now (and only just got around for searching for a solution). THANK you @jdtsmith for reporting this fix. A warning message would be super super helpful.

[tarsius]

Hm. Could it be that Github changed that in the 18 minutes since you posted that? (Alternatively I it might be that I just should have gone to bed already.)

So I did git clone git@github.com:pahfit/pahfit.git, even though the correct url would have been git@github.com:PAHFIT/pahfit.git. Then I did f y and it worked. And I did it again and it worked again. Are you experiencing something else?

[jdtsmith]

Don’t have my computer in front of me but I believe I added the upstream as a remote to my fork (with the wrong name), pulled it successfully, but then encountered the forge issues.

[gonsie]

Yes, the forge-add-repo call is successful (and in fact I think the database is correctly built). But subsequent commands, such as list topics or view issues fail with the error to run M-x forge-add-repository.

[tarsius]

For me these actions succeed as well.

[jdtsmith]

Here's how I just reproduced the WrongCaseName error (magit 20210406.454):

1. git clone git@github.com:jdtsmith/pahfit.git
2. Edit a file in pahfit directory.
3. Invoke magit-status.
4. M a to add a remote, call it upstream, with url git@github.com:pahfit/pahfit.git (after editing the suggested upstream/pahfit.git). Answer no when asked whether to set it as default pushRemote.
5. f a to fetch everything.
6. ' r as need to select the new remote upstream as the forge.repository.
7. ' f f to fetch all topics with forge: succeeds.
8. ' l t (and all other topic/issue commands) fails with

   Cannot use ‘forge-list-topics’ in "/Users/jdsmith/tmp/pahfit/" yet.
   Use `M-x forge-add-repository' before trying again.

And then to recover I simply killed the poorly name remote, and repeated steps 3–8 using the true organization name (correctly capitalized) with success. This was easy to fix once I understood the issue, but it took some time to do so.

[tarsius]

I don't get that error after doing these steps. Instead a single topic is pulled https://github.com/jdtsmith/pahfit/pull/1, from your fork instead of the upstream.

That's because you don't follow the convention of naming the upstream remote origin. It's only a convention so you don't have to follow it but some tools will work better if you do. Forge for example adds the only remote as the upstream, but if there is more than one, then it uses origin; and if that does not exist, then it has to somehow pick one.

Given the above report it appears it does the picking somewhat randomly. There's probably room for improvement here.

So either name the upstream origin or set forge.remote.

[jdtsmith]

My apologies, I forgot one step I had done, between 5, and 6; updated above. Since my fork's capitalization is correct, it's possible to work on topics from it; but as you noted, that is not where the topics reside.

The typical GitHub convention is to call the parent repository from which you forked the upstream, and your own fork's repository origin. But again, I don't think the naming convention is really the issue, since in fact I had specifically selected the remote from which I wanted forge to pull issues. And after I kill the mis-capitalized remote and re-add it correctly, steps 4–8 work fine. So it does truly seem to be case sensitivity of the issue API forge uses, combined with lack of case sensitivity of the git API (combined with me thinking org names never have capital letters!).

[tarsius]

My apologies, I forgot one step I had done, between 5, and 6; updated above. Since my fork's capitalization is correct, it's possible to work on topics from it; but as you noted, that is not where the topics reside.

I followed the updated instructions and it still works for me. ' l t works and it gets the topics from the upstream repository.

[tarsius]

(This is off topic but I would still like to follow up. I am using a separate commend so that this side-thread can later be hidden.)

The typical GitHub convention is to call the parent repository from which you forked the upstream, and your own fork's repository origin.

Can you link me to any Github documentation that actually says one should do that?

I can see how one might prefer to call the upstream repository upstream instead of origin (similar to how some people prefer trunk or main over master).

That's fair, but at the same time calling some other repository origin does not make sense. (The term "origin" isn't a suitable term for something that is reasonably described as a "somehow modified and/or extended version of the... uhm... prototypical (one might also say 'original') version".)

And even if one felt that it makes sense, then one should still not do it because what we then end up with some people calling the fork origin and some other people calling the upstream origin. Sadly that's what actually has happened but I don't think Github actively encouraged it.

[jdtsmith]

I followed the updated instructions and it still works for me. ' l t works and it gets the topics from the upstream repository.

Fascinating. Is there any chance there is an update to forge which made this work for you and not me? I see I'm on forge 20210406.1356.

[jdtsmith]

(This is off topic but I would still like to follow up. I am using a separate commend so that this side-thread can later be hidden.)

The typical GitHub convention is to call the parent repository from which you forked the upstream, and your own fork's repository origin.

Can you link me to any Github documentation that actually says one should do that?

I can see how one might prefer to call the upstream repository upstream instead of origin (similar to how some people prefer trunk or main over master).

That's fair, but at the same time calling some other repository origin does not make sense. (The term "origin" isn't a suitable term for something that is reasonably described as a "somehow modified and/or extended version of the... uhm... prototypical (one might also say 'original') version".)

True, but it is in fact a very good term for something which is described as "the place where all my work goes, the center of all of my effort." I think this could all flow from the change in perspective of "top-level" maintainers vs. 'pro users' (see below).

And even if one felt that it makes sense, then one should still not do it because what we then end up with some people calling the fork origin and some other people calling the upstream origin. Sadly that's what actually has happened but I don't think Github actively encouraged it.

I think I probably picked that up from things like triangular workflow or similar or others. There are differing methods for sync, but one constant convention is that upstream is the name of the repository you made your fork from. GitHub even gets in on this (though it uses both "upstream" and "original" in the same sentence).

This popular stack overflow answer makes this even clearer:

• upstream generally refers to the original repo that you have forked (see also "Definition of “downstream” and “upstream”" for more on upstream term)
• origin is your fork: your own repo on GitHub, clone of the original repo of GitHub

I admit that if instead origin means "the original repository from which you forked", I don't then understand the use of the name upstream for anything. What would it be but the source fork from which you made your fork? Changes which occur there are quite literally "upstream" from your own changes. You will need to merge them in and accommodate them, before submitting pull requests (paddling hard to make headway... upstream). Even as your pull requests languish, you will need to monitor upstream and stay up to date with it. And since I always start by git cloneing my own fork, and in git origin is already the default name for the remote, that seems natural to keep.

So what else could I do? I guess if I first cloned the original repository, let its remote remain named origin, then added my own fork as fork or similar, that would also work. But that seems eminently unnatural: I WORK on fork, that's where all commits I make go; it's the default workspace. So it seems natural to clone my fork and push to it, and adopt the git default origin name for its remote.

I've honestly never seen other perspectives deviate much from this convention, so (possibly as a result) I've always been confused by magit's understanding of origin and upstream.

Perhaps this summarizes the difference?

1. If you are a maintainer of a popular "original" (like magit!), from which many people make forks, you think of yours as "origin". You work there directly, commit to it, and occasionally entertain PR requests coming in from all those forks. Convention: origin -> fork
2. If you are just a "user" of "original", you can't work there; you don't have commit privileges. You live in your fork, you commit to it as origin, and you try your best to make PR's that upstream might deem acceptable. You are clearly "downstream" from "upstream". The original maintainer might at any time make changes that break all the work you have done. Convention: upstream -> origin
3. You are a group of equals who all can contribute to a joint project. All of you have write access to said project's repository, but none of you exercise that right except for housekeeping or emergencies. For clarity, and to encourage the same workflow for outside contributors as for maintainers, all of you use your personal forks of project, and submit changes as PR's. Convention: upstream -> origin

But git is so flexible, perhaps there are other conventions that I'm entirely unaware of.

[jdtsmith]

Can you link me to any Github documentation that actually says one should do that?

Another useful source (from a book on forking workflow):

Whereas other Git workflows use a single origin remote that points to the central repository, the Forking Workflow requires two remotes—one for the official repository, and one for the developer’s personal server-side repository. While you can call these remotes anything you want, a common convention is to use origin as the remote for your forked repository (this will be created automatically when you run git clone) and upstream for the official repository.

[jdtsmith]

And even if one felt that it makes sense, then one should still not do it because what we then end up with some people calling the fork origin and some other people calling the upstream origin. Sadly that's what actually has happened but I don't think Github actively encouraged it.

And one final thought: I agree it's an issue that #1 vs. #2/#3 have different ideas of where origin points, but it's not directly clear which of them is right and which is wrong. How could you even decide? #2/#3 (upstream -> origin) far outnumber #1 (origin -> fork), since there are many more forks than the official repo. By "activity" (commits, etc.) perhaps #1 would win in many cases.

Maybe there's a way out? Perhaps (except in rare circular cases) we can all agree on who is upstream right? Maybe not.... if someone forks my fork of magit, my fork will be their upstream! What if the original repository has been taken offline (I actually have a fork of such a project on disk)? Is it still to be referred to forever more as "origin", despite being inaccessible? You see the logical trapdoor creaking opening here...

But here's a simple thought: since origin is the default remote name in git, it will almost certainly correspond in people's minds and practice to "the default place I upload my work to the server". It just happens that this place sometimes is upstream from other repositories, and sometimes downstream from them, and sometimes both.

I don't think magit or forge are well-served by presuming anything otherwise.

[tarsius]

Seems this question is similar to spaces vs. tabs. I've started a twitter poll and early results indicate that I'll have to support both mental modals. More on that when the poll is over.

[pkryger]

I do agree that the tabs vs. spaces resemblance is quite striking. For that reason I almost don't use origin nor upstream. I keep naming remotes after a fork name and I use solution I suggested in #235 to help me forge to automatically track an upstream remote. I'm not suggesting to revive that. Just pointing out there are even more ways of managing remotes, and with the power of lisp and configurability of Emacs libraries I can use whatever workflow I think fills my needs best.

[jdtsmith]

Seems this question is similar to spaces vs. tabs. I've started a twitter poll and early results indicate that I'll have to support both mental modals. More on that when the poll is over.

Count me firmly in the camp of "default origin is wherever you submit most of your commits to". Which is almost always "my very own fork". I found your poll and wasn't clear what the 3rd option means: for me option 3 also counts towards option 2!

BTW, are there any pages suggesting the name origin be reserved for the authoritative repo? I see tons of articles, stack-overflows, etc. strongly suggesting upstream -> origin (to avoid confusion! hah!), but none really advocating origin -> XXXX in all cases.

[tarsius]

Due to https://github.com/magit/magit/pull/4366 people who name the upstream upstream now get the same dwim magic as those who name it origin. I might blog about that at some point, but since this now just works for both camps, that's not urgent.

[jdtsmith]

FYI, I just tried again the steps I listed above, using the latest forge 20210507.1554, and received the exact same error at the end (from ' l t):

Cannot use ‘forge-list-topics’ in "/Users/jdsmith/code/python/scraps/pahfit/" yet.
Use `M-x forge-add-repository' before trying again.

I am still uncertain how it is you are avoiding the incorrect capitalization error on your end. If anyone else could try with a fresh session and confirm/deny that would be helpful.

[tarsius]

Please try again in a few hours once the changes that I just pushed have landed on Melpa.

I wasn't trying to fix this particular issue per see, but another issue with might turn out to be the actual issue in this case too.

[tarsius]

Guessing that this is fixed.

[jdtsmith]

Sorry missed this will take a look and report back if not.