"Operation Not Permitted" no Clone

[Blackhawke]

Describe the bug I just tried to Clone a repository. The files were pulled and the app was in the process of creating the git index files and then... Boom! "Operation Not Permitted" and the repo was deleted. For verification, I tried in another, older app, that I used to use called PocketGit -- which still works -- and same result. The only difference is, PocketGit didn't delete the files that had been cloned, so I was able to verify that the user/working files in the repo were actually pulled, the .git folder tree was created, etc.

This is a GitoLite repo sitting on my own servers, so I was also able to check the auth logs and so on. The SSH keys were accessed and accepted as normal. The connection to git and the repo was fine (as you would expect, since it actually pulled the files), and was terminated "by the user" -- meaning the apps (Mgit/PocketGit). A bit of digging around the web leads me to believe this is Android related. There has been an epidemic of "Operation Not Permitted" errors related to file access happening since Android 12 -- which so far hadn't affected me or my git repos. But in October I was updated to Android 13.

Smartphone (please complete the following information):

• Device: Samsung Galaxy Tab S6 Lite (SMP910)
• OS: Android 13, Kernel 4.14.113-27209537
• App Version 1.7

Additional context What's interesting is, this isn't the only repo on my tablet, and the repos that were on my device before the upgrade, and that I haven't re-cloned, still work! I can push/pull just fine. So if it is Android 13, so far this is the only repo affected, and it only happened when I had to delete the files and re-clone.

[maks]

Thanks for reporting the issue and the detailed description. This does sound like #620 . For now the only work around to this is to install the latest version of MGit from F-Droid.

[Blackhawke]

Sorry about that! I read through, and even searched, the issues and apparently didn't catch #620.

Anyway I thought 1.7 was the latest version. You mean the one we can download here on github isn't the latest version of the app?

Just out of curiosity, why F-droid? I run several apps not permitted on the PLayStore that I sideload from here on github -- and they all have internal notice systems that tell me when there's an update. No app for apps needed.

Anyway, thanks! I guess I'll have to install F-droid and that version of your app and see what happens. I'll let you know.

[maks]

No worries at all. When you say "down here on github" do you mean you are building the app yourself from source? If so you shouldn't have any issue with the restriction as long as you are using the master branch and allow the all file permission via Android System settings, I only mentioned F-Droid for those not able to build it themselves as I don't provide installable builds here.

[Blackhawke]

No, I mean downloading the APK from the URL provided, then sideloading. In my case, throught FireFox. My problem with F-Droid is that I have to disable tracking control (another app here on github) in order for it to work, and I don't like that. :smile: I like knowing whose tracking what.

Anyway. I did install F-Droid and download the version of the app you have up there. Same results. Cloned the repo, went through the indexing process, then died with the "Operation Not Permitted" error.

[maks]

Sorry which provided url?

[Blackhawke]

Well... That's a fair question. When I first loaded the page there was the PlayStore link and the F-Droid link, and then there were links to the apk with the warning that there would be no update notices if the app wasn't downloaded from F-Droid.

Now I don't see that on the page. Only the PlayStore and F-Droid links.

Sorry! I know it was your app I installed because it looked the same, worked the same, and was even the same version as the one I installed from F-Droid.

Color me confused. :lol:

(But I'm leaving F-Droid on my device for when you get this sorted.)

[maks]

Ah ok, well I did previously provide APK builds here in the releases section, but the last one was back v1.5.2 back in 2017. Otherwise you probably got the APK from F-Droid as they (not me) provide a direct download of the APK also.

If you are using the version from F-Droid that you side-loaded, you should check if the "all files" permission is enabled for MGit in the system settings, as that not being enabled is the likely cause of the issue you are seeing.

[Blackhawke]

No, i'm downloading and installing directly from F-Droid now. Tried again with "no permissions denied" in settings (so all files, including photos, etc. allowed) and same result.

[maks]

Not sure we are on the same page here.

When you first installed and ran MGit from Droid you should have been shown this screen:

And then once you followed the instructions you should have been taken to the all files permissions screen (which is NOT a regular Android app permission) and enabled it for MGit:

Can you confirm you have that permission enabled for MGit?

[Blackhawke]

Yes. And after your last comment, i then also went into settings and made sure the rest of the files permissions (for photos and videos) were also enabled.

[maks]

Hmm, then in that case I'm not sure what the issue could be. And the photos/videos permissions is not relevant here, only the "All file permission". Note that they are not the same, the all files permission is a "special permission" and hence in a separate area of the permissions settings. If you have that specifically enabled for MGit then I dont know what the issue would be.

[Blackhawke]

Well... Maybe we're looking at this the wrong way. What do we know?

• Existing repositories still push/pull/commit just fine.

On a clone:

• The folders are created and files are successfully fetched and written to disk.
• The .git folder tree is successfully created and written to disk, as are some files: HEAD, .idx, .pack, etc. and...
• It only errors out after it's started indexing the repo. But since the *.idx file has been written to disk, it may well be another operation that's causing the system to throw the not permitted error, not the indexing itself.

So it seems to me the real question may be: What operation does Git have to perform on a clone that it doesn't have to do to maintain an existing repo? It may not actually be a disk permissions error at all.

Too bad you don't have a debug mode written into your app that creates a log file. That would help.

[maks]

How are you able to see that files are being written to disk if you previously said that there were no files after the clone operation had failed? Have you tested with a publicly available, moderately sized repo hosted here on github (eg. this repo)?

[Blackhawke]

As I said in my initial post, I used to use an app called PocketGit to verify the error. It's been abandoned since 2017 (hence your app), but still works. While it produced the same error, unlike your app, it didn't delete the files and folder when it erred out. So I was able to see what had been written to disk. Since it's the same error, and it's occurring at the same time (during or after "indexing files"), I infer that Android 13 is throwing the error at the same point, so it's the same problem.

Obviously, I could be wrong.

[maks]

Sorry I think we are talking at cross purposes here. Are you able to check that you get the same error with MGit when you try to clone the MGit repo from here on Github using https?

[Blackhawke]

No, I don't think we're at cross purposes. As I said above, I get the same error in the same place -- during or after "indexing files" -- on both apps. Whether Git is trying to complete the indexing, or write the results to disk, or something else, I don't know.

[Blackhawke]

Just tried a couple of things. Hope it helps with your investigation.

First, i tried "importing" the repo that wouldn't clone from the files left by PocketGit. Your app didn't complain at all. Mind, I'm not sure what the state of the repo really is, and it's a big, working repo, so that's where I've left it.

Second, i forgot a had a little test repo I'd setup when I installed GitoLite. So I cloned it. It cloned just fine. So I modified a file, committed/pushed. Perfect. Modified the file on my desktop. Committed/pushed, then pulled on the tablet. Perfect!

So now I'm cornfused. 🤣

What's up with that one repo?

Oh, and the error occurs after indexing. (indexing 100%) Tried cloning one more time just to check.

Hope this is useful.

[Blackhawke]

Just found this in the Termux read.me file:

Android OS will kill any (phantom) processes greater than 32 (limit is for all apps combined) and also kill any processes using excessive CPU. 

Might it be related part of the problem?

[maks]

Thanks for investigating this more.

That makes more sense now that a small repo clones without any issues. Could you let me know how big (repo size in total) that fails is? If its large, you may be running into another long standing issue #54 .

I don't think a process limit would have anything to do with MGit as it uses a pure Java implementation of Git (JGit library) so doesn't spawn any native processes apart from in some niche corner cases, while Termux is essentially a Linux chroot and undoubtedly spawns any number of processes.

[Blackhawke]

Thank you for that link. I think you may be right. It would explain why existing repos still work, but the clone attempt failed.

Currently the repo is 75.5MB is size and contains 3,146 files.

EDIT: So I just split off a part off the part of the repo that we're currently using and made a new repo out of it. It now had 548 something files and is just over 42MB in size. But I'm still getting the "Operation not permitted" error at the exact same place. By comparison. The little test repo has 120 files and is 260KB is size.

[Blackhawke]

Okay... I'm a dork!

Well, maybe I'm a dork, since I don't know if it matters or not. Anyway...

The numbers I gave you above include the files in the .git folder. So here are the numbers for the 3 repo's without the .git folder: The folder in question: 482 files/37MB. The test folder: 18 files/28Kb. The new repo I split off of the big repo: 197 files/29MB.

[Blackhawke]

One final thing I did yesterday is attempt to clone both the "big" repo and the new repo I split off of it onto other devices. One, my wife's Samsung TAB isn't really fair because I've rooted it and installed vanilla Android on it. So of course git works fine. The other is my Samsung Note 9. It's stock and cloned the repos fine too, but Samsung quit updating the OS at version 10. So in some senses I'm not sure it's a worthy data point either, other than to show that changes made to Android 13, which is what my S6 is now running, might be the source of the problem.

[Blackhawke]

In the last two weeks I've spent about 8 hours in chat with Samsung. In typical two-faced manner, on the one hand they're not admitting this is their problem. On the other, they're now admitting they're going to have to issue a patch to fix the problem.

I guess that's some sort of progress.

[cloudylong]

Maybe your file name contains special characters like <. After I deleted those, it works.

[Blackhawke]

Yeah, i'm aware of the character limitations. These are not new repos. They've been in use for years and are accessed git running on multiple OSes.

On Tue, Mar 5, 2024, 2:04 AM Walter @.***> wrote:

Maybe your file name contains special characters like <. After I deleted those, it works.

— Reply to this email directly, view it on GitHub https://github.com/maks/MGit/issues/708#issuecomment-1978390897, or unsubscribe https://github.com/notifications/unsubscribe-auth/AANNFBNSTRSDWFVPTTW2E6TYWWKD7AVCNFSM6AAAAABBJH64AKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSNZYGM4TAOBZG4 . You are receiving this because you authored the thread.Message ID: @.***>

[Blackhawke]

Got an OS update from Samsung and... Now git works just fine. So it was an OS problem.