search

jedbrown / git-fat

Simple way to handle fat files without committing them to git, supports synchronization using rsync
BSD 2-Clause "Simplified" License
622 stars 137 forks source link

python3 Support #92

Open Jwink3101 opened 5 years ago

Jwink3101 commented 5 years ago

python2 is EOL Jan 1, 2020.

I am going to see what is involved in updating git-fat but even if I can't it is good to have an issue open to track progress

Jwink3101 commented 5 years ago

I am extremely close to having this done but I am having issues with 2.6. It may be that I do not have the best test platform for it but I am still digging in.

With that said, I argue that we really do not need to support 2.6 anymore. I think that is an acceptable break.

Thoughts?

I'll keep trying to fix it in the mean time

Jwink3101 commented 5 years ago

Just another followup,

My changes don't break 2.6 per se. I couldn't get it to work with 2.6 even with the current version!

Seems to be a known issue (see #46). I guess it is therefore out of scope here.

And there really is no need to support 2.6!

jrollins commented 3 years ago

Is this project dead? I hope not. I like the simplicity of the model compared to other large file support systems.

Jwink3101 commented 3 years ago

I haven't had the motivation to go back and fix the few things.

I developed an alternative with a few more features but I lost steam. I've considered parring it down and releasing it. It's basically like git-fat but uses fewer command chains and does more logic in Python. And the biggest thing is that it lets you keep data on the server only. And support rclone

Are you interested? I haven't found the time to do it but it shouldn't be to hard to get to early beta.

jrollins commented 3 years ago

I'm certainly curious. I just found this:

https://github.com/ciena-blueplanet/git-fat

but it also appears to be defunct :(

jedbrown commented 3 years ago

Hi, sorry to be unresponsive. The simple state is that this project's test suite isn't sufficient for me to be able to quickly review PRs and I'm stretched super thin these days. I'm not using git-fat in any current projects and I feel it's more important to ensure that all former users continue to be supported without surprises than to deliver new features. Yes, Python-3 support is super overdue, and I'd definitely have finished if I were using git-fat.

The best case scenario in my opinion would be for someone else to take over maintainership. Second best would be to solicit contributions that improve testing.

leggewie commented 3 years ago

@jedbrown Thank you for creating git-fat. I strongly believe it has a place.

You say the test suite is insufficient. I'd love to help, but would need some mentoring. You've already approved #95 which apparently improves the test suite. What's holding it back from being merged?

malaybasu commented 2 years ago

Can someone take over this project? Python3 support is urgently needed. I don't know enough python to take over. Otherwise I would have. Git-fat is the simplest way to large files with git.

Jwink3101 commented 2 years ago

I had pull request #95 but didn't finish. I kind of lost steam and it didn't feel worth it.

I also started my own project that was heavily inspired by git-fat but worked with rclone. It was pretty much down but I decided to abandon it since there were some edge cases I didn't want to deal with and it was large and complex.

I started a simpler one but also gave up.

The issue is, there is a lot of complexity and keeping up with the latest APIs. Plus, I kept finding that my use case didn't match others.

My Suggestion

I think the best answer is to use git-lfs with a custom transfer agent to git the best of both worlds. You get the support, documentation, etc of git-lfs with the custom storage of git-fat.

And there are a few options including an alpha/beta one I wrote!

malaybasu commented 2 years ago

Thank you for your reply. When I tested last time, none of these plugin supported simple ssh. the 'rsync-agent' is simply flawed. It uses tmp directory to transfer the file completely ignoring the $TMPDIR variable. This breaks our server which has small tmp size. Which one has ssh support and a replacement for git-fat? Did not try your tool. But lsf-folderstore is not transparent and requires git configuration. Frankly, there is no replacement for git-fat I could find. git-lfs is simply evil. Also almost all of them will try to reconsitute the entire fat objects. Here are the crucial features I need:

  1. Transparent config. A user should be able to use it right away about cloning the repository. git-fat allows the config files to be synced.
  2. ssh support with no additional config
  3. On demand file download at the individual file object level.
  4. Transparent use using git GUIs
malaybasu commented 1 year ago

Has anyone tried using 2to3 to support python3?