search

SRombauts / UEGitPlugin

Unreal Engine 5 Git LFS 2 Source Control Plugin (beta)
http://srombauts.github.io/UEGitPlugin
MIT License
811 stars 165 forks source link

Git LFS 2.0.0 File Locking Mechanism (Work In Progress) #36

Closed SRombauts closed 7 years ago

SRombauts commented 7 years ago

I've started work on the Git Plugin to add explicite Git LFS support with the new "lock" mechanism.

I am developing this in a branch on my fork of the UnrealEngine repository (git-plugin-add-git-lfs-lock-support branch) Now available in this repository add-git-lfs2-file-lock-support (release "2.3-lfs2-lock-preview")

SRombauts commented 7 years ago

I currently have a "Proof Of Concept" branch more or less working with LFS locks :+1:

fire commented 7 years ago

What do I need to test this?

fire commented 7 years ago

Can you separate the changes into a plugin? It would be so helpful.

SRombauts commented 7 years ago

Hi @fire, thanks for you interest!

I believe it's still a bit too early to test: everything is on a branch on my fork, as stated above (https://github.com/SRombauts/UnrealEngine) so you need to rebuild the full Editor from source.

I've started the process of backporting all into this repository so that it will be easier for other to test, but it will take a few more hours/nights to get it in.

fire commented 7 years ago

I was able to extract it as a plugin. What servers support the git-lfs locking?

SRombauts commented 7 years ago

I am testing it with Github. They are behind Git LFS and are probably the only one with a server implementation ready at this time.

Edit: I hope to finish the work on the plugin configuration in a few days, there are currently some big holes (remote, credentials...) and a few hard-coded spots (in the "State" class for instance)

SRombauts commented 7 years ago

Hi @fire, I discovered that the two last commits, about discovering Git LFS capabilities, and about Settings to enable/disable the Locking workflow was both broken: this was effectively disabling the "lock"/"unlock" commands entirely!

Also, I have a lot of TODOs and small in-progress improvement locally that are not ready to go public:

  1. don't try to lock a new ("Added") asset
  2. don't try to unlock a file that is not locked

At this stage, the whole thing assumes you have set up the repository correctly and have it working in command line, so that the plugin can just reuse the credentials.

Also everything is awfully slow (LFS is slow, and the Editor double some status commands) and awfully buggy

fire commented 7 years ago

Ah, how did it go? Did you get stuck on something?

SRombauts commented 7 years ago

Hi @fire, I got it to a point where I could make a checkin and push, but it needs far more works to be production ready. For now:

  1. you need to complete the setup of the repository by command line (remote url and Github user credentials)
  2. for now, I assume the whole Content/ directory is tracked, but this does not enable editing of Config/ and uproject files since the Lock command fails for them

In the meantime, I discovered more bugs/limitations of the current Git plugin, so I am working on a few merge requests to fix those ASAP.

I have also updated this plugin repository, and am going to switch it to the main dev as the "Git LFS" plugin so you could have both in Engine.

I will keep you inform, for now my main problem is trying to focus on one task at the time...

fire commented 7 years ago

Can't wait!

SRombauts commented 7 years ago

I've spent some time polishing the UE4.16 release.

I've spent a greater amount of time updating this repo and shaping it into a standalone "Git (dev)" replacement plugin that you can put in your project's Plugins folder to replace the default in Engine "Git (beta version)" plugin (so you can not have both at the same time)

@fire I am currently busy on another project, but please keep asking to push me :) !

BenjaminDSmithy commented 7 years ago

@SRombauts any new advancements? Also GitLab by far has the best implementation of Git LFS v2.x File Locking capabilities as they actually offer a front-end website GUI solution as seen here: https://docs.gitlab.com/ee/user/project/file_lock.html

GitHub comes in probably at 2nd place in that regards and Bitbucket Cloud + Server offerings aren't even supported yet and they have open community tickets pending for requesting those features as seen here: https://jira.atlassian.com/browse/BSERV-9623

jfaz13 commented 7 years ago

@SRombauts I don't know why this isn't being talked about more. Locking files is one of the main reasons UE4 devs tend to use Perforce

SRombauts commented 7 years ago

Well, in my experience, Git LFS 2.0.0 was a bit shaky, and verrrry slow compared to Perforce and Plastic SCM, so that got me a bit demotivated. I am also very busy on other projects, sorry about that :(

faemir commented 7 years ago

Just chiming in to say that I'm eagerly awaiting this feature, and happy to help test it!

Totoro83 commented 7 years ago

I can't wait too! (Hoping this will help pushing the work ❤️ )

SRombauts commented 7 years ago

Hi @fire @BenjaminDSmithy @aztekyi @faemir @Totoro83, thanks for you support on this!

I really mean it, that is, I really need some motivation to get back on this! It's a bit of a daunting task since I let it in a not so good shape, and during this work I've discovered some bugs on other parts of the plugin. And now the code is diverging more and more on each release...

So hey, I should be back on the plugin, I have no other personal project going on currently, so I'll try to start by refreshing the code, and probably at least doing another general bugfix before Git LFS 2.

YotaVGA commented 7 years ago

Really thanks for the work you're doing!

SRombauts commented 7 years ago

Hi all!

I have published yesterday a "Early-Access" preview-version of the Git LFS 2 plugin with some first level of support for File Lock.

Please note that:

  1. Fixed in Git LFS 2.3.0 released 7 days ago! I have found Git LFS 2 to be not totally reliable (under Windows at least). An early tester found that sometimes "read-only" flags are not in sync with "file lock" which can totally mess with the Editor assumptions on those!
    1. This was reported testing OUSTIDE UE, so WITHOUT the plugin!
    2. The plugin would probably have to handle this, but this could be non-trivial (think medium-term goal).
  2. This pre-release have a known issue about File Locked by others not correctly handled!. This needs to be addressed ASAP, but I felt that I could still published the code as-is for early tests.

You have been warned enough ;) add-git-lfs2-file-lock-support (release "2.3-lfs2-lock-preview")

Cheers!

SRombauts commented 7 years ago

I found an issue about this on the Git LFS project issue tracker: "git lfs unlock" does not reset the readonly file system attribute. #2494 It is solved in Git LFS v2.3.0 released 7 days ago!

So you cannot rely on the previous Git LFS v2.2.1 release, the one integrated in latest "git for windows" (2.14.1) (like I did)

SRombauts commented 7 years ago

Okay, so initial support of Git LFS 2 File Locks is release in v2.3 of the plugin :)

Let me know if it breaks/works for you.

Documentation should be improved on how to use this...

SRombauts commented 7 years ago

So if you want my opinion on the remaining works:

  1. LFS locks on Windows command line needs to improve performance wise
  2. I shall discover and eradicate a few minor bugs (like errors on logs and some quirks on some specific commands
  3. I have to try to improve the robustness of the whole thing (like how to handle disconnection, failure modes...)
  4. I have to document the whole thing!
jfaz13 commented 7 years ago

That's great! Thanks!

On Thu, Sep 21, 2017 at 9:48 AM, Sébastien Rombauts < notifications@github.com> wrote:

Hi all!

I have published yesterday a "Early-Access" preview-version of the Git LFS 2 plugin with some first level of support for File Lock.

Please note that:

  1. I have found Git LFS 2 to be not totally reliable (under Windows at least). An early tester found that sometimes "read-only" flags are not in sync with "file lock" which can totally mess with the Editor assumptions on those!
  2. This was reported testing OUSTIDE UE, so WITHOUT the plugin!
  3. The plugin would probably have to handle this, but this could be non-trivial (think medium-term goal).
  4. This pre-release have a known issue about File Locked by others not correctly handled!. This needs to be addressed ASAP, but I felt that I could still published the code as-is for early tests.

You have been warned enough ;) add-git-lfs2-file-lock-support (release "2.3-lfs2-lock-preview") https://github.com/SRombauts/UE4GitPlugin/tree/add-git-lfs2-file-lock-support

Cheers!

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/SRombauts/UE4GitPlugin/issues/36#issuecomment-331161804, or mute the thread https://github.com/notifications/unsubscribe-auth/AX__1AEa3aAYcn6v9__jvLEerZC79kU4ks5skmksgaJpZM4MW6Kl .

ArtyMcLabin commented 6 years ago

hey, any news about that project?

SRombauts commented 6 years ago

Well yes, Git LFS 2 was released in v2.3 about 4 months ago, I've released bug fixes and a few new features in two releases, with v2.5 yesterday.

I am deeply missing feedbacks since I don't use LFS 2 myself in my daily workflow!

ArtyMcLabin commented 6 years ago

as i understand it will still not work like perforce mechanism, and wont prevent people from editing locked files, and instead just disabling their pushes, am i right? if i might ask, what do you use instead?

SRombauts commented 6 years ago
  1. The Git Plugin enforce the Lock, so the user is not allowed to checkout already locked by someone else, exactly like in Perforce. It is still possible to make it writable, like in Perforce, but then you are not able to Checkin, like in Perforce.
  2. I am using plain git most of the time, that is without LFS or LFS 2 as I am working mostly alone.
SRombauts commented 6 years ago

To install this, please just download the latest release and put it in your project Plugins/ folder (not in Engine, to override the Git plugin present in Engine)

ArtyMcLabin commented 6 years ago

Yes thank you, i have erased my previous comment after realising that :] It works. thank you very much for your contribution!