Issues using Git after installing Netlify Large Media

jeremyzilar commented 4 years ago

We are trying to migrate the Brooklyn Rail to Netlify + NetlifyCMS and one of the early snags we are running into is with setting up Netlify Large Media + Git-LFS to manage the media in the repo.

The issue is that after installing all of the required elements, we are no longer able to commit back to GitHub 😬

Specifically,

not able to push or pull using GitHub in the terminal when the remote path is set to https://github.com/brooklynrail/brooklynrail-platform.git. But we are able to push and pull when the remote is set to git@github.com:brooklynrail/brooklynrail-platform.git, though it asks for our SSH key password twice before pulling/pushing.
not able to push or pull using GitHub Desktop, both when the remote path is set to https://github.com/brooklynrail/brooklynrail-platform.git or git@github.com:brooklynrail/brooklynrail-platform.git.

(note, this seems to only affect this repo, and this branch)

If you are interested in looking at :eyes: our NetlifyCMS config, it can be found at /themes/brooklynrail/static/workflow

What have we tried?

re-installing everything
following a lot of troubleshooting advice here https://piperhaywood.com/configuring-and-troubleshooting-netlify-large-media/

I ran git config --list --show-origin to help with any debugging. Here is what I got:

jeremy@Field-Work brooklynrail-platform % git config --list --show-origin
file:/usr/local/etc/gitconfig   credential.helper=osxkeychain
file:/usr/local/etc/gitconfig   filter.lfs.clean=git-lfs clean -- %f
file:/usr/local/etc/gitconfig   filter.lfs.smudge=git-lfs smudge -- %f
file:/usr/local/etc/gitconfig   filter.lfs.process=git-lfs filter-process
file:/usr/local/etc/gitconfig   filter.lfs.required=true
file:/Users/jeremy/.gitconfig   filter.lfs.process=git-lfs filter-process
file:/Users/jeremy/.gitconfig   filter.lfs.required=true
file:/Users/jeremy/.gitconfig   filter.lfs.clean=git-lfs clean -- %f
file:/Users/jeremy/.gitconfig   filter.lfs.smudge=git-lfs smudge -- %f
file:/Users/jeremy/.gitconfig   user.name=Jeremy Zilar
file:/Users/jeremy/.gitconfig   user.email=jeremyzilar@gmail.com
file:/Users/jeremy/.gitconfig   include.path=/Users/jeremy/.netlify/helper/git-config
file:/Users/jeremy/.netlify/helper/git-config   credential.helper=
file:/Users/jeremy/.netlify/helper/git-config   credential.usehttppath=true
file:/Users/jeremy/.netlify/helper/git-config   credential.helper=netlify
file:/Users/jeremy/.netlify/helper/git-config   credential.helper=osxkeychain
file:.git/config        core.repositoryformatversion=0
file:.git/config        core.filemode=true
file:.git/config        core.bare=false
file:.git/config        core.logallrefupdates=true
file:.git/config        core.ignorecase=true
file:.git/config        core.precomposeunicode=true
file:.git/config        submodule.active=.
file:.git/config        remote.origin.url=git@github.com:brooklynrail/brooklynrail-platform.git
file:.git/config        remote.origin.fetch=+refs/heads/*:refs/remotes/origin/*
file:.git/config        branch.master.remote=origin
file:.git/config        branch.master.merge=refs/heads/master
file:.git/config        branch.pa11y.remote=origin
file:.git/config        branch.pa11y.merge=refs/heads/pa11y
file:.git/config        branch.hugo_env.remote=origin
file:.git/config        branch.hugo_env.merge=refs/heads/hugo_env
file:.git/config        branch.cms/authors/jeremy/_index.remote=origin
file:.git/config        branch.cms/authors/jeremy/_index.merge=refs/heads/cms/authors/jeremy/_index
file:.git/config        lfs.https://db6c835f-87e3-47c4-8f4d-53b4af3c6be8.netlify.app/.netlify/large-media.access=basic
file:.git/config        lfs.https://db6c835f-87e3-47c4-8f4d-53b4af3c6be8.netlify.app/.netlify/large-media.locksverify=false
file:.git/config        credential.helper=netlify
file:.git/config        lfs.https://github.com/brooklynrail/brooklynrail-platform.git/info/lfs.access=basic
file:.git/config        branch.images.remote=origin
file:.git/config        branch.images.merge=refs/heads/images
file:.git/config        branch.media-cms-fix.remote=origin
file:.git/config        branch.media-cms-fix.merge=refs/heads/media-cms-fix

Also

Still unsure if we need to be using Git Gateway with Netlify to get this set up.
If so, do we need netlify-identity-widget as part of the identity set up? If so, why is it not mentioned here?

erezrokah commented 4 years ago

Thanks for reporting @jeremyzilar, I'll try setting this up and see if I have the same issues. To use Large Media with the CMS you do need to configure it with git-gateway and Identity since the CMS uses the Identity access token to access the Large Media service. The Identity widget is needed to facilitate authentication with Identity. Is this the place in the docs we should mention it?

Not sure it is related, but found this https://github.com/netlify/netlify-cms/issues/2148

erezrokah commented 4 years ago

Linking https://community.netlify.com/t/common-issue-troubleshooting-your-netlify-large-media-configuration/188 for more information

jeremyzilar commented 4 years ago

Thanks @erezrokah, I am going through the git-gateway setup now, and will take notes on the steps to set it up. I will also start a new issue around any bits of confusion with Git-Gateway and Identity (since that is a separate topic) https://github.com/netlify/netlify-cms/issues/3718

jeremyzilar commented 4 years ago

From this troubleshooting guide, I ran netlify lm:info and got

jeremy@Field-Work brooklynrail-platform % netlify lm:info
  ✔ Checking Git version [2.22.0]
  ✔ Checking Git LFS version [2.10.0]
  ✔ Checking Git LFS filters
  ✔ Checking Netlify's Git Credentials version [0.1.9]

Here's what I did next

I double-checked that I had Netlify CLI installed, and I ran the global install found here npm install netlify-cli -g
Updated netlify-cli in my local repo npm i netlify-cli
Ran netlify init to ensure that my local repo is connected
ran netlify plugins:install netlify-lm-plugin (did this previously, but why not try again)
ran netlify lm:install (also did this previously)
lastly, I ran the command to use Netlify Large Media in my current shell

Results

It works 🎉— I am now able to push and pull using GitHub Desktop and the command line!

jeremyzilar commented 4 years ago

Now onto getting the Git Gateway setup and working...

jeremyzilar commented 4 years ago

Without a doubt, there are still issues with using GIT after installing LFS and getting this setup :/ Especially when trying to collaborate with others and merge in changes. I will try to document the steps and the specific issues.

jeremyzilar commented 4 years ago

@erezrokah this is the specific issue I am getting:

batch response: Post https://db6c835f-87e3-47c4-8f4d-53b4af3c6be8.netlify.app/.netlify/large-media/objects/batch: dial tcp: lookup db6c835f-87e3-47c4-8f4d-53b4af3c6be8.netlify.app: no such host
error: failed to push some refs to 'https://github.com/brooklynrail/brooklynrail-platform.git'

This thread on GIT-LFS issues seems relevant https://github.com/git-lfs/git-lfs/issues/2887 Still looking...

overlordofmu commented 4 years ago

Hi, @jeremyzilar, as mentioned in the Git LFS issue thread (git-lfs/git-lfs#2887), this looks like a DNS resolution issue.

The domain name in this case works for me when I test:

$ nslookup db6c835f-87e3-47c4-8f4d-53b4af3c6be8.netlify.app
Server:     8.8.8.8
Address:    8.8.8.8#53

Non-authoritative answer:
Name:   db6c835f-87e3-47c4-8f4d-53b4af3c6be8.netlify.app
Address: 138.68.244.143

Would you be willing to test the same command on the affected system if the issue is still occurring? That command being:

nslookup db6c835f-87e3-47c4-8f4d-53b4af3c6be8.netlify.app

The the DNS lookup fails locally, that is the root cause of the error above. If you do get local DNS errors we'll be happy to troubleshoot to find out why and resolve it.

martinjagodic commented 1 month ago

Closing as not planned because Netlify Large Media is sadly deprecated

decaporg / decap-cms