GitCommandError "Host key verification failed" after mits3 Gitolite upgrade

[gjost]

Editor clients are seeing this error in the web UI after the mits3 Gitolite upgrade (https://github.com/denshoproject/ansible-colo/issues/36)

Traceback (most recent call last):
  File "/opt/ddr-local/venv/ddrlocal/lib/python3.7/site-packages/django/core/handlers/exception.py", line 34, in inner
    response = get_response(request)
  File "/opt/ddr-local/venv/ddrlocal/lib/python3.7/site-packages/django/core/handlers/base.py", line 115, in _get_response
    response = self.process_exception_by_middleware(e, request)
  File "/opt/ddr-local/venv/ddrlocal/lib/python3.7/site-packages/django/core/handlers/base.py", line 113, in _get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
  File "/opt/ddr-local/ddrlocal/webui/decorators.py", line 18, in wrapper
    return f(*args, **kwargs)
  File "/opt/ddr-local/ddrlocal/webui/views/decorators.py", line 18, in inner
    return func(request, *args, **kwargs)
  File "/opt/ddr-local/ddrlocal/storage/decorators.py", line 83, in inner
    return func(request, *args, **kwargs)
  File "/opt/ddr-local/ddrlocal/webui/views/collections.py", line 312, in edit
    collection.repo_fetch()
  File "/opt/ddr-local/ddrlocal/webui/models.py", line 407, in repo_fetch
    data = super(Collection, self).repo_fetch()
  File "/opt/ddr-local/venv/ddrlocal/lib/python3.7/site-packages/ddr_cmdln-3.0.0.post1-py3.7.egg/DDR/models/collection.py", line 457, in repo_fetch
    result = dvcs.fetch(dvcs.repository(self.path))
  File "/opt/ddr-local/venv/ddrlocal/lib/python3.7/site-packages/ddr_cmdln-3.0.0.post1-py3.7.egg/DDR/dvcs.py", line 543, in fetch
    return repo.git.fetch()
  File "/opt/ddr-local/venv/ddrlocal/lib/python3.7/site-packages/git/cmd.py", line 542, in <lambda>
    return lambda *args, **kwargs: self._call_process(name, *args, **kwargs)
  File "/opt/ddr-local/venv/ddrlocal/lib/python3.7/site-packages/git/cmd.py", line 1005, in _call_process
    return self.execute(call, **exec_kwargs)
  File "/opt/ddr-local/venv/ddrlocal/lib/python3.7/site-packages/git/cmd.py", line 822, in execute
    raise GitCommandError(command, status, stderr_value, stdout_value)

Exception Type: GitCommandError at /ui/collection/ddr-densho-383/edit/
Exception Value: Cmd('git') failed due to: exit code(128)
  cmdline: git fetch
  stderr: 'Host key verification failed.
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.'

This is after @pkikawa updated the keys ssh-keygen -f "/home/ddr/.ssh/known_hosts" -R "mits.densho.org" and removed old keys from /home/ddr/.ssh/known_hosts.

[gjost]

@pkikawa What do you get when you log in to the VM as ddr and run ssh git@mits.densho.org?

[pkikawa]

(ddrlocal) ddr@ddrlocal:/home/densho$ ssh git@mits.densho.org Warning: the ECDSA host key for 'mits.densho.org' differs from the key for the IP address '173.225.30.115' Offending key for IP in /home/ddr/.ssh/known_hosts:1 Matching host key in /home/ddr/.ssh/known_hosts:2 Are you sure you want to continue connecting (yes/no)? yes PTY allocation request failed on channel 0 hello densho, this is git@mits3 running gitolite3 3.6.11-2 (Debian) on git 2.20.1

R W C ddr-chi-[0-9]+ R W C ddr-chi-[0-9]+-[0-9]+ R W C ddr-csujad-[0-9]+ ......

[gjost]

Edit /home/ddr/.ssh/known_hosts and remove lines 1 and 2 and try again. You'll get a notice asking you to accept the key (choose yes). Then try ssh git@mits.densho.org again.

[gjost]

Now if you're getting that hello densho, this is git@mits3 running gitolite3 message and a list of R W C ddr-... patterns that means that Gitolite is working. You should then be able to clone repositories as the ddr user thusly: git clone git@mits.densho.org:ddr-densho-10.git, and the web Ui should work.

[pkikawa]

yup, working now. hum... i went thru all those steps before. nothing should've been cached right?

[gjost]

Should not have been. So everything's good now?

[pkikawa]

yup. tripple-checking now but it looks good

[pkikawa]

exporting and gonna test on sara's computer, but if it works then ill close this out