ssh key file not created

[munzeitig]

Hello, I am dealing with a problem, when assigning a new SSH key to user account, the plugin does not create a file in the gitolite keydir directory.

This worked at some moment (have some keys there), but stopped to work. No new keys are being added now. (I upgraded to 1.3 and your fork of the Eric's plugin in meantime, but that is not the cause) In the DB (using PostgreSQL) I can see the new key record is created. But the file is not created. No error is generated anywhere.

Running "rake redmine_git_hosting:update_repositories RAILS_ENV=production" does not help either. I have spent many hours on that and have no more ideas. Any ideas or hints how to debug this effectively further is appreciated.

[kubitron]

Sorry for the dumb question, but -- this plugin will only add a key to the keydir if there is some repository under control of Redmine that needs that key. Thus, if you have a user that isn't on the access list of any repositories, then none of their keys will go into the keydir. So, my question is: are you adding a key to a user that isn't authorized for anything?

[munzeitig]

Thanks for answer. However, the answer is yes - the user has permissions for several git repositories, so the key should be added. Also permissions should be fine - I have checked that. All checks at plugin page (hook, run git as root, root as git, etc.) are also ok. Is there any way how to debug what is happening while the keys should be synced? I am not much familiar with ruby, so any help is appreciated.

Btw. I am now testing this with 2 users, one is LDAP user, the second is internal Redmine user. Does not work for either of them.

[kubitron]

Ok. Another question and a suggestion:

Are you sure that the git repositories of interest have been created? Just because someone is added to the access control for a project doesn't mean that the repository is created. If the repo hasn't been created, then the key won't be added. If you go to the "repository" link of the project settings page and it is asking you to create a repository, then the key wouldn't have been added.

Now, to help debug this, what does your conf/gitolite.conf file look like? Does the project of interest appear there? Further, is there a .git file in ~git/repositories for that project? Note that for <1.4 redmine, there is only one repo/project that is maintained by this plugin and it is named after the project.

[munzeitig]

Hello, yes, the projects repos do exist and they are also listed in conf/gitolite.conf file repo joomla-extensions RW+ = ... repo magento-implementation/officepro RW+ = ...

When going to Repository project in Redmine, it shows the repository ok. And yes, the .git folders do exist. e.g. /home/gitolite/repositories/joomla-extensions.git /home/gitolite/repositories/magento-implementation/officepro.git

However, the key of user in question in not listed in the gitolite.conf file. In public keys of the respective users I can see e.g. keydir/redmine_test_1331461543_359134.pub but although this user is assigned to a project with existing git reposiroty, he is not listed in the gitolite.conf and the key was not stored to the keydir.

[kubitron]

Ok. Do the following: look in log/production.log to see if it ever says something like "adding gitolite key redmine_test_1331461543_359134.pub.

I am wondering if you forgot to pull the latest version of the gitolite-admin repository before looking in the keydir?

Note that the plugin pushes new changes to the gitolite-admin repo with every change in configuration. Thus, if you happen to be looking at an old version of the repo, you might have the wrong idea about what is in the conf file and in keydir....

Also, if you look in ~git/.ssh/authorized_keys, does the physical key appear as a line somewhere there?

[kubitron]

Incidentally, you might try looking in the tmp location that the plugin uses:

/tmp/redmine_git_hosting/git/gitolite-admin/{conf,keydir}

(Assuming that you didn't change the /tmp location by using the absolutely latest version of the plugin, which allows you to change that).

Now, assuming that things still look messed up, can you show me what the log entry is that you get from executing:

rake redmine_git_hosting:fetch_changesets RAILS_ENV=production

Make sure that you only run this as www-data (or whatever your web user might be).

I would like the complete "paragraph" that starts with a reference to "RESYNC_ALL" and ends at the end of the log. Example:

Executing RESYNC_ALL operation on gitolite configuration
Fetching changes from gitolite-admin repository to /tmp/redmine_git_hosting/git-tess/gitolite-admin
Creating MD5 digests for Redmine Git Hosting hook
Digest for post-receive.redmine_gitolite.rb: 08e9b3cf99fa6285331ccdcef8b9a0d3
Our hook is already installed
Updating Hook URL: http://tessellation.cs.berkeley.edu/test-site/redmine/githooks/post-receive

[munzeitig]

Hello, after looking into production.log, I think we are getting closer, as it says there:

git clone ssh://gitolite@xxx/gitolite-admin.git /tmp/redmine_git_hosting/gitolite/gitolite-admin
***> Cloning into /tmp/redmine_git_hosting/gitolite/gitolite-admin...
***> R access for gitolite-admin DENIED to id_rsa

in .gitolite/keydir, I can see id_rsa.pub, but in conf/gitolite.conf, at gitolite-admin project, there is repo gitolite-admin RW+ = gitolite_admin_id_rsa

which is same name as is in .ssh/gitolite_admin_id_rsa.pub. (and configured in plugin settings: /var/www/rails/redmine-1.3/.ssh/gitolite_admin_id_rsa)

So I am wondering, why it is accessing as id_rsa, while the name should be probably gitolite_admin_id_rsa? Should I update the gitolite.conf and change gitolite_admin_id_rsa to id_rsa only?

[kubitron]

Can you show me that whole log entry? It should attempt to reconnect under messed up circumstances by reinstalling the key. However, gl-setup needs to be in your path. Are you using my version of the plugin?

[kubitron]

Look further back in the log for when errors start happening. If you are using my version of the plugin, the errors should be quite clear. I figured from the way you started this query that there were no errors listed in the log.

[kubitron]

Incidentally, the name of the key for gitolite-admin access doesn't need to be called the same thing as the keys you have in REDMINE_ROOT/.ssh/, it just needs to be consistent between the conf file and the keydir file.

[munzeitig]

Sorry, if I misinformed about the log. I was checking the /var/log/httpd/redmine-error.log, not the production.log. Yes, I am using your latest version of the plugin. The full log shows:

...
Processing RepositoriesController#show (for 78.108.152.10 at 2012-03-12 01:38:57) [GET]
  Parameters: {"action"=>"show", "id"=>"joomla-extensions", "controller"=>"repositories"}
Rendering template within layouts/base
Rendering repositories/show
Completed in 347ms (View: 270, DB: 41) | 200 OK [http://xxx/projects/joomla-extensions/repository]

Processing SysController#fetch_changesets (for 78.108.152.10 at 2012-03-12 02:10:22) [GET]
  Parameters: {"key"=>"XXX"}
Executing RESYNC_ALL operation on gitolite configuration
Cloning gitolite-admin repository to /tmp/redmine_git_hosting/gitolite/gitolite-admin
***> Command failed (return 128): env GIT_SSH=/var/www/rails/redmine-1.3/vendor/plugins/redmine_git_hosting/bin/gitolit
e_admin_ssh git clone ssh://gitolite@xxx/gitolite-admin.git /tmp/redmine_git_hosting/gitolite/gitolite-admin
***> Cloning into /tmp/redmine_git_hosting/gitolite/gitolite-admin...
***> R access for gitolite-admin DENIED to id_rsa
***> (Or there may be no repository at the given path. Did you spell it correctly?)
***> fatal: The remote end hung up unexpectedly
***> Failure to access gitolite-admin repository.  Attempting to fix...
  Reestablishing gitolite key
  Deleting and recloning gitolite-admin to /tmp/redmine_git_hosting/gitolite/gitolite-admin
***> Command failed (return 128): env GIT_SSH=/var/www/rails/redmine-1.3/vendor/plugins/redmine_git_hosting/bin/gitolit
e_admin_ssh git clone ssh://gitolite@xxx/gitolite-admin.git /tmp/redmine_git_hosting/gitolite/gitolite-admin
***> Cloning into /tmp/redmine_git_hosting/gitolite/gitolite-admin...
***> R access for gitolite-admin DENIED to id_rsa
***> (Or there may be no repository at the given path. Did you spell it correctly?)
***> fatal: The remote end hung up unexpectedly
***> Failure again.  Probably requires human intervention
***> git_hosting: update_repositories() failed
Completed in 49082ms (View: 3, DB: 3228) | 200 OK [http://xxx/sys/fetch_changesets?key=XXX]

[kubitron]

Ok. Can you try something for me?

su to the gitolite user
cd ~
cp /var/www/rails/redmine-1.3/.ssh/gitolite_admin_id_rsa.pub ~/id_rsa.pub
gl-setup ~/id_rsa.pub

At this point, if you were to do a:

git clone ~/repositories/gitolite-admin.git

You should be able to cd to gitolite-admin/ and look at the conf and keydir directories. I'm curious whether you have (1) any errors during the above commands, (2) whether conf/gitolite.conf still has gitolite_admin_id_rsa as the admin key. The gl-setup should reset this key to be id_rsa. I'm wondering why it doesn't....

[munzeitig]

(1) Well, there was a warning with the gl-setup ~/id_rsa.pub

        ***** WARNING *****
        the following users (pubkey files in parens) do not appear in the config file:
id_rsa(id_rsa.pub)

        ***** WARNING *****
        the following users have no pubkeys:
gitolite_admin_id_rsa

(2) yes, it still has the same name as key

repo    gitolite-admin
    RW+ =   gitolite_admin_id_rsa 

[munzeitig]

I have now solved this by manually updating the line gitolite.conf and changing gitolite_admin_id_rsa to just id_rsa and then running

rake redmine_git_hosting:fetch_changesets RAILS_ENV=production

That has worked and new keys are being added now. Thanks for help. Anyway, I am still wondering, why it stopped to work.

[kubitron]

Understood.

It appears that there is a weird correction case that I missed with gl-setup. I'm glad that you got things working for now.

I'm sorry for leaving you hanging. I don't know what happened originally, but you have uncovered a bug in my code that tries to correct for this problem. I'll leave this bug open for now.

[kubitron]

Ok. I have uploaded a better (I hope) recovery mechanism from this type of failure. It is on my master branch.

I am going to close this bug for now. Let me know if you have problems (you could, for instance, try changing the name of the key in your conf file back to 'gitolite_admin_id_rsa' and then try an update_repositories operation again (after pulling latest changes). It should fix that problem.