search

redmine-git-hosting / redmine_git_hosting

A Redmine plugin which makes configuring your own Git hosting easy ;)
http://redmine-git-hosting.io
MIT License
275 stars 116 forks source link

Repository not found when accessed through HTTPS #568

Closed sdinot closed 4 years ago

sdinot commented 8 years ago

Hi,

Summary:

URL displayed by Redmine:

SSH: ssh://git@forge.example.net/orekit/orekit-main.git
HTTPS: https://sdinot@forge.example.net/git/orekit/orekit-main.git

Cloning attempts:

$ git clone ssh://git@forge.example.net/orekit/orekit-main.git
=> Success
$ git clone https://sdinot@forge.example.net/git/orekit/orekit-main.git
=> Fail, repository not found
$ git clone https://sdinot@forge.example.net:git/orekit/orekit-main.git
=> Success

The difference between the second and the third tries is that "/git/" was replaced by ":git/"

What's wrong?

Detailed explanation:

I just deployed Redmine 3.2 and Redmine Git Hosting 1.2 plugin in a fresh Debian Jessie 8.3 VM, using Apache 2.4.10, PostgreSQL 9.4.5, Gitolite 3.6.1 and RVM 1.26.11:

Environment:
  Redmine version                3.2.0.stable
  Ruby version                   2.1.5-p273 (2014-11-13) [i686-linux]
  Rails version                  4.2.5
  Environment                    production
  Database adapter               PostgreSQL
SCM:
  Git                            2.1.4
  Filesystem                     
  Xitolite                       2.1.4
Redmine plugins:
  redmine_bootstrap_kit          0.2.4
  redmine_git_hosting            1.2.0

I configured SSH and HTTPS access:

SSH server domain: forge.example.net
HTTPS server domain: forge.example.net
Subdirectory for HTTP access: git/
Enable Smart HTTP mode for new repositories by default?: HTTPS Only
Redmine has RW access on all Gitolite repositories: Yes
Directory and URL structure for Redmine-managed repositories: Hierarchical

In the "Config Test" tab of the plugin configuration interface, all items are green.

I created a project named "orekit" and a repository named "orekit-main". Therefore, the interface offers the following access url:

SSH: ssh://git@forge.example.net/orekit/orekit-main.git
HTTPS: https://sdinot@forge.example.net/git/orekit/orekit-main.git

So, I tried to clone this repository through the two protocols:

SSH attempt with offered url:

$ git clone ssh://git@forge.example.net/orekit/orekit-main.git
Cloning into 'orekit-main'...
remote: Counting objects: 51656, done.
remote: Compressing objects: 100% (11199/11199), done.
remote: Total 51656 (delta 31731), reused 51645 (delta 31725)
Receiving objects: 100% (51656/51656), 32.70 MiB | 11.26 MiB/s, done.
Resolving deltas: 100% (31731/31731), done.
Checking connectivity... done.

/opt/gitolite/.gitolite/logs/gitolite-2016-01.log:

2016-01-29.21:58:42     2767    ssh     ARGV=redmine_sdinot_10  SOC=git-upload-pack '/orekit/orekit-main.git'   FROM=192.168.0.12
2016-01-29.21:58:42     2767    pre_git orekit/orekit-main      redmine_sdinot_10       R       any     refs/.*
2016-01-29.21:58:42     2767            system,git,shell,-c,git-upload-pack '/opt/gitolite/repositories/orekit/orekit-main.git'
2016-01-29.21:58:45     2767    END

HTTPS attempt with offered url:

$ git clone https://sdinot@forge.example.net/git/orekit/orekit-main.git
Cloning into 'orekit-main'...
remote: Not Found
fatal: repository 'https://sdinot@forge.example.net/git/orekit/orekit-main.git/' not found

/var/log/apache2/forge.example.net.access_ssl.log:

192.168.0.12 - - [29/Jan/2016:22:01:16 +0100] "GET /git/orekit/orekit-main.git/info/refs?service=git-upload-pack HTTP/1.1" 404 1916 "-" "git/2.5.0"

/opt/redmine/current/log/redmine.log:

I, [2016-01-29T22:01:16.269120 #1235]  INFO -- : Started GET "/git/orekit/orekit-main.git/info/refs?service=git-upload-pack" for 192.168.0.12 at 2016-01-29 22:01:16 +0100

HTTPS attempt with alternative to offered url ("/git/" was replaced by ":git/"):

$ git clone https://sdinot@forge.example.net:git/orekit/orekit-main.git
Cloning into 'orekit-main'...
remote: Counting objects: 51656, done.
remote: Compressing objects: 100% (11199/11199), done.
remote: Total 51656 (delta 31731), reused 51645 (delta 31725)
Receiving objects: 100% (51656/51656), 32.70 MiB | 8.25 MiB/s, done.
Resolving deltas: 100% (31731/31731), done.
Checking connectivity... done.

/var/log/apache2/forge.example.net.access_ssl.log:

192.168.0.12 - - [29/Jan/2016:22:03:06 +0100] "GET /orekit/orekit-main.git/info/refs?service=git-upload-pack HTTP/1.1" 200 2324 "-" "git/2.5.0"
192.168.0.12 - - [29/Jan/2016:22:03:07 +0100] "POST /orekit/orekit-main.git/git-upload-pack HTTP/1.1" 200 35274723 "-" "git/2.5.0"

/opt/redmine/current/log/redmine.log:

I, [2016-01-29T22:03:06.766854 #1235]  INFO -- : Started GET "/orekit/orekit-main.git/info/refs?service=git-upload-pack" for 192.168.0.12 at 2016-01-29 22:03:06 +0100
I, [2016-01-29T22:03:07.149621 #1235]  INFO -- : Started POST "/orekit/orekit-main.git/git-upload-pack" for 192.168.0.12 at 2016-01-29 22:03:07 +0100

I don't understand that is wrong in the configuration. :(

Thanks in advance for your help!

sdinot commented 8 years ago

URL https://sdinot@forge.example.net/orekit/orekit-main.git (without the "git/" subdirectory) works too!

sdinot commented 8 years ago

Any idea? :(

Considering that the url "https://<host>/<project>/<repo>.git" works but not the url "https://<host>/git/<project>/<repo>.git", it is as though the value of "http_server_subdir" was not taken into account.

n-rodriguez commented 8 years ago

Considering that the url "https:////.git" works but not the url "https:///git//.git", it is as though the value of "http_server_subdir" was not taken into account.

This sounds like a bug... I don't have time to take a look on it for now and for the month coming.

baby-gnu commented 8 years ago

+1 I just figure out this issue

The problem is that I want to increase the client_max_body_size in my nginx configuration file, but just for ^/git/.

baby-gnu commented 8 years ago

This one is really annoying, I can not upgrade my plugin without breaking every user clone URLs :confused:.

sdinot commented 8 years ago

Any news? :(

I just reproduced the issue with an instance of the last released version of Redmine (3.3.0) using the last released version of the plugin (1.2.1):

Plugin configuration:

SSH server domain: forge.example.net
HTTP server domain: forge.example.net
HTTPS server domain: forge.example.net
Subdirectory for HTTP access: git/
Enable Smart HTTP mode for new repositories by default? HTTPS Only

Try: $ git clone ssh://git@forge.example.net/project/project-main.git => Success $ git clone https://sdinot@forge.example.net/git/project/project-main.git => Fail, repository not found

But it works if I empty the "Subdirectory for HTTP access" field:

$ git clone https://sdinot@forge.example.net/project/project-main.git => Success

baby-gnu commented 8 years ago

Hello,

I'm trying to look at this issue, it may be related to the Grack route:

# Enable SmartHTTP Grack support
mount Grack::Bundle.new({}), at: '/', constraints: lambda { |request| /[-\/\w\.]+\.git\//.match(request.path_info) }, via: [:get, :post]

in 0.6.3 it was:

    # SMART HTTP
    match ':repo_path/*git_params', :prefix => GitHostingConf.http_server_subdir, :repo_path => /([^\/]+\/)*?[^\/]+\.git/, :to => 'smart_http#index'

in 0.7.10 it was:

  # SMART HTTP
  match ':repo_path/*git_params', :prefix    => RedmineGitolite::ConfigRedmine.get_setting(:http_server_subdir),
                                  :repo_path => /([^\/]+\/)*?[^\/]+\.git/,
                                  :to        => 'smart_http#index',
                                  :via       => [:get, :post]

I don't know how to express the :prefix used before with the mount :-/

Regards.

baby-gnu commented 8 years ago

According to rails documentation mounting with at make the rake application receive requests at the root path.

So I think we should start with:

mount Grack::Bundle.new({}),
      at: (RedmineGitHosting::Config.http_server_subdir rescue '/'),
      constraints: lambda { |request| /[-\/\w\.]+\.git\//.match(request.path_info) },
      via: [:get, :post]

so Grack will always handle requests at the root path.

Unfortunately, this is not sufficient, now I have the following in my logs:

NoMethodError (undefined method `chomp' for true:TrueClass):
  plugins/redmine_git_hosting/lib/grack/auth.rb:44:in `auth!'
  plugins/redmine_git_hosting/lib/grack/auth.rb:22:in `call'

Regards.

baby-gnu commented 8 years ago

In log/git_hosting.log I have 

2016-10-05 13:36:20 +0200 [ERROR] Problems while getting SmartHttp params

I changed debug logging to error in lib/redmine_git_hosting/utils/exec.rb and now I have:

2016-10-05 13:47:34 +0200 [ERROR] Non-zero exit code pid 11845 exit 1 for `sudo -n -u git -i env GL_LIBDIR=/home/git/bin/lib/ GL_REPO=testing GL_USER= git --git-dir repositories/testing.git config http.uploadpack`
2016-10-05 13:47:34 +0200 [ERROR] Problems while getting SmartHttp params
baby-gnu commented 7 years ago

Any idea?

baby-gnu commented 7 years ago

Replying to myself, I finally found the answer, as git user I need to do

git config --global http.uploadpack /usr/lib/git-core/git-upload-pack

I think the plugin should try to default on several PATHes.

baby-gnu commented 7 years ago

And to push you need:

git config --global http.receivepack /usr/lib/git-core/git-receive-pack
baby-gnu commented 7 years ago

Now I can clone with #621 and the global git configuration set for user git but I can't push, I have the following error:

remote: Not Found
fatal: repository 'https://user@redmine.example.net/git/sandbox.git/' not found

I see nothing in logs except:

So, redmine return a 404 but even in debug mode git_hosting.log contains nothing :-/

According to sudo logs the latest command run is update-server-info:

nov. 24 02:37:53 server sudo[10223]:  redmine : TTY=unknown ; PWD=/home/redmine/redmine-3.3.1 ; USER=git ; COMMAND=/bin/bash -c env GL_LIBDIR=/home/git/bin/lib/ GL_REPO=sandbox GL_USER=redmine_dad_17 git --git-dir repositories/sandbox.git update-server-info

Do you have any hints?

baby-gnu commented 7 years ago

I found the push issue, it's due to the way https://github.com/jbox-web/grack (v0.2) works compared to https://github.com/gitlabhq/grack (v2.0.2).

The problem comes from get_config_setting.

Your older version requires:

The new code of has_access check if this is a string with the corresponding service name in it.

So for now I must configure as the following:

git config --global http.uploadpack /usr/lib/git-core/git-upload-pack
git config --global http.receivepack true
n-rodriguez commented 7 years ago

Many thanks baby-gnu! i owe you a pack!

n-rodriguez commented 7 years ago

@baby-gnu :

About http.upload-pack : actually the code has not really changed before and after the refactoring :

Here get_config_setting (v0.x ) Here config_setting (v2.x )

And Here has_access (v0.x ) Here has_access (v2.x )

The biggest issue here was the prefix not set in the routes.rb file. But you must restart Redmine after the change (and also check that SmartHTTP is enabled for the repository)

baby-gnu commented 7 years ago

Thanks for the note, for now it's working with

git config --global http.uploadpack /usr/lib/git-core/git-upload-pack
git config --global http.receivepack true

I think the new code will requires 

git config --global http.uploadpack /usr/lib/git-core/git-upload-pack
git config --global http.receivepack /usr/lib/git-core/git-receive-pack

Regards.

n-rodriguez commented 7 years ago

Well, this is weird because I don't need this to make it works...

ahmedosama80 commented 7 years ago

658

my Redmine is running in sub URI is it can cause such problem

[ERROR] Problems while getting SmartHttp params

NoMethodError (undefined method chomp' for true:TrueClass): plugins/redmine_git_hosting/lib/grack/auth.rb:44:inauth!' plugins/redmine_git_hosting/lib/grack/auth.rb:22:in `call'

mnencia commented 7 years ago

I've just updated my installation to a new environment and I've had to execute the following command to make https clone/push working again.

git config --global http.uploadpack /usr/lib/git-core/git-upload-pack
mnencia commented 7 years ago

I've finally found what happened in my Redmine installation. One of the user uploaded an invalid SSH key that was breaking the gitolite config compilation.

2017-05-25.14:53:28    11490        system,gitolite trigger POST_COMPILE
2017-05-25.14:53:28    11490        system,/srv/git/bin/triggers/post-compile/ssh-authkeys,POST_COMPILE
2017-05-25.14:53:28    11490    warn    Use of uninitialized value $output in scalar chomp at /srv/git/bin/lib/Gitolite/Common.pm line 352.<<newline>>
2017-05-25.14:53:28    11490    warn    Use of uninitialized value $output in pattern match (m//) at /srv/git/bin/lib/Gitolite/Common.pm line 357.<<newline>>
2017-05-25.14:53:28    11490    warn    Use of uninitialized value $output in pattern match (m//) at /srv/git/bin/lib/Gitolite/Common.pm line 357.<<newline>>
2017-05-25.14:53:28    11490    warn    Use of uninitialized value $output in concatenation (.) or string at /srv/git/bin/triggers/post-compile/ssh-authkeys line 116.<<newline>>
2017-05-25.14:53:28    11490    die    fingerprinting failed for 'keydir/redmine_git_hosting/redmine_<REDACTED>_306/redmine_<REDACTED>_306.pub':
2017-05-25.14:53:28    11490        system() failed,/srv/git/bin/triggers/post-compile/ssh-authkeys,POST_COMPILE,-> 6400
2017-05-25.14:53:28    11490        system() failed,gitolite trigger POST_COMPILE,-> 6400
2017-05-25.14:53:28    11490    END

As result of that, the new repositories where created, but without the necessary git configuration values. Once I've removed the bad key everything started working again even without the global http.uploadpack setting.

ayanbon commented 5 years ago

Hi i am having problem cloning repo on termux

PowerKiKi commented 4 years ago

As discussed in https://github.com/jbox-web/redmine_git_hosting/issues/732#issuecomment-532362560, issues related to Redmine < 4.0 or severely outdated issues are being closed to help clean up the issue tracker.

If this issue is still relevant to you and you are running Redmine >= 4.0, please open a new issue including all new relevant information.