search

spuder / puppet-gitlab

A puppet module to install gitlab 7
Apache License 2.0
36 stars 44 forks source link

Working fix for CentOS with new 7.10.0 omnibus packages - Debian could e... #141

Closed ahuffman closed 9 years ago

ahuffman commented 9 years ago

...asily be written from this work, but would need testing.

ahuffman commented 9 years ago

Have you looked into these 7.10.0 changes yet?

spuder commented 9 years ago

I'm getting this error on centos 6.5

Debug: Executing 'chkconfig --list postfix | grep -q 'on' 2>/dev/null '
Debug: Executing '/sbin/service postfix status'
Debug: Class[Gitlab::Packages]: The container Stage[main] will propagate my refresh event
Debug: Exec[download gitlab](provider=posix): Executing 'curl -o /var/tmp/gitlab-ce-7.10.0~omnibus.1.x86_64.rpm https://downloads-packages.s3.amazonaws.com/centos-6.5/gitlab-ce-7.10.0~omnibus.1.x86_64.rpm'
Debug: Executing 'curl -o /var/tmp/gitlab-ce-7.10.0~omnibus.1.x86_64.rpm https://downloads-packages.s3.amazonaws.com/centos-6.5/gitlab-ce-7.10.0~omnibus.1.x86_64.rpm'
Notice: /Stage[main]/Gitlab::Install/Exec[download gitlab]/returns: executed successfully
Debug: /Stage[main]/Gitlab::Install/Exec[download gitlab]: The container Class[Gitlab::Install] will propagate my refresh event
Debug: Prefetching rpm resources for package
Debug: Executing '/bin/rpm --version'
Debug: Executing '/bin/rpm -qa --nosignature --nodigest --qf '%{NAME} %|EPOCH?{%{EPOCH}}:{0}| %{VERSION} %{RELEASE} %{ARCH} :DESC: %{SUMMARY}\n''
Debug: Executing '/bin/rpm -q gitlab-ce --nosignature --nodigest --qf %{NAME} %|EPOCH?{%{EPOCH}}:{0}| %{VERSION} %{RELEASE} %{ARCH} :DESC: %{SUMMARY}\n'
Debug: Executing '/bin/rpm -q --qf %{NAME} %|EPOCH?{%{EPOCH}}:{0}| %{VERSION} %{RELEASE} %{ARCH} :DESC: %{SUMMARY}\n -p /var/tmp/gitlab-ce-7.10.0~omnibus.1.x86_64.rpm'
Error: Could not update: Puppet::Error
Error: /Stage[main]/Gitlab::Install/Package[gitlab-ce]/ensure: change from absent to latest failed: Could not update: Puppet::Error
Debug: /Stage[main]/Gitlab::Install/Package[gitlab-ce]: Nothing to manage: no ensure and the resource doesn't exist
Debug: Class[Gitlab::Install]: The container Stage[main] will propagate my refresh event
Notice: /Stage[main]/Gitlab::Config/Exec[stop gitlab]: Dependency Package[gitlab-ce] has failures: true

I'll have to look into this more. Strange that the download has a ~ character in it. 

curl -o /var/tmp/gitlab-ce-7.10.0~omnibus.1.x86_64.rpm https://downloads-packages.s3.amazonaws.com/centos-6.5/gitlab-ce-7.10.0~omnibus.1.x86_64.rpm
ahuffman commented 9 years ago

@igalic I understand that, but the problem is that the RPM and DEB filenames change as well as the package name. In my opinion it makes it trickier to use a name in this case.

You'd have to do something like this:

$gitlab_pkg = $::gitlab::gitlab_branch ? {
  7.10.0 => 'gitlab-ce',
  default => 'gitlab'
}
$omnibus_release = $::gitlab::gitlab_branch ? {
  7.10.0 => "omnibus.${::gitlab::omnibus_build_ver}.x86_64.rpm",
  default => "omnibus-1.el${cent_maj_version}.x86_64.rpm"
}
$omnibus_filename = $::gitlab::gitlab_branch ? {
  7.10.0 => "gitlab-ce${url_separator}${::gitlab::gitlab_branch}~${omnibus_release}",
  default => "gitlab${url_separator}${::gitlab::gitlab_branch}_${omnibus_release}"
}
ahuffman commented 9 years ago

Ok, I just tested this on Debian (ubuntu 12.04) and fixed a problem with the omnibus release naming convention. It now works with both CentOS and Debian.

Here's the config I'm using, and due to gitlab making a lot of bug changes the parameter omnibus_build_ver that I added is very relevant and necessary. The default is set to 1, which is normally fine, but with bug re-releases this increments from 1-1 to 2-1 etc.

Here's the config I'm using:

class profiles::linux::gitlabtest {

 #Manages gitlab omnibus install
  class { '::gitlab':
    puppet_manage_config                                 => true,
    puppet_manage_backups                                => true,
    puppet_manage_packages                               => true,
    gitlab_branch                                        => '7.10.1',
    omnibus_build_ver                                    => '2-1',
    gitlab_release                                       => 'basic',
    external_url                                         => 'http://gitlab-test.com',
    backup_keep_time                                     => '604800', #7 days
    gitlab_default_projects_limit                        => '100',
    gravatar_enabled                                     => true,
    gitlab_default_can_create_group                      => false,
    gitlab_username_changing_enabled                     => false,
    gitlab_signup_enabled                                => false,
    gitlab_default_projects_features_visibility_level    => 'private',
    git_data_dir                                         => '/var/opt/gitlab/git-data',
    backup_path                                          => '/var/opt/gitlab/backups',
    gitlab_email_from                                    => 'git@gitlab-test.com',
    ldap_enabled                                         => true,
    ldap_servers                                         => ['
     {
       "primary" => {
         "label"                           => "MyLDAP",
         "host"                            => "myldap.com",
         "port"                            => "389",
         "uid"                             => "cn",
         "method"                          => "plain",
         "bind_dn"                         => "cn=admin,o=com",
         "password"                        => "mypassword",
         "active_directory"                => false,
         "allow_username_or_email_login"   => true,
         "block_auto_created_users"        => true,
         "base"                            => "ou=users,o=com",
         "admin_group"                     => "",
         "sync_ssh_keys"                   => false,
         "sync_time"                       => 3600
       }
     }'],
  }

}
ahuffman commented 9 years ago

Any update on this merge? It works fine on ubuntu and centos. Not sure why it fails the build tests.

spuder commented 9 years ago

I haven't had a chance to look at it yet. Travis is failing because they updated their images.

On Mon, May 11, 2015 at 8:38 AM Andrew J Huffman notifications@github.com wrote:

Any update on this merge? It works fine on ubuntu and centos. Not sure why it fails the build tests.

— Reply to this email directly or view it on GitHub https://github.com/spuder/puppet-gitlab/pull/141#issuecomment-100928575.

spuder commented 9 years ago

I'm setting up an environment to test this.

Could you do a squash on the commits?

ahuffman commented 9 years ago

Just squashed them down to 1. Let me know if you have any problems. I was looking at one of the other issues about having the gitlab package installed and trying to upgrade to gitlab-ce package. I think we still need to add one piece to ensure gitlab is not installed if doing branch >=7.10.0. Let me know how the testing goes. I think I manually uninstalled the gitlab pkg, but can't recall now, so that piece would need to be thrown in.

ahuffman commented 9 years ago

So this is useless to test at this point. As you know gitlab changed the location of their builds once again.

I've made an attempt at working this out in branch issue_140 where the naming conventions changed and installations break.

I've reworked the install.pp, but I'm having trouble getting the config.pp to work due to the subscribe on the gitlab/gitlab-ce package. I tried throwing a class wrapper around the packagecloud installer and subscribing to that class, but I'm getting and error that is slightly over my head, and not sure how to correct, or if I'm defining something incorrectly.

If you get a chance to look at the new code in issue_140 over the next few days/week let me know what you think. I had to majorly rework the install.pp to retain backward compatibility while trying to handle the new installation mechanisms from gitlab's repository. Basically I need an assist to get it nailed down.

On Tue, May 12, 2015 at 12:56 AM, Spencer Owen notifications@github.com wrote:

I'm setting up an environment to test this.

Could you do a squash on the commits?

— Reply to this email directly or view it on GitHub https://github.com/spuder/puppet-gitlab/pull/141#issuecomment-101130389.

Regards,

Andrew

spuder commented 9 years ago

Thanks for all the work on this. I'll look at issue_140 soon

ahuffman commented 9 years ago

It's working now!!! It was a documentation issue on the packagecloud module config.

I had to specify the server_address parameter because the repo is at packages.gitlab.com not the default packagecloud.io.

        packagecloud::repo { "gitlab/gitlab-ce":
          type    => 'deb',
          server_address => 'https://packages.gitlab.com',
          require => Package['wget'],
        }

I pushed the final commit up to branch_140 after a quick CentOS 6.6 test. It upgraded perfectly from 7.10.0_2-1 on my production system.

Let me know if you have any questions, but it should be a functional fix at this point. I would still have to test backward compatibility for the older version installs, but I don't have the bandwidth to do that at the moment.

-Andrew

On Wed, May 20, 2015 at 12:12 PM, Spencer Owen notifications@github.com wrote:

Thanks for all the work on this. I'll look at issue_140 soon

— Reply to this email directly or view it on GitHub https://github.com/spuder/puppet-gitlab/pull/141#issuecomment-103943345.

Regards,

Andrew

spuder commented 9 years ago

Rather than just merging this as is, I'm looking at incorporating the package cloud repo code into a much larger overhaul.

This has been pretty low priority for me since I switched jobs, and I don't use puppet every day anymore.

How did you work around issue https://github.com/computology/computology-packagecloud/issues/11 ?

ahuffman commented 9 years ago

It's an ugly module really. Igor is right, they really should be using the puppet resources instead of reinventing their own for some reason. I'm not really sure why.

So I took it upon myself to make the changes on the computology/packagecloud repo and push the commit up on the pull request. We'll see if they're interested in taking the changes.

Maybe we'll have to permanently fork and build. There is some logic in there that determines paths. I was able to easily amend the custom junk they were doing with templates and translate into yumrepos and apt::source.

Check out the changes here: https://github.com/ahuffman/computology-packagecloud/commit/49e1ff1894730e718ef2779c25103966978e87de

Feel free to test out the debian side of things. I don't really have anything debian here or the time to fuss about doing so. I think what I have written should do the trick.

Thanks, Andrew

On Mon, Jun 1, 2015 at 8:52 PM, Spencer Owen notifications@github.com wrote:

Rather than just merging this as is, I'm looking at incorporating the package cloud repo code into a much larger overhaul.

This has been pretty low priority for me since I switched jobs, and I don't use puppet every day anymore.

How did you work around issue computology/computology-packagecloud#11 https://github.com/computology/computology-packagecloud/issues/11 ?

— Reply to this email directly or view it on GitHub https://github.com/spuder/puppet-gitlab/pull/141#issuecomment-107757900.

Regards,

Andrew

spuder commented 9 years ago

Yikes, your improvements are much better than their module. I only see 2 pull requests, and I don't think either one includes the apt overhaul.

https://github.com/computology/computology-packagecloud/pulls

I will make sure this works with both debian and cent. There is a debian vagrant box inside the puppet module.

ahuffman commented 9 years ago

It seems to be there for me. The pull request is called "Fixing repeated node changes on yumcache by making this a refreshonly". There's 3 commits to it.

Thanks for doing some testing work on this. It'd be good to get this thing cleaned up a little bit if you're going to have a dependency to that module. Otherwise a complete rewrite of the packagecloud module might be a better path for us and gitlab. I think the new changes make it more acceptable, but packagecloud module doesn't do a whole lot other than figure out what paths to use and insert into the standard apt and yumrepo resources.

Yikes, your improvements are much better than their module. I only see 2 pull requests, and I don't think either one includes the apt overhaul.

https://github.com/computology/computology-packagecloud/pulls

I will make sure this works with both debian and cent. There is a debian vagrant box inside the puppet module.

— Reply to this email directly or view it on GitHub https://github.com/spuder/puppet-gitlab/pull/141#issuecomment-108029478.