puppetlabs / r10k

Smarter Puppet deployment
Other
800 stars 354 forks source link

Question regarding git and pack files with r10k #426

Closed audimann closed 3 years ago

audimann commented 9 years ago

Hi Folks, My cachedir containing my repo has 150MB as 'pack file'

ll objects/pack/
total 150M
drwxr-xr-x 2 root root 4.0K May 20 14:34 .
drwxr-xr-x 4 root root 4.0K May 20 14:33 ..
-r--r--r-- 1 root root 390K May 20 14:34 pack-82ddb7b456b100883535758735bb91045c5a3ed8.idx
-r--r--r-- 1 root root 149M May 20 14:34 pack-82ddb7b456b100883535758735bb91045c5a3ed8.pack

When I execute r10k deploy environments it copies the pack file into each environment (I have about 80 in /etc/puppet/environments), creating many gigabytes. I thought the idea was that the history of the repo was supposed to be kept only once (in cachedir) and the environments would not contain the pack files (using the 'alternate' file which actually exists in /etc/puppet/environments//.git/objects/info/alternates) Am I doing something wrong or do I simply misunderstand the concept? I hope this is not a too stupid question. I used r10k 1.2.1 and 1.3.1 and 1.5.1 getting the same result. Thanks in advance Michael

adrienthebo commented 9 years ago

Your understanding is generally correct; those packfiles shouldn't be copied into each module. What version of Git are you using?

audimann commented 9 years ago

Hi Adrien,

The node (puppet master) running r10k is RHEL 6.5 and has git version 1.7.1 installed. The repository itself is living in “Stash”.

Thanks a bunch

  Michael

From: Adrien Thebo [mailto:notifications@github.com] Sent: 26 May 2015 19:07 To: puppetlabs/r10k Cc: Michael EHRIG Subject: Re: [r10k] Question regarding git and pack files with r10k (#426)

Your understanding is generally correct; those packfiles shouldn't be copied into each module. What version of Git are you using?

— Reply to this email directly or view it on GitHubhttps://github.com/puppetlabs/r10k/issues/426#issuecomment-105604776.

IMPORTANT - CONFIDENTIALITY NOTICE - This e-mail is intended only for the use of the individual or entity shown above as addressees . It may contain information which is privileged, confidential or otherwise protected from disclosure under applicable laws . If the reader of this transmission is not the intended recipient, you are hereby notified that any dissemination, printing, distribution, copying, disclosure or the taking of any action in reliance on the contents of this information is strictly prohibited. If you have received this transmission in error, please immediately notify us by reply e-mail or using the address below and delete the message and any attachments from your system . Amadeus Data Processing GmbH Geschäftsführer: Dr. Wolfgang Krips Sitz der Gesellschaft: Erding HR München 48 199 Berghamer Strasse 6 85435 Erding Germany

adrienthebo commented 9 years ago

This might mean that the Git alternate object database paths aren't being set correctly, could you go into one of the repositories showing this issue and provide the content of .git/objects/info/alternates ?

audimann commented 9 years ago

Hi Adrien,

Sure:

cat .git/objects/info/alternates
/tmp/r10k/http---myuser-mypassword@rndwww.nce.amadeus.net-git-scm-dzb-puppet-modules.git/objects<mailto:/tmp/r10k/http---myuser-mypassword@rndwww.nce.amadeus.net-git-scm-dzb-puppet-modules.git/objects>

This directory exists :

[root@btpv0100 /etc/puppet/environments/prd]# ll `cat .git/objects/info/alternates` | tail
drwxr-xr-x   2 root root 4.0K May 26 09:08 f2
drwxr-xr-x   2 root root 4.0K May 21 16:45 f3
drwxr-xr-x   2 root root 4.0K May 21 16:45 f5
drwxr-xr-x   2 root root 4.0K May 21 15:53 f6
drwxr-xr-x   2 root root 4.0K May 21 15:53 fa
drwxr-xr-x   2 root root 4.0K May 21 13:45 fb
drwxr-xr-x   2 root root 4.0K May 26 09:54 fc
drwxr-xr-x   2 root root 4.0K May 21 12:51 ff
drwxr-xr-x   2 root root 4.0K May 20 19:37 info
drwxr-xr-x   2 root root 4.0K May 20 19:37 pack

Thanks Michael

From: Adrien Thebo [mailto:notifications@github.com] Sent: 26 May 2015 20:06 To: puppetlabs/r10k Cc: Michael EHRIG Subject: Re: [r10k] Question regarding git and pack files with r10k (#426)

This might mean that the Git alternate object database paths aren't being set correctly, could you go into one of the repositories showing this issue and provide the content of .git/objects/info/alternates ?

— Reply to this email directly or view it on GitHubhttps://github.com/puppetlabs/r10k/issues/426#issuecomment-105620803.

IMPORTANT - CONFIDENTIALITY NOTICE - This e-mail is intended only for the use of the individual or entity shown above as addressees . It may contain information which is privileged, confidential or otherwise protected from disclosure under applicable laws . If the reader of this transmission is not the intended recipient, you are hereby notified that any dissemination, printing, distribution, copying, disclosure or the taking of any action in reliance on the contents of this information is strictly prohibited. If you have received this transmission in error, please immediately notify us by reply e-mail or using the address below and delete the message and any attachments from your system . Amadeus Data Processing GmbH Geschäftsführer: Dr. Wolfgang Krips Sitz der Gesellschaft: Erding HR München 48 199 Berghamer Strasse 6 85435 Erding Germany

adrienthebo commented 9 years ago

This seems odd:

cat .git/objects/info/alternates
/tmp/r10k/http---myuser-mypassword@rndwww.nce.amadeus.net-git-scm-dzb-puppet-modules.git/objects<mailto:/tmp/r10k/http---myuser-mypassword@rndwww.nce.amadeus.net-git-scm-dzb-puppet-modules.git/objects>

Was that your email client adding formatting to the email or was that the raw content of the file?

audimann commented 9 years ago

Hi Adrien,

Sorry, must have been outlook foo, creating a hyperlink:

cat .git/objects/info/alternates

/tmp/r10k/http---myuser-mypassword@rndwww.nce.amadeus.net-git-scm-dzb-puppet-modules.git/objects

The <mailto stuff was apparently created by outlook, detecting the ‘@’ character.

Thanks again Michael

From: Adrien Thebo [mailto:notifications@github.com] Sent: 26 May 2015 21:27 To: puppetlabs/r10k Cc: Michael EHRIG Subject: Re: [r10k] Question regarding git and pack files with r10k (#426)

This seems odd:

cat .git/objects/info/alternates

/tmp/r10k/http---myuser-mypassword@rndwww.nce.amadeus.net-git-scm-dzb-puppet-modules.git/objectsmailto:/tmp/r10k/http---myuser-mypassword@rndwww.nce.amadeus.net-git-scm-dzb-puppet-modules.git/objectsmailto:/tmp/r10k/http---myuser-mypassword@rndwww.nce.amadeus.net-git-scm-dzb-puppet-modules.git/objects%3cmailto:/tmp/r10k/http---myuser-mypassword@rndwww.nce.amadeus.net-git-scm-dzb-puppet-modules.git/objects

Was that your email client adding formatting to the email or was that the raw content of the file?

— Reply to this email directly or view it on GitHubhttps://github.com/puppetlabs/r10k/issues/426#issuecomment-105641611.

IMPORTANT - CONFIDENTIALITY NOTICE - This e-mail is intended only for the use of the individual or entity shown above as addressees . It may contain information which is privileged, confidential or otherwise protected from disclosure under applicable laws . If the reader of this transmission is not the intended recipient, you are hereby notified that any dissemination, printing, distribution, copying, disclosure or the taking of any action in reliance on the contents of this information is strictly prohibited. If you have received this transmission in error, please immediately notify us by reply e-mail or using the address below and delete the message and any attachments from your system . Amadeus Data Processing GmbH Geschäftsführer: Dr. Wolfgang Krips Sitz der Gesellschaft: Erding HR München 48 199 Berghamer Strasse 6 85435 Erding Germany

adrienthebo commented 9 years ago

That path looks generally correct, but the fact that the repositories are being cached in /tmp might be a little strange. What sort of filesystem is /tmp? The repositories that r10k is creating are expected to be long lived, so using /tmp might have some interesting behaviors; (https://github.com/puppetlabs/r10k/blob/master/doc/git/cloning-and-mirroring.mkd) has more information on this.

audimann commented 9 years ago

Hi Adrien,

OK, I changed the directory from /tmp to /opt (ext4):

[root@btxv0100 ~]# mount | grep /opt
/dev/mapper/vg00-opt on /opt type ext4 (rw)

I removed the puppet environment, the /tmp/r10k directory

[root@btxv0100 ~]# rm -rf /etc/puppet/environments/*
[root@btxv0100 ~]# rm -rf /tmp/r10k/

Then I changed the location in /etc/r10k from /tmp/r10k to /opt/r10k:

[root@btxv0100 ~]# cat /etc/r10k.yaml
# The location to use for storing cached Git repos
:cachedir: '/opt/r10k'

# A list of git repositories to create
:sources:
  # This will clone the git repository and instantiate an environment per
  # branch in /etc/puppet/environments
  :puppet-modules:
    remote: 'http://myuser:mypassword@rndwww.nce.amadeus.net/git/scm/dzb/puppet-modules.git'
    basedir: '/etc/puppet/environments'

The I redeployed using the command:

[root@btxv0100 ~]# r10k deploy environment –v
[R10K::Task::Deployment::DeployEnvironments - INFO] Loading environments from all sources
[R10K::Deployment::Environment - WARN] Environment "bugfix/DSS-108-snmp_conf" contained non-word characters; sanitizing to "bugfix_DSS_108_snmp_conf"
[R10K::Deployment::Environment - WARN] Environment "bugfix/DSS-112-error-hpacucli-cpqacuxe" contained non-word characters; sanitizing to "bugfix_DSS_112_error_hpacucli_cpqacuxe"
[R10K::Deployment::Environment - WARN] Environment "bugfix/DSS-113-couchbase-logfiles-perm" contained non-word characters; sanitizing to "bugfix_DSS_113_couchbase_logfiles_perm"
[R10K::Deployment::Environment - WARN] Environment "bugfix/DSS-116-couchbase_app_value" contained non-word characters; sanitizing to "bugfix_DSS_116_couchbase_app_value"
[R10K::Deployment::Environment - WARN] Environment "bugfix/DSS-136-firecell-location" contained non-word characters; sanitizing to "bugfix_DSS_136_firecell_location"
[R10K::Deployment::Environment - WARN] Environment "bugfix/DSS-137-XFS-missing" contained non-word characters; sanitizing to "bugfix_DSS_137_XFS_missing"
[R10K::Deployment::Environment - WARN] Environment "bugfix/DSS-138_remove_reboot" contained non-word characters; sanitizing to "bugfix_DSS_138_remove_reboot"
[R10K::Deployment::Environment - WARN] Environment "bugfix/DSS-140-lvm-regex-fix" contained non-word characters; sanitizing to "bugfix_DSS_140_lvm_regex_fix"
[R10K::Deployment::Environment - WARN] Environment "bugfix/DSS-152-mapr-lvm" contained non-word characters; sanitizing to "bugfix_DSS_152_mapr_lvm"
[R10K::Deployment::Environment - WARN] Environment "bugfix/DSS-153-remove-version-check" contained non-word characters; sanitizing to "bugfix_DSS_153_remove_version_check"
[R10K::Deployment::Environment - WARN] Environment "bugfix/DSS-154-remove-stringify_facts-false-line" contained non-word characters; sanitizing to "bugfix_DSS_154_remove_stringify_facts_false_line"
[R10K::Deployment::Environment - WARN] Environment "bugfix/DSS-155-var-log-messages-not-compressed" contained non-word characters; sanitizing to "bugfix_DSS_155_var_log_messages_not_compressed"
[R10K::Deployment::Environment - WARN] Environment "bugfix/DSS-159-rhel66-upgrade" contained non-word characters; sanitizing to "bugfix_DSS_159_rhel66_upgrade"
[R10K::Deployment::Environment - WARN] Environment "bugfix/DSS-225-firecell-timeout" contained non-word characters; sanitizing to "bugfix_DSS_225_firecell_timeout"
[R10K::Deployment::Environment - WARN] Environment "bugfix/DSS-230-cmdb-url-update" contained non-word characters; sanitizing to "bugfix_DSS_230_cmdb_url_update"
[R10K::Deployment::Environment - WARN] Environment "bugfix/DSS-234-wrong-couchbase-permission" contained non-word characters; sanitizing to "bugfix_DSS_234_wrong_couchbase_permission"
[R10K::Deployment::Environment - WARN] Environment "bugfix/DSS-235-transparenthugepages-not-working" contained non-word characters; sanitizing to "bugfix_DSS_235_transparenthugepages_not_working"
[R10K::Deployment::Environment - WARN] Environment "bugfix/dss-107" contained non-word characters; sanitizing to "bugfix_dss_107"
[R10K::Deployment::Environment - WARN] Environment "bugfix/epel" contained non-word characters; sanitizing to "bugfix_epel"
[R10K::Deployment::Environment - WARN] Environment "bugfix/mapr-storage-disable" contained non-word characters; sanitizing to "bugfix_mapr_storage_disable"
[R10K::Deployment::Environment - WARN] Environment "bugfix/mapr-storage-disable2" contained non-word characters; sanitizing to "bugfix_mapr_storage_disable2"
[R10K::Deployment::Environment - WARN] Environment "feature/CDP-770-bashrc-change" contained non-word characters; sanitizing to "feature_CDP_770_bashrc_change"
[R10K::Deployment::Environment - WARN] Environment "feature/DSS-111-filepermssions" contained non-word characters; sanitizing to "feature_DSS_111_filepermssions"
[R10K::Deployment::Environment - WARN] Environment "feature/DSS-114-couchbase-login-shell" contained non-word characters; sanitizing to "feature_DSS_114_couchbase_login_shell"
[R10K::Deployment::Environment - WARN] Environment "feature/DSS-115-cb-backup-restore-wrapper" contained non-word characters; sanitizing to "feature_DSS_115_cb_backup_restore_wrapper"
[R10K::Deployment::Environment - WARN] Environment "feature/DSS-118-firmware-version-fact" contained non-word characters; sanitizing to "feature_DSS_118_firmware_version_fact"
[R10K::Deployment::Environment - WARN] Environment "feature/DSS-123-Core-File-Permissions" contained non-word characters; sanitizing to "feature_DSS_123_Core_File_Permissions"
[R10K::Deployment::Environment - WARN] Environment "feature/DSS-132-puppet-acl" contained non-word characters; sanitizing to "feature_DSS_132_puppet_acl"
[R10K::Deployment::Environment - WARN] Environment "feature/DSS-132-puppet-couchbase-logfilepermissions" contained non-word characters; sanitizing to "feature_DSS_132_puppet_couchbase_logfilepermissions"
[R10K::Deployment::Environment - WARN] Environment "feature/DSS-134-puppet-bios-release-fact" contained non-word characters; sanitizing to "feature_DSS_134_puppet_bios_release_fact"
[R10K::Deployment::Environment - WARN] Environment "feature/DSS-135-topodb-module" contained non-word characters; sanitizing to "feature_DSS_135_topodb_module"
[R10K::Deployment::Environment - WARN] Environment "feature/DSS-144_couchbase3" contained non-word characters; sanitizing to "feature_DSS_144_couchbase3"
[R10K::Deployment::Environment - WARN] Environment "feature/DSS-224-test" contained non-word characters; sanitizing to "feature_DSS_224_test"
[R10K::Deployment::Environment - WARN] Environment "feature/DSS-229-controlm" contained non-word characters; sanitizing to "feature_DSS_229_controlm"
[R10K::Deployment::Environment - WARN] Environment "feature/DSS-231-mdb-logrotation" contained non-word characters; sanitizing to "feature_DSS_231_mdb_logrotation"
[R10K::Deployment::Environment - WARN] Environment "feature/DSS-232-network-config" contained non-word characters; sanitizing to "feature_DSS_232_network_config"
[R10K::Deployment::Environment - WARN] Environment "feature/DSS-233-lom" contained non-word characters; sanitizing to "feature_DSS_233_lom"
[R10K::Deployment::Environment - WARN] Environment "feature/DSS-237-couchbase-in-DRsite" contained non-word characters; sanitizing to "feature_DSS_237_couchbase_in_DRsite"
[R10K::Deployment::Environment - WARN] Environment "feature/DSS-33-mongodb-qradar" contained non-word characters; sanitizing to "feature_DSS_33_mongodb_qradar"
[R10K::Deployment::Environment - WARN] Environment "feature/DSS-62-mongo" contained non-word characters; sanitizing to "feature_DSS_62_mongo"
[R10K::Deployment::Environment - WARN] Environment "feature/DSS-89-hostsfile-dist" contained non-word characters; sanitizing to "feature_DSS_89_hostsfile_dist"
[R10K::Deployment::Environment - WARN] Environment "feature/DSS-99-Graphite" contained non-word characters; sanitizing to "feature_DSS_99_Graphite"
[R10K::Deployment::Environment - WARN] Environment "feature/DSS_96_patrol_user" contained non-word characters; sanitizing to "feature_DSS_96_patrol_user"
[R10K::Deployment::Environment - WARN] Environment "feature/docker_tests" contained non-word characters; sanitizing to "feature_docker_tests"
[R10K::Deployment::Environment - WARN] Environment "feature/logserver" contained non-word characters; sanitizing to "feature_logserver"
[R10K::Deployment::Environment - WARN] Environment "feature/mapr" contained non-word characters; sanitizing to "feature_mapr"
[R10K::Deployment::Environment - WARN] Environment "hotfix/DSS-128-bash-vulnerability-patch-installation" contained non-word characters; sanitizing to "hotfix_DSS_128_bash_vulnerability_patch_installation"
[R10K::Deployment::Environment - WARN] Environment "hotfix/DSS-128-bash-vulnerability-patch-installation-dw" contained non-word characters; sanitizing to "hotfix_DSS_128_bash_vulnerability_patch_installation_dw"
[R10K::Deployment::Environment - WARN] Environment "mongodb-prd" contained non-word characters; sanitizing to "mongodb_prd"
[R10K::Deployment::Environment - WARN] Environment "release/1.0.0" contained non-word characters; sanitizing to "release_1_0_0"
[R10K::Deployment::Environment - WARN] Environment "release/1.0.1" contained non-word characters; sanitizing to "release_1_0_1"
[R10K::Deployment::Environment - WARN] Environment "release/1.0.10" contained non-word characters; sanitizing to "release_1_0_10"
[R10K::Deployment::Environment - WARN] Environment "release/1.0.11" contained non-word characters; sanitizing to "release_1_0_11"
[R10K::Deployment::Environment - WARN] Environment "release/1.0.12" contained non-word characters; sanitizing to "release_1_0_12"
[R10K::Deployment::Environment - WARN] Environment "release/1.0.2" contained non-word characters; sanitizing to "release_1_0_2"
[R10K::Deployment::Environment - WARN] Environment "release/1.0.3" contained non-word characters; sanitizing to "release_1_0_3"
[R10K::Deployment::Environment - WARN] Environment "release/1.0.4" contained non-word characters; sanitizing to "release_1_0_4"
[R10K::Deployment::Environment - WARN] Environment "release/1.0.5" contained non-word characters; sanitizing to "release_1_0_5"
[R10K::Deployment::Environment - WARN] Environment "release/1.0.6" contained non-word characters; sanitizing to "release_1_0_6"
[R10K::Deployment::Environment - WARN] Environment "release/1.0.7" contained non-word characters; sanitizing to "release_1_0_7"
[R10K::Deployment::Environment - WARN] Environment "release/1.0.8" contained non-word characters; sanitizing to "release_1_0_8"
[R10K::Deployment::Environment - WARN] Environment "release/1.0.9" contained non-word characters; sanitizing to "release_1_0_9"
[R10K::Task::Environment::Deploy - NOTICE] Deploying environment ap1
[R10K::Task::Environment::Deploy - NOTICE] Deploying environment btprd
[R10K::Task::Environment::Deploy - NOTICE] Deploying environment bugfix_DSS_108_snmp_conf
[R10K::Task::Environment::Deploy - NOTICE] Deploying environment bugfix_DSS_112_error_hpacucli_cpqacuxe
[R10K::Task::Environment::Deploy - NOTICE] Deploying environment bugfix_DSS_113_couchbase_logfiles_perm
[R10K::Task::Environment::Deploy - NOTICE] Deploying environment bugfix_DSS_116_couchbase_app_value
[R10K::Task::Environment::Deploy - NOTICE] Deploying environment bugfix_DSS_136_firecell_location
[R10K::Task::Environment::Deploy - NOTICE] Deploying environment bugfix_DSS_137_XFS_missing
[R10K::Task::Environment::Deploy - NOTICE] Deploying environment bugfix_DSS_138_remove_reboot
[R10K::Task::Environment::Deploy - NOTICE] Deploying environment bugfix_DSS_140_lvm_regex_fix
[R10K::Task::Environment::Deploy - NOTICE] Deploying environment bugfix_DSS_152_mapr_lvm
[R10K::Task::Environment::Deploy - NOTICE] Deploying environment bugfix_DSS_153_remove_version_check
[R10K::Task::Environment::Deploy - NOTICE] Deploying environment bugfix_DSS_154_remove_stringify_facts_false_line
[R10K::Task::Environment::Deploy - NOTICE] Deploying environment bugfix_DSS_155_var_log_messages_not_compressed
[R10K::Task::Environment::Deploy - NOTICE] Deploying environment bugfix_DSS_159_rhel66_upgrade
[R10K::Task::Environment::Deploy - NOTICE] Deploying environment bugfix_DSS_225_firecell_timeout
[R10K::Task::Environment::Deploy - NOTICE] Deploying environment bugfix_DSS_230_cmdb_url_update
[R10K::Task::Environment::Deploy - NOTICE] Deploying environment bugfix_DSS_234_wrong_couchbase_permission
[R10K::Task::Environment::Deploy - NOTICE] Deploying environment bugfix_DSS_235_transparenthugepages_not_working
[R10K::Task::Environment::Deploy - NOTICE] Deploying environment bugfix_dss_107
[R10K::Task::Environment::Deploy - NOTICE] Deploying environment bugfix_epel
[R10K::Task::Environment::Deploy - NOTICE] Deploying environment bugfix_mapr_storage_disable
[R10K::Task::Environment::Deploy - NOTICE] Deploying environment bugfix_mapr_storage_disable2
[R10K::Task::Environment::Deploy - NOTICE] Deploying environment cb
[R10K::Task::Environment::Deploy - NOTICE] Deploying environment dev
[R10K::Task::Environment::Deploy - NOTICE] Deploying environment dr
[R10K::Task::Environment::Deploy - NOTICE] Deploying environment feature_CDP_770_bashrc_change
[R10K::Task::Environment::Deploy - NOTICE] Deploying environment feature_DSS_111_filepermssions
[R10K::Task::Environment::Deploy - NOTICE] Deploying environment feature_DSS_114_couchbase_login_shell
[R10K::Task::Environment::Deploy - NOTICE] Deploying environment feature_DSS_115_cb_backup_restore_wrapper
[R10K::Task::Environment::Deploy - NOTICE] Deploying environment feature_DSS_118_firmware_version_fact
[R10K::Task::Environment::Deploy - NOTICE] Deploying environment feature_DSS_123_Core_File_Permissions
[R10K::Task::Environment::Deploy - NOTICE] Deploying environment feature_DSS_132_puppet_acl
[R10K::Task::Environment::Deploy - NOTICE] Deploying environment feature_DSS_132_puppet_couchbase_logfilepermissions
[R10K::Task::Environment::Deploy - NOTICE] Deploying environment feature_DSS_134_puppet_bios_release_fact
[R10K::Task::Environment::Deploy - NOTICE] Deploying environment feature_DSS_135_topodb_module
[R10K::Task::Environment::Deploy - NOTICE] Deploying environment feature_DSS_144_couchbase3
[R10K::Task::Environment::Deploy - NOTICE] Deploying environment feature_DSS_224_test
[R10K::Task::Environment::Deploy - NOTICE] Deploying environment feature_DSS_229_controlm
[R10K::Task::Environment::Deploy - NOTICE] Deploying environment feature_DSS_231_mdb_logrotation
[R10K::Task::Environment::Deploy - NOTICE] Deploying environment feature_DSS_232_network_config
[R10K::Task::Environment::Deploy - NOTICE] Deploying environment feature_DSS_233_lom
[R10K::Task::Environment::Deploy - NOTICE] Deploying environment feature_DSS_237_couchbase_in_DRsite
[R10K::Task::Environment::Deploy - NOTICE] Deploying environment feature_DSS_33_mongodb_qradar
[R10K::Task::Environment::Deploy - NOTICE] Deploying environment feature_DSS_62_mongo
[R10K::Task::Environment::Deploy - NOTICE] Deploying environment feature_DSS_89_hostsfile_dist
[R10K::Task::Environment::Deploy - NOTICE] Deploying environment feature_DSS_99_Graphite
[R10K::Task::Environment::Deploy - NOTICE] Deploying environment feature_DSS_96_patrol_user
[R10K::Task::Environment::Deploy - NOTICE] Deploying environment feature_docker_tests
[R10K::Task::Environment::Deploy - NOTICE] Deploying environment feature_logserver
[R10K::Task::Environment::Deploy - NOTICE] Deploying environment feature_mapr
[R10K::Task::Environment::Deploy - NOTICE] Deploying environment fvt
[R10K::Task::Environment::Deploy - NOTICE] Deploying environment hotfix_DSS_128_bash_vulnerability_patch_installation
[R10K::Task::Environment::Deploy - NOTICE] Deploying environment hotfix_DSS_128_bash_vulnerability_patch_installation_dw
[R10K::Task::Environment::Deploy - NOTICE] Deploying environment mig
[R10K::Task::Environment::Deploy - NOTICE] Deploying environment mongodb_prd
[R10K::Task::Environment::Deploy - NOTICE] Deploying environment pdt
[R10K::Task::Environment::Deploy - NOTICE] Deploying environment ppt
[R10K::Task::Environment::Deploy - NOTICE] Deploying environment prd
[R10K::Task::Environment::Deploy - NOTICE] Deploying environment qrt
[R10K::Task::Environment::Deploy - NOTICE] Deploying environment release_1_0_0
[R10K::Task::Environment::Deploy - NOTICE] Deploying environment release_1_0_1
[R10K::Task::Environment::Deploy - NOTICE] Deploying environment release_1_0_10
[R10K::Task::Environment::Deploy - NOTICE] Deploying environment release_1_0_11
[R10K::Task::Environment::Deploy - NOTICE] Deploying environment release_1_0_12
[R10K::Task::Environment::Deploy - NOTICE] Deploying environment release_1_0_2
[R10K::Task::Environment::Deploy - NOTICE] Deploying environment release_1_0_3
[R10K::Task::Environment::Deploy - NOTICE] Deploying environment release_1_0_4
[R10K::Task::Environment::Deploy - NOTICE] Deploying environment release_1_0_5
[R10K::Task::Environment::Deploy - NOTICE] Deploying environment release_1_0_6
[R10K::Task::Environment::Deploy - NOTICE] Deploying environment release_1_0_7
[R10K::Task::Environment::Deploy - NOTICE] Deploying environment release_1_0_8
[R10K::Task::Environment::Deploy - NOTICE] Deploying environment release_1_0_9
[R10K::Task::Environment::Deploy - NOTICE] Deploying environment rf2
[R10K::Task::Environment::Deploy - NOTICE] Deploying environment rfq
[R10K::Task::Environment::Deploy - NOTICE] Deploying environment skl
[R10K::Task::Environment::Deploy - NOTICE] Deploying environment stg
[R10K::Task::Environment::Deploy - NOTICE] Deploying environment sup
[R10K::Task::Environment::Deploy - NOTICE] Deploying environment trunk
[R10K::Task::Environment::Deploy - NOTICE] Deploying environment tst
[R10K::Task::Environment::Deploy - NOTICE] Deploying environment uat
[R10K::Task::Deployment::PurgeEnvironments - INFO] Purging stale environments from /etc/puppet/environments

Unfortunately each puppet environment checked out has a copy of the .pack file (150MB each):

[root@btxv0100 /etc/puppet/environments]# ls -l  */.git/objects/pack/pack-e226576c95766eb1334d279ca28fdb298a890108.pack
-r--r--r-- 1 root root 155858303 May 29 13:54 ap1/.git/objects/pack/pack-e226576c95766eb1334d279ca28fdb298a890108.pack
-r--r--r-- 1 root root 155858763 May 29 13:54 btprd/.git/objects/pack/pack-e226576c95766eb1334d279ca28fdb298a890108.pack
-r--r--r-- 1 root root 155858303 May 29 13:58 bugfix_dss_107/.git/objects/pack/pack-e226576c95766eb1334d279ca28fdb298a890108.pack
-r--r--r-- 1 root root 155858763 May 29 13:54 bugfix_DSS_108_snmp_conf/.git/objects/pack/pack-e226576c95766eb1334d279ca28fdb298a890108.pack
-r--r--r-- 1 root root 155858763 May 29 13:55 bugfix_DSS_112_error_hpacucli_cpqacuxe/.git/objects/pack/pack-e226576c95766eb1334d279ca28fdb298a890108.pack
-r--r--r-- 1 root root 155858763 May 29 13:55 bugfix_DSS_113_couchbase_logfiles_perm/.git/objects/pack/pack-e226576c95766eb1334d279ca28fdb298a890108.pack
-r--r--r-- 1 root root 155858763 May 29 13:55 bugfix_DSS_116_couchbase_app_value/.git/objects/pack/pack-e226576c95766eb1334d279ca28fdb298a890108.pack
-r--r--r-- 1 root root 155858303 May 29 13:55 bugfix_DSS_136_firecell_location/.git/objects/pack/pack-e226576c95766eb1334d279ca28fdb298a890108.pack
-r--r--r-- 1 root root 155858303 May 29 13:55 bugfix_DSS_137_XFS_missing/.git/objects/pack/pack-e226576c95766eb1334d279ca28fdb298a890108.pack
-r--r--r-- 1 root root 155858763 May 29 13:56 bugfix_DSS_138_remove_reboot/.git/objects/pack/pack-e226576c95766eb1334d279ca28fdb298a890108.pack
-r--r--r-- 1 root root 155858303 May 29 13:56 bugfix_DSS_140_lvm_regex_fix/.git/objects/pack/pack-e226576c95766eb1334d279ca28fdb298a890108.pack
-r--r--r-- 1 root root 155858303 May 29 13:56 bugfix_DSS_152_mapr_lvm/.git/objects/pack/pack-e226576c95766eb1334d279ca28fdb298a890108.pack
-r--r--r-- 1 root root 155858303 May 29 13:56 bugfix_DSS_153_remove_version_check/.git/objects/pack/pack-e226576c95766eb1334d279ca28fdb298a890108.pack
-r--r--r-- 1 root root 155858303 May 29 13:56 bugfix_DSS_154_remove_stringify_facts_false_line/.git/objects/pack/pack-e226576c95766eb1334d279ca28fdb298a890108.pack
-r--r--r-- 1 root root 155858763 May 29 13:57 bugfix_DSS_155_var_log_messages_not_compressed/.git/objects/pack/pack-e226576c95766eb1334d279ca28fdb298a890108.pack
-r--r--r-- 1 root root 155858303 May 29 13:57 bugfix_DSS_159_rhel66_upgrade/.git/objects/pack/pack-e226576c95766eb1334d279ca28fdb298a890108.pack
-r--r--r-- 1 root root 155858763 May 29 13:57 bugfix_DSS_225_firecell_timeout/.git/objects/pack/pack-e226576c95766eb1334d279ca28fdb298a890108.pack
-r--r--r-- 1 root root 155858303 May 29 13:57 bugfix_DSS_230_cmdb_url_update/.git/objects/pack/pack-e226576c95766eb1334d279ca28fdb298a890108.pack
-r--r--r-- 1 root root 155858303 May 29 13:57 bugfix_DSS_234_wrong_couchbase_permission/.git/objects/pack/pack-e226576c95766eb1334d279ca28fdb298a890108.pack
-r--r--r-- 1 root root 155858763 May 29 13:58 bugfix_DSS_235_transparenthugepages_not_working/.git/objects/pack/pack-e226576c95766eb1334d279ca28fdb298a890108.pack
-r--r--r-- 1 root root 155858763 May 29 13:58 bugfix_epel/.git/objects/pack/pack-e226576c95766eb1334d279ca28fdb298a890108.pack
-r--r--r-- 1 root root 155858303 May 29 13:59 bugfix_mapr_storage_disable2/.git/objects/pack/pack-e226576c95766eb1334d279ca28fdb298a890108.pack
-r--r--r-- 1 root root 155858763 May 29 13:58 bugfix_mapr_storage_disable/.git/objects/pack/pack-e226576c95766eb1334d279ca28fdb298a890108.pack
-r--r--r-- 1 root root 155858303 May 29 13:59 cb/.git/objects/pack/pack-e226576c95766eb1334d279ca28fdb298a890108.pack
-r--r--r-- 1 root root 155858763 May 29 13:59 dev/.git/objects/pack/pack-e226576c95766eb1334d279ca28fdb298a890108.pack
-r--r--r-- 1 root root 155858303 May 29 13:59 dr/.git/objects/pack/pack-e226576c95766eb1334d279ca28fdb298a890108.pack
-r--r--r-- 1 root root 155858303 May 29 13:59 feature_CDP_770_bashrc_change/.git/objects/pack/pack-e226576c95766eb1334d279ca28fdb298a890108.pack
-r--r--r-- 1 root root 155858303 May 29 14:03 feature_docker_tests/.git/objects/pack/pack-e226576c95766eb1334d279ca28fdb298a890108.pack
-r--r--r-- 1 root root 155858303 May 29 13:59 feature_DSS_111_filepermssions/.git/objects/pack/pack-e226576c95766eb1334d279ca28fdb298a890108.pack
-r--r--r-- 1 root root 155858303 May 29 14:00 feature_DSS_114_couchbase_login_shell/.git/objects/pack/pack-e226576c95766eb1334d279ca28fdb298a890108.pack
-r--r--r-- 1 root root 155858763 May 29 14:00 feature_DSS_115_cb_backup_restore_wrapper/.git/objects/pack/pack-e226576c95766eb1334d279ca28fdb298a890108.pack
-r--r--r-- 1 root root 155858763 May 29 14:00 feature_DSS_118_firmware_version_fact/.git/objects/pack/pack-e226576c95766eb1334d279ca28fdb298a890108.pack
-r--r--r-- 1 root root 155858303 May 29 14:00 feature_DSS_123_Core_File_Permissions/.git/objects/pack/pack-e226576c95766eb1334d279ca28fdb298a890108.pack
-r--r--r-- 1 root root 155858763 May 29 14:01 feature_DSS_132_puppet_acl/.git/objects/pack/pack-e226576c95766eb1334d279ca28fdb298a890108.pack
-r--r--r-- 1 root root 155858303 May 29 14:01 feature_DSS_132_puppet_couchbase_logfilepermissions/.git/objects/pack/pack-e226576c95766eb1334d279ca28fdb298a890108.pack
-r--r--r-- 1 root root 155858763 May 29 14:01 feature_DSS_134_puppet_bios_release_fact/.git/objects/pack/pack-e226576c95766eb1334d279ca28fdb298a890108.pack
-r--r--r-- 1 root root 155858303 May 29 14:01 feature_DSS_135_topodb_module/.git/objects/pack/pack-e226576c95766eb1334d279ca28fdb298a890108.pack
-r--r--r-- 1 root root 155858303 May 29 14:01 feature_DSS_144_couchbase3/.git/objects/pack/pack-e226576c95766eb1334d279ca28fdb298a890108.pack
-r--r--r-- 1 root root 155858763 May 29 14:01 feature_DSS_224_test/.git/objects/pack/pack-e226576c95766eb1334d279ca28fdb298a890108.pack
-r--r--r-- 1 root root 155858303 May 29 14:02 feature_DSS_229_controlm/.git/objects/pack/pack-e226576c95766eb1334d279ca28fdb298a890108.pack
-r--r--r-- 1 root root 155858303 May 29 14:02 feature_DSS_231_mdb_logrotation/.git/objects/pack/pack-e226576c95766eb1334d279ca28fdb298a890108.pack
-r--r--r-- 1 root root 155858303 May 29 14:02 feature_DSS_232_network_config/.git/objects/pack/pack-e226576c95766eb1334d279ca28fdb298a890108.pack
-r--r--r-- 1 root root 155858303 May 29 14:02 feature_DSS_233_lom/.git/objects/pack/pack-e226576c95766eb1334d279ca28fdb298a890108.pack
-r--r--r-- 1 root root 155858303 May 29 14:02 feature_DSS_237_couchbase_in_DRsite/.git/objects/pack/pack-e226576c95766eb1334d279ca28fdb298a890108.pack
-r--r--r-- 1 root root 155858303 May 29 14:02 feature_DSS_33_mongodb_qradar/.git/objects/pack/pack-e226576c95766eb1334d279ca28fdb298a890108.pack
-r--r--r-- 1 root root 155858763 May 29 14:03 feature_DSS_62_mongo/.git/objects/pack/pack-e226576c95766eb1334d279ca28fdb298a890108.pack
-r--r--r-- 1 root root 155858763 May 29 14:03 feature_DSS_89_hostsfile_dist/.git/objects/pack/pack-e226576c95766eb1334d279ca28fdb298a890108.pack
-r--r--r-- 1 root root 155858763 May 29 14:03 feature_DSS_96_patrol_user/.git/objects/pack/pack-e226576c95766eb1334d279ca28fdb298a890108.pack
-r--r--r-- 1 root root 155858303 May 29 14:03 feature_DSS_99_Graphite/.git/objects/pack/pack-e226576c95766eb1334d279ca28fdb298a890108.pack
-r--r--r-- 1 root root 155858303 May 29 14:03 feature_logserver/.git/objects/pack/pack-e226576c95766eb1334d279ca28fdb298a890108.pack
-r--r--r-- 1 root root 155858303 May 29 14:04 feature_mapr/.git/objects/pack/pack-e226576c95766eb1334d279ca28fdb298a890108.pack
-r--r--r-- 1 root root 155858303 May 29 14:04 fvt/.git/objects/pack/pack-e226576c95766eb1334d279ca28fdb298a890108.pack
-r--r--r-- 1 root root 155858763 May 29 14:04 hotfix_DSS_128_bash_vulnerability_patch_installation_dw/.git/objects/pack/pack-e226576c95766eb1334d279ca28fdb298a890108.pack
-r--r--r-- 1 root root 155858303 May 29 14:04 hotfix_DSS_128_bash_vulnerability_patch_installation/.git/objects/pack/pack-e226576c95766eb1334d279ca28fdb298a890108.pack
-r--r--r-- 1 root root 155858763 May 29 14:04 mig/.git/objects/pack/pack-e226576c95766eb1334d279ca28fdb298a890108.pack
-r--r--r-- 1 root root 155858763 May 29 14:05 mongodb_prd/.git/objects/pack/pack-e226576c95766eb1334d279ca28fdb298a890108.pack
-r--r--r-- 1 root root 155858303 May 29 14:05 pdt/.git/objects/pack/pack-e226576c95766eb1334d279ca28fdb298a890108.pack
-r--r--r-- 1 root root 155858303 May 29 14:05 ppt/.git/objects/pack/pack-e226576c95766eb1334d279ca28fdb298a890108.pack
-r--r--r-- 1 root root 155858763 May 29 14:05 prd/.git/objects/pack/pack-e226576c95766eb1334d279ca28fdb298a890108.pack
-r--r--r-- 1 root root 155858763 May 29 14:05 qrt/.git/objects/pack/pack-e226576c95766eb1334d279ca28fdb298a890108.pack
-r--r--r-- 1 root root 155858763 May 29 14:05 release_1_0_0/.git/objects/pack/pack-e226576c95766eb1334d279ca28fdb298a890108.pack
-r--r--r-- 1 root root 155858303 May 29 14:06 release_1_0_10/.git/objects/pack/pack-e226576c95766eb1334d279ca28fdb298a890108.pack
-r--r--r-- 1 root root 155858303 May 29 14:06 release_1_0_11/.git/objects/pack/pack-e226576c95766eb1334d279ca28fdb298a890108.pack
-r--r--r-- 1 root root 155858763 May 29 14:06 release_1_0_12/.git/objects/pack/pack-e226576c95766eb1334d279ca28fdb298a890108.pack
-r--r--r-- 1 root root 155858303 May 29 14:06 release_1_0_1/.git/objects/pack/pack-e226576c95766eb1334d279ca28fdb298a890108.pack
-r--r--r-- 1 root root 155858303 May 29 14:06 release_1_0_2/.git/objects/pack/pack-e226576c95766eb1334d279ca28fdb298a890108.pack
-r--r--r-- 1 root root 155858303 May 29 14:07 release_1_0_3/.git/objects/pack/pack-e226576c95766eb1334d279ca28fdb298a890108.pack
-r--r--r-- 1 root root 155858303 May 29 14:07 release_1_0_4/.git/objects/pack/pack-e226576c95766eb1334d279ca28fdb298a890108.pack
-r--r--r-- 1 root root 155858303 May 29 14:07 release_1_0_5/.git/objects/pack/pack-e226576c95766eb1334d279ca28fdb298a890108.pack
-r--r--r-- 1 root root 155858303 May 29 14:07 release_1_0_6/.git/objects/pack/pack-e226576c95766eb1334d279ca28fdb298a890108.pack
-r--r--r-- 1 root root 155858303 May 29 14:07 release_1_0_7/.git/objects/pack/pack-e226576c95766eb1334d279ca28fdb298a890108.pack
-r--r--r-- 1 root root 155858763 May 29 14:08 release_1_0_8/.git/objects/pack/pack-e226576c95766eb1334d279ca28fdb298a890108.pack
-r--r--r-- 1 root root 155858303 May 29 14:08 release_1_0_9/.git/objects/pack/pack-e226576c95766eb1334d279ca28fdb298a890108.pack
-r--r--r-- 1 root root 155858763 May 29 14:08 rf2/.git/objects/pack/pack-e226576c95766eb1334d279ca28fdb298a890108.pack
-r--r--r-- 1 root root 155858303 May 29 14:08 rfq/.git/objects/pack/pack-e226576c95766eb1334d279ca28fdb298a890108.pack
-r--r--r-- 1 root root 155858763 May 29 14:08 skl/.git/objects/pack/pack-e226576c95766eb1334d279ca28fdb298a890108.pack
-r--r--r-- 1 root root 155858303 May 29 14:09 stg/.git/objects/pack/pack-e226576c95766eb1334d279ca28fdb298a890108.pack
-r--r--r-- 1 root root 155858303 May 29 14:09 sup/.git/objects/pack/pack-e226576c95766eb1334d279ca28fdb298a890108.pack
-r--r--r-- 1 root root 155858303 May 29 14:09 trunk/.git/objects/pack/pack-e226576c95766eb1334d279ca28fdb298a890108.pack
-r--r--r-- 1 root root 155858763 May 29 14:09 tst/.git/objects/pack/pack-e226576c95766eb1334d279ca28fdb298a890108.pack
-r--r--r-- 1 root root 155858303 May 29 14:09 uat/.git/objects/pack/pack-e226576c95766eb1334d279ca28fdb298a890108.pack

What I find really strange is that not all pack files have the same size, using md5sum I get:

[root@btxv0100 /etc/puppet/environments]# md5sum  */.git/objects/pack/pack-e226576c95766eb1334d279ca28fdb298a890108.pack
ec192e1fe49248560a68bb726436ed7a  ap1/.git/objects/pack/pack-e226576c95766eb1334d279ca28fdb298a890108.pack
34d9dd26744be43512f6c2a777d1ed59  btprd/.git/objects/pack/pack-e226576c95766eb1334d279ca28fdb298a890108.pack
ec192e1fe49248560a68bb726436ed7a  bugfix_dss_107/.git/objects/pack/pack-e226576c95766eb1334d279ca28fdb298a890108.pack
34d9dd26744be43512f6c2a777d1ed59  bugfix_DSS_108_snmp_conf/.git/objects/pack/pack-e226576c95766eb1334d279ca28fdb298a890108.pack
34d9dd26744be43512f6c2a777d1ed59  bugfix_DSS_112_error_hpacucli_cpqacuxe/.git/objects/pack/pack-e226576c95766eb1334d279ca28fdb298a890108.pack
34d9dd26744be43512f6c2a777d1ed59  bugfix_DSS_113_couchbase_logfiles_perm/.git/objects/pack/pack-e226576c95766eb1334d279ca28fdb298a890108.pack
34d9dd26744be43512f6c2a777d1ed59  bugfix_DSS_116_couchbase_app_value/.git/objects/pack/pack-e226576c95766eb1334d279ca28fdb298a890108.pack
ec192e1fe49248560a68bb726436ed7a  bugfix_DSS_136_firecell_location/.git/objects/pack/pack-e226576c95766eb1334d279ca28fdb298a890108.pack
ec192e1fe49248560a68bb726436ed7a  bugfix_DSS_137_XFS_missing/.git/objects/pack/pack-e226576c95766eb1334d279ca28fdb298a890108.pack
34d9dd26744be43512f6c2a777d1ed59  bugfix_DSS_138_remove_reboot/.git/objects/pack/pack-e226576c95766eb1334d279ca28fdb298a890108.pack
ec192e1fe49248560a68bb726436ed7a  bugfix_DSS_140_lvm_regex_fix/.git/objects/pack/pack-e226576c95766eb1334d279ca28fdb298a890108.pack
ec192e1fe49248560a68bb726436ed7a  bugfix_DSS_152_mapr_lvm/.git/objects/pack/pack-e226576c95766eb1334d279ca28fdb298a890108.pack
ec192e1fe49248560a68bb726436ed7a  bugfix_DSS_153_remove_version_check/.git/objects/pack/pack-e226576c95766eb1334d279ca28fdb298a890108.pack
ec192e1fe49248560a68bb726436ed7a  bugfix_DSS_154_remove_stringify_facts_false_line/.git/objects/pack/pack-e226576c95766eb1334d279ca28fdb298a890108.pack
34d9dd26744be43512f6c2a777d1ed59  bugfix_DSS_155_var_log_messages_not_compressed/.git/objects/pack/pack-e226576c95766eb1334d279ca28fdb298a890108.pack
ec192e1fe49248560a68bb726436ed7a  bugfix_DSS_159_rhel66_upgrade/.git/objects/pack/pack-e226576c95766eb1334d279ca28fdb298a890108.pack
34d9dd26744be43512f6c2a777d1ed59  bugfix_DSS_225_firecell_timeout/.git/objects/pack/pack-e226576c95766eb1334d279ca28fdb298a890108.pack
ec192e1fe49248560a68bb726436ed7a  bugfix_DSS_230_cmdb_url_update/.git/objects/pack/pack-e226576c95766eb1334d279ca28fdb298a890108.pack
ec192e1fe49248560a68bb726436ed7a  bugfix_DSS_234_wrong_couchbase_permission/.git/objects/pack/pack-e226576c95766eb1334d279ca28fdb298a890108.pack
34d9dd26744be43512f6c2a777d1ed59  bugfix_DSS_235_transparenthugepages_not_working/.git/objects/pack/pack-e226576c95766eb1334d279ca28fdb298a890108.pack
34d9dd26744be43512f6c2a777d1ed59  bugfix_epel/.git/objects/pack/pack-e226576c95766eb1334d279ca28fdb298a890108.pack
ec192e1fe49248560a68bb726436ed7a  bugfix_mapr_storage_disable2/.git/objects/pack/pack-e226576c95766eb1334d279ca28fdb298a890108.pack
34d9dd26744be43512f6c2a777d1ed59  bugfix_mapr_storage_disable/.git/objects/pack/pack-e226576c95766eb1334d279ca28fdb298a890108.pack
ec192e1fe49248560a68bb726436ed7a  cb/.git/objects/pack/pack-e226576c95766eb1334d279ca28fdb298a890108.pack
34d9dd26744be43512f6c2a777d1ed59  dev/.git/objects/pack/pack-e226576c95766eb1334d279ca28fdb298a890108.pack
ec192e1fe49248560a68bb726436ed7a  dr/.git/objects/pack/pack-e226576c95766eb1334d279ca28fdb298a890108.pack
ec192e1fe49248560a68bb726436ed7a  feature_CDP_770_bashrc_change/.git/objects/pack/pack-e226576c95766eb1334d279ca28fdb298a890108.pack
ec192e1fe49248560a68bb726436ed7a  feature_docker_tests/.git/objects/pack/pack-e226576c95766eb1334d279ca28fdb298a890108.pack
ec192e1fe49248560a68bb726436ed7a  feature_DSS_111_filepermssions/.git/objects/pack/pack-e226576c95766eb1334d279ca28fdb298a890108.pack
ec192e1fe49248560a68bb726436ed7a  feature_DSS_114_couchbase_login_shell/.git/objects/pack/pack-e226576c95766eb1334d279ca28fdb298a890108.pack
34d9dd26744be43512f6c2a777d1ed59  feature_DSS_115_cb_backup_restore_wrapper/.git/objects/pack/pack-e226576c95766eb1334d279ca28fdb298a890108.pack
34d9dd26744be43512f6c2a777d1ed59  feature_DSS_118_firmware_version_fact/.git/objects/pack/pack-e226576c95766eb1334d279ca28fdb298a890108.pack
ec192e1fe49248560a68bb726436ed7a  feature_DSS_123_Core_File_Permissions/.git/objects/pack/pack-e226576c95766eb1334d279ca28fdb298a890108.pack
34d9dd26744be43512f6c2a777d1ed59  feature_DSS_132_puppet_acl/.git/objects/pack/pack-e226576c95766eb1334d279ca28fdb298a890108.pack
ec192e1fe49248560a68bb726436ed7a  feature_DSS_132_puppet_couchbase_logfilepermissions/.git/objects/pack/pack-e226576c95766eb1334d279ca28fdb298a890108.pack
34d9dd26744be43512f6c2a777d1ed59  feature_DSS_134_puppet_bios_release_fact/.git/objects/pack/pack-e226576c95766eb1334d279ca28fdb298a890108.pack
ec192e1fe49248560a68bb726436ed7a  feature_DSS_135_topodb_module/.git/objects/pack/pack-e226576c95766eb1334d279ca28fdb298a890108.pack
ec192e1fe49248560a68bb726436ed7a  feature_DSS_144_couchbase3/.git/objects/pack/pack-e226576c95766eb1334d279ca28fdb298a890108.pack
34d9dd26744be43512f6c2a777d1ed59  feature_DSS_224_test/.git/objects/pack/pack-e226576c95766eb1334d279ca28fdb298a890108.pack
ec192e1fe49248560a68bb726436ed7a  feature_DSS_229_controlm/.git/objects/pack/pack-e226576c95766eb1334d279ca28fdb298a890108.pack
ec192e1fe49248560a68bb726436ed7a  feature_DSS_231_mdb_logrotation/.git/objects/pack/pack-e226576c95766eb1334d279ca28fdb298a890108.pack
ec192e1fe49248560a68bb726436ed7a  feature_DSS_232_network_config/.git/objects/pack/pack-e226576c95766eb1334d279ca28fdb298a890108.pack
ec192e1fe49248560a68bb726436ed7a  feature_DSS_233_lom/.git/objects/pack/pack-e226576c95766eb1334d279ca28fdb298a890108.pack
ec192e1fe49248560a68bb726436ed7a  feature_DSS_237_couchbase_in_DRsite/.git/objects/pack/pack-e226576c95766eb1334d279ca28fdb298a890108.pack
ec192e1fe49248560a68bb726436ed7a  feature_DSS_33_mongodb_qradar/.git/objects/pack/pack-e226576c95766eb1334d279ca28fdb298a890108.pack
34d9dd26744be43512f6c2a777d1ed59  feature_DSS_62_mongo/.git/objects/pack/pack-e226576c95766eb1334d279ca28fdb298a890108.pack
34d9dd26744be43512f6c2a777d1ed59  feature_DSS_89_hostsfile_dist/.git/objects/pack/pack-e226576c95766eb1334d279ca28fdb298a890108.pack
34d9dd26744be43512f6c2a777d1ed59  feature_DSS_96_patrol_user/.git/objects/pack/pack-e226576c95766eb1334d279ca28fdb298a890108.pack
ec192e1fe49248560a68bb726436ed7a  feature_DSS_99_Graphite/.git/objects/pack/pack-e226576c95766eb1334d279ca28fdb298a890108.pack
ec192e1fe49248560a68bb726436ed7a  feature_logserver/.git/objects/pack/pack-e226576c95766eb1334d279ca28fdb298a890108.pack
ec192e1fe49248560a68bb726436ed7a  feature_mapr/.git/objects/pack/pack-e226576c95766eb1334d279ca28fdb298a890108.pack
ec192e1fe49248560a68bb726436ed7a  fvt/.git/objects/pack/pack-e226576c95766eb1334d279ca28fdb298a890108.pack
34d9dd26744be43512f6c2a777d1ed59  hotfix_DSS_128_bash_vulnerability_patch_installation_dw/.git/objects/pack/pack-e226576c95766eb1334d279ca28fdb298a890108.pack
ec192e1fe49248560a68bb726436ed7a  hotfix_DSS_128_bash_vulnerability_patch_installation/.git/objects/pack/pack-e226576c95766eb1334d279ca28fdb298a890108.pack
34d9dd26744be43512f6c2a777d1ed59  mig/.git/objects/pack/pack-e226576c95766eb1334d279ca28fdb298a890108.pack
34d9dd26744be43512f6c2a777d1ed59  mongodb_prd/.git/objects/pack/pack-e226576c95766eb1334d279ca28fdb298a890108.pack
ec192e1fe49248560a68bb726436ed7a  pdt/.git/objects/pack/pack-e226576c95766eb1334d279ca28fdb298a890108.pack
ec192e1fe49248560a68bb726436ed7a  ppt/.git/objects/pack/pack-e226576c95766eb1334d279ca28fdb298a890108.pack
34d9dd26744be43512f6c2a777d1ed59  prd/.git/objects/pack/pack-e226576c95766eb1334d279ca28fdb298a890108.pack
34d9dd26744be43512f6c2a777d1ed59  qrt/.git/objects/pack/pack-e226576c95766eb1334d279ca28fdb298a890108.pack
34d9dd26744be43512f6c2a777d1ed59  release_1_0_0/.git/objects/pack/pack-e226576c95766eb1334d279ca28fdb298a890108.pack
ec192e1fe49248560a68bb726436ed7a  release_1_0_10/.git/objects/pack/pack-e226576c95766eb1334d279ca28fdb298a890108.pack
ec192e1fe49248560a68bb726436ed7a  release_1_0_11/.git/objects/pack/pack-e226576c95766eb1334d279ca28fdb298a890108.pack
34d9dd26744be43512f6c2a777d1ed59  release_1_0_12/.git/objects/pack/pack-e226576c95766eb1334d279ca28fdb298a890108.pack
ec192e1fe49248560a68bb726436ed7a  release_1_0_1/.git/objects/pack/pack-e226576c95766eb1334d279ca28fdb298a890108.pack
ec192e1fe49248560a68bb726436ed7a  release_1_0_2/.git/objects/pack/pack-e226576c95766eb1334d279ca28fdb298a890108.pack
ec192e1fe49248560a68bb726436ed7a  release_1_0_3/.git/objects/pack/pack-e226576c95766eb1334d279ca28fdb298a890108.pack
ec192e1fe49248560a68bb726436ed7a  release_1_0_4/.git/objects/pack/pack-e226576c95766eb1334d279ca28fdb298a890108.pack
ec192e1fe49248560a68bb726436ed7a  release_1_0_5/.git/objects/pack/pack-e226576c95766eb1334d279ca28fdb298a890108.pack
ec192e1fe49248560a68bb726436ed7a  release_1_0_6/.git/objects/pack/pack-e226576c95766eb1334d279ca28fdb298a890108.pack
ec192e1fe49248560a68bb726436ed7a  release_1_0_7/.git/objects/pack/pack-e226576c95766eb1334d279ca28fdb298a890108.pack
34d9dd26744be43512f6c2a777d1ed59  release_1_0_8/.git/objects/pack/pack-e226576c95766eb1334d279ca28fdb298a890108.pack
ec192e1fe49248560a68bb726436ed7a  release_1_0_9/.git/objects/pack/pack-e226576c95766eb1334d279ca28fdb298a890108.pack
34d9dd26744be43512f6c2a777d1ed59  rf2/.git/objects/pack/pack-e226576c95766eb1334d279ca28fdb298a890108.pack
ec192e1fe49248560a68bb726436ed7a  rfq/.git/objects/pack/pack-e226576c95766eb1334d279ca28fdb298a890108.pack
34d9dd26744be43512f6c2a777d1ed59  skl/.git/objects/pack/pack-e226576c95766eb1334d279ca28fdb298a890108.pack
ec192e1fe49248560a68bb726436ed7a  stg/.git/objects/pack/pack-e226576c95766eb1334d279ca28fdb298a890108.pack
ec192e1fe49248560a68bb726436ed7a  sup/.git/objects/pack/pack-e226576c95766eb1334d279ca28fdb298a890108.pack
ec192e1fe49248560a68bb726436ed7a  trunk/.git/objects/pack/pack-e226576c95766eb1334d279ca28fdb298a890108.pack
34d9dd26744be43512f6c2a777d1ed59  tst/.git/objects/pack/pack-e226576c95766eb1334d279ca28fdb298a890108.pack
ec192e1fe49248560a68bb726436ed7a  uat/.git/objects/pack/pack-e226576c95766eb1334d279ca28fdb298a890108.pack

There seem to be 2 groups of files and they are different than the pack file in cachedir:

[root@btxv0100 /etc/puppet/environments]# md5sum /opt/r10k/http---myuser-mypassword\@rndwww.nce.amadeus.net-git-scm-dzb-puppet-modules.git/objects/pack/pack-744c2fde41305656915b5fe8a1cb0be96f0c9e02.pack
30d953536788ee1cfc810356696f22a5  /opt/r10k/http---myuser-mypassword@rndwww.nce.amadeus.net-git-scm-dzb-puppet-modules.git/objects/pack/pack-744c2fde41305656915b5fe8a1cb0be96f0c9e02.pack<mailto:/opt/r10k/http---myuser-mypassword@rndwww.nce.amadeus.net-git-scm-dzb-puppet-modules.git/objects/pack/pack-744c2fde41305656915b5fe8a1cb0be96f0c9e02.pack>

Thanks again. I will be on vacation next week. Will try to come back the week after.

Have a good weekend Michael

From: Adrien Thebo [mailto:notifications@github.com] Sent: 26 May 2015 21:55 To: puppetlabs/r10k Cc: Michael EHRIG Subject: Re: [r10k] Question regarding git and pack files with r10k (#426)

That path looks generally correct, but the fact that the repositories are being cached in /tmp might be a little strange. What sort of filesystem is /tmp? The repositories that r10k is creating are expected to be long lived, so using /tmp might have some interesting behaviors; (https://github.com/puppetlabs/r10k/blob/master/doc/git/cloning-and-mirroring.mkd) has more information on this.

— Reply to this email directly or view it on GitHubhttps://github.com/puppetlabs/r10k/issues/426#issuecomment-105647330.

IMPORTANT - CONFIDENTIALITY NOTICE - This e-mail is intended only for the use of the individual or entity shown above as addressees . It may contain information which is privileged, confidential or otherwise protected from disclosure under applicable laws . If the reader of this transmission is not the intended recipient, you are hereby notified that any dissemination, printing, distribution, copying, disclosure or the taking of any action in reliance on the contents of this information is strictly prohibited. If you have received this transmission in error, please immediately notify us by reply e-mail or using the address below and delete the message and any attachments from your system . Amadeus Data Processing GmbH Geschäftsführer: Dr. Wolfgang Krips Sitz der Gesellschaft: Erding HR München 48 199 Berghamer Strasse 6 85435 Erding Germany

adrienthebo commented 9 years ago

Sorry for the slow response to this, I was on vacation around the time I was responding to this and it looks to have slipped my mind.

So if you have packfiles with the same file name but different content, this is extremely bad. Packfiles should be named based on the md5sum of their content so if you have two packfiles with the same name but different content, some basic assumptions about the world are no longer applicable. Could you use git-verify-pack against the packfiles/packfile indexes to verify that they're valid?

adrienthebo commented 9 years ago

This issue hasn't seen any updates in a few weeks, is this still an issue? Have you been able to verify the integrity of your Git packfiles?

audimann commented 9 years ago

Hi Adrien,

I apologize for not being responsive but after the vacation there was so much to do, I didn’t get to work on this problem before today. Sorry! Actually, I just deleted everything in /etc/puppet/environments again and ran the command:

r10k deploy environment -v debug > /tmp/r10k_deploy.txt 2>&1

which populated the directory again (82 branches). And indeed, I don’t always get the same md5sum for the packfile although its filename is identical for each branch. Example:

[root@btxv0100 /etc/puppet/environments]# md5sum ./release_1_0_2/.git/objects/pack/pack-9621a86bde1f56987200bc632cb228f8b5f6d3da.pack
c8e7cec63044377ec055551a07d84a1d  ./release_1_0_2/.git/objects/pack/pack-9621a86bde1f56987200bc632cb228f8b5f6d3da.pack
[root@btxv0100 /etc/puppet/environments]# md5sum ./release_1_0_1/.git/objects/pack/pack-9621a86bde1f56987200bc632cb228f8b5f6d3da.pack
2cad98c67a1be4b36b3c656f56daba5e  ./release_1_0_1/.git/objects/pack/pack-9621a86bde1f56987200bc632cb228f8b5f6d3da.pack

[root@btxv0100 /etc/puppet/environments]# ll ./release_1_0_2/.git/objects/pack/pack-9621a86bde1f56987200bc632cb228f8b5f6d3da.pack ./release_1_0_1/.git/objects/pack/pack-9621a86bde1f56987200bc632cb228f8b5f6d3da.pack
-r--r--r-- 1 root root 149M Jul 20 15:38 ./release_1_0_1/.git/objects/pack/pack-9621a86bde1f56987200bc632cb228f8b5f6d3da.pack
-r--r--r-- 1 root root 149M Jul 20 15:39 ./release_1_0_2/.git/objects/pack/pack-9621a86bde1f56987200bc632cb228f8b5f6d3da.pack

When I run the git verify-pack I get:

[root@btxv0100 /etc/puppet/environments]# git verify-pack ./release_1_0_1/.git/objects/pack/pack-9621a86bde1f56987200bc632cb228f8b5f6d3da.pack
[root@btxv0100 /etc/puppet/environments]# echo $?
0
[root@btxv0100 /etc/puppet/environments]# git verify-pack ./release_1_0_2/.git/objects/pack/pack-9621a86bde1f56987200bc632cb228f8b5f6d3da.pack
[root@btxv0100 /etc/puppet/environments]# echo $?
0

Both see to be OK but they have a different md5sum and the same name!

If I add the –v flag to git verify-pack I get something like:

git verify-pack -v  ./release_1_0_1/.git/objects/pack/pack-9621a86bde1f56987200bc632cb228f8b5f6d3da.pack
…
fff166dfe08787d001df783bcea097827ec5af97 blob   612 626 916343 2 72be0c2811266fcf1d91e901eae3441ca78dbee3
fff24fb44db89d4ddbaa88963bb66f5be92bbf14 tree   33 49 1386613 12 0def472388b4ad4c36c880f65305d77545c621f1
fff466177c58be988a62facfd1a728e946944b8d tree   25 39 1672511 15 071ab0c19f6101ec0340dac8fffb91f3888321c6
fff620bfc8ec1c3e56a34e0fc5eb34a3089ed859 tree   71 77 155978904
fff7a7c72facb1b6495a7b3307945d70d70ec64a commit 276 190 142162
fff8bcae190c174fbc9b01cefe9897c4f9a6a02e commit 222 154 287672
fffc57b2030b81a4b95b626c46656dc4e8063cf4 tree   33 42 156036898
non delta: 9057 objects
chain length = 1: 2672 objects
chain length = 2: 929 objects
chain length = 3: 583 objects
chain length = 4: 463 objects
chain length = 5: 286 objects
chain length = 6: 329 objects
chain length = 7: 230 objects
…

And for the second file:

git verify-pack -v  ./release_1_0_2/.git/objects/pack/pack-9621a86bde1f56987200bc632cb228f8b5f6d3da.pack
…
fff166dfe08787d001df783bcea097827ec5af97 blob   612 626 916128 2 72be0c2811266fcf1d91e901eae3441ca78dbee3
fff24fb44db89d4ddbaa88963bb66f5be92bbf14 tree   33 49 1388917 8 0def472388b4ad4c36c880f65305d77545c621f1
fff466177c58be988a62facfd1a728e946944b8d tree   25 39 1674815 15 071ab0c19f6101ec0340dac8fffb91f3888321c6
fff620bfc8ec1c3e56a34e0fc5eb34a3089ed859 tree   71 77 155981208
fff7a7c72facb1b6495a7b3307945d70d70ec64a commit 276 190 142162
fff8bcae190c174fbc9b01cefe9897c4f9a6a02e commit 222 154 287672
fffc57b2030b81a4b95b626c46656dc4e8063cf4 tree   33 42 156039202
non delta: 9059 objects
chain length = 1: 2685 objects
chain length = 2: 968 objects
chain length = 3: 636 objects
chain length = 4: 552 objects
chain length = 5: 392 objects
chain length = 6: 382 objects
chain length = 7: 272 objects
…

So, they are both valid pack files with the same name and a different content.

I must admit I am really a bit lost now ;-)

Many thanks!

adrienthebo commented 9 years ago

Well, that's quite peculiar. In each of those repositories you should have a .git/objects/info/alternates file which will point to the bare repository used for sharing those objects, could you inspect the packfiles at that location in the same manner that you did above?

audimann commented 9 years ago

Hi Adrien

Same pointer to the alternates file:

[root@btxv0100 /etc/puppet/environments]# cat release_1_0_1/.git/objects/info/alternates
/opt/r10k/http---myuser-mypassword@rndwww.nce.amadeus.net-git-scm-dzb-puppet-modules.git/objects
[root@btxv0100 /etc/puppet/environments]# cat release_1_0_2/.git/objects/info/alternates
/opt/r10k/http---myuser-mypassword@rndwww.nce.amadeus.net-git-scm-dzb-puppet-modules.git/objects<mailto:/opt/r10k/http---myuser-mypassword@rndwww.nce.amadeus.net-git-scm-dzb-puppet-modules.git/objects>

Name is identical, md5sum is not :

[root@btxv0100 /etc/puppet/environments]# md5sum release_1_0_1/.git/objects/pack/pack-9621a86bde1f56987200bc632cb228f8b5f6d3da.pack
2cad98c67a1be4b36b3c656f56daba5e  release_1_0_1/.git/objects/pack/pack-9621a86bde1f56987200bc632cb228f8b5f6d3da.pack
[root@btxv0100 /etc/puppet/environments]# md5sum release_1_0_2/.git/objects/pack/pack-9621a86bde1f56987200bc632cb228f8b5f6d3da.pack
c8e7cec63044377ec055551a07d84a1d  release_1_0_2/.git/objects/pack/pack-9621a86bde1f56987200bc632cb228f8b5f6d3da.pack

[root@btxv0100 /etc/puppet/environments]# ll /opt/r10k/http---myuser-mypassword\@rndwww.nce.amadeus.net-git-scm-dzb-puppet-modules.git/objects/
total 108K
drwxr-xr-x 27 root root 4.0K Jun 10 14:25 .
drwxr-xr-x  7 root root 4.0K Jun 10 08:08 ..
drwxr-xr-x  2 root root 4.0K Jun 10 08:08 08
drwxr-xr-x  2 root root 4.0K Jun 10 08:08 15
drwxr-xr-x  2 root root 4.0K Jun 10 08:08 1f
drwxr-xr-x  2 root root 4.0K Jun 10 08:08 34
drwxr-xr-x  2 root root 4.0K Jun 10 08:08 38
drwxr-xr-x  2 root root 4.0K Jun 10 08:08 45
drwxr-xr-x  2 root root 4.0K Jun 10 08:08 60
drwxr-xr-x  2 root root 4.0K Jun 10 08:08 66
drwxr-xr-x  2 root root 4.0K Jun 10 14:25 6a
drwxr-xr-x  2 root root 4.0K Jun 10 08:08 79
drwxr-xr-x  2 root root 4.0K Jun 10 08:08 7e
drwxr-xr-x  2 root root 4.0K Jun 10 08:08 84
drwxr-xr-x  2 root root 4.0K Jun 10 08:08 a4
drwxr-xr-x  2 root root 4.0K Jun 10 08:08 ac
drwxr-xr-x  2 root root 4.0K Jun 10 08:08 af
drwxr-xr-x  2 root root 4.0K Jun 10 14:25 bb
drwxr-xr-x  2 root root 4.0K Jun 10 14:25 c4
drwxr-xr-x  2 root root 4.0K Jun 10 14:25 c8
drwxr-xr-x  2 root root 4.0K Jun 10 08:08 ce
drwxr-xr-x  2 root root 4.0K Jun 10 08:08 d6
drwxr-xr-x  2 root root 4.0K Jun 10 14:25 df
drwxr-xr-x  2 root root 4.0K Jun 10 14:25 ed
drwxr-xr-x  2 root root 4.0K Jun 10 08:08 ef
drwxr-xr-x  2 root root 4.0K May 29 13:53 info
drwxr-xr-x  2 root root 4.0K Jul 20 15:31 pack

The content of the pack file:

[root@btxv0100 /etc/puppet/environments]# ll /opt/r10k/http---myuser-mypassword\@rndwww.nce.amadeus.net-git-scm-dzb-puppet-modules.git/objects/pack/
total 150M
drwxr-xr-x  2 root root 4.0K Jul 20 15:31 .
drwxr-xr-x 27 root root 4.0K Jun 10 14:25 ..
-r--r--r--  1 root root  70K Jul 20 15:31 pack-60c5b66948d344d07d964517e58d089262b91b75.idx
-r--r--r--  1 root root 238K Jul 20 15:31 pack-60c5b66948d344d07d964517e58d089262b91b75.pack
-r--r--r--  1 root root 396K May 29 13:54 pack-744c2fde41305656915b5fe8a1cb0be96f0c9e02.idx
-r--r--r--  1 root root 149M May 29 13:54 pack-744c2fde41305656915b5fe8a1cb0be96f0c9e02.pack

No file with the name pack-9621a86bde1f56987200bc632cb228f8b5f6d3da.pack exists there, the only file with 149MB in size is pack-744c2fde41305656915b5fe8a1cb0be96f0c9e02.pack

If I verify it I get:

git verify-pack -v /opt/r10k/http---myuser-mypassword\@rndwww.nce.amadeus.net-git-scm-dzb-puppet-modules.git/objects/pack/pack-744c2fde41305656915b5fe8a1cb0be96f0c9e02.pack
fff166dfe08787d001df783bcea097827ec5af97 blob   612 626 619951 2 72be0c2811266fcf1d91e901eae3441ca78dbee3
fff24fb44db89d4ddbaa88963bb66f5be92bbf14 tree   33 49 1164227 8 0def472388b4ad4c36c880f65305d77545c621f1
fff466177c58be988a62facfd1a728e946944b8d tree   25 39 1450080 15 071ab0c19f6101ec0340dac8fffb91f3888321c6
fff620bfc8ec1c3e56a34e0fc5eb34a3089ed859 tree   71 77 155756473
fff7a7c72facb1b6495a7b3307945d70d70ec64a commit 276 190 90690
fff8bcae190c174fbc9b01cefe9897c4f9a6a02e commit 222 154 236200
fffc57b2030b81a4b95b626c46656dc4e8063cf4 tree   33 42 155814467
non delta: 7693 objects
chain length = 1: 2146 objects
chain length = 2: 850 objects
chain length = 3: 544 objects
chain length = 4: 482 objects
chain length = 5: 386 objects
chain length = 6: 361 objects
chain length = 7: 213 objects
chain length = 8: 230 objects

Return value is 0, pack file is valid.

adrienthebo commented 9 years ago

Well, none of this really makes sense. In each of the working repositories and the cached repository, could you run git fsck --full and git fsck --no-full? The latter command should fail in each of the working repositories since they should be borrowing objects from the cached repo, but given that they have those packfiles I think that might not be the case.

If this doesn't turn up anything, could you delete all of the cached repositories and environments and re-run r10k with an entirely fresh cache?

audimann commented 9 years ago

Hi Adrien,

Sure. Here is the cache copyunder /opt/r10k: [root@btxv0100 /opt/r10k/http---myuser-mypassword@rndwww.nce.amadeus.net-git-scm-dzb-puppet-modules.git]# git fsck --full [root@btxv0100 /opt/r10k/http---myuser-mypassword@rndwww.nce.amadeus.net-git-scm-dzb-puppet-modules.git]# git fsck --no-full dangling commit 6acd23fa1b4fdb01fb7a883589091fc457b69cf7 dangling commit d6a128618c283d2e3b54c3ead0a4de9f14607f54

Now, in the release_1_0_1 directory: [root@btxv0100 /etc/puppet/environments/release_1_0_1]# git fsck --full dangling commit a33b7237bf40addb72e4f0d77194fffc760bc2b4 dangling commit b26d8cfd5b67d32452b696afd13c6fe3a63937e4 dangling commit 32814ce4667c2c75b13ebf9d0cb123f9a68f23ee dangling commit 04871395c4af82ca1b187f0df3deefb9fdfb0ff9 [root@btxv0100 /etc/puppet/environments/release_1_0_1]# git fsck --no-full dangling commit d6a128618c283d2e3b54c3ead0a4de9f14607f54 dangling commit 6acd23fa1b4fdb01fb7a883589091fc457b69cf7

And the releae_1_0_2 directory: [root@btxv0100 /etc/puppet/environments/release_1_0_2]# git fsck --full dangling commit a33b7237bf40addb72e4f0d77194fffc760bc2b4 dangling commit b26d8cfd5b67d32452b696afd13c6fe3a63937e4 dangling commit 32814ce4667c2c75b13ebf9d0cb123f9a68f23ee dangling commit 04871395c4af82ca1b187f0df3deefb9fdfb0ff9 [root@btxv0100 /etc/puppet/environments/release_1_0_2]# git fsck --no-full dangling commit d6a128618c283d2e3b54c3ead0a4de9f14607f54 dangling commit 6acd23fa1b4fdb01fb7a883589091fc457b69cf7

Does this throw a hint? I can of course delete the cache and the environments directory and re-populate everything if that helps.

Many thanks Michael

From: Adrien Thebo [mailto:notifications@github.com] Sent: 21 July 2015 21:28 To: puppetlabs/r10k Cc: Michael EHRIG Subject: Re: [r10k] Question regarding git and pack files with r10k (#426)

Well, none of this really makes sense. In each of the working repositories and the cached repository, could you run git fsck --full and git fsck --no-full? The latter command should fail in each of the working repositories since they should be borrowing objects from the cached repo, but given that they have those packfiles I think that might not be the case.

If this doesn't turn up anything, could you delete all of the cached repositories and environments and re-run r10k with an entirely fresh cache?

— Reply to this email directly or view it on GitHubhttps://github.com/puppetlabs/r10k/issues/426#issuecomment-123454488.

IMPORTANT - CONFIDENTIALITY NOTICE - This e-mail is intended only for the use of the individual or entity shown above as addressees . It may contain information which is privileged, confidential or otherwise protected from disclosure under applicable laws . If the reader of this transmission is not the intended recipient, you are hereby notified that any dissemination, printing, distribution, copying, disclosure or the taking of any action in reliance on the contents of this information is strictly prohibited. If you have received this transmission in error, please immediately notify us by reply e-mail or using the address below and delete the message and any attachments from your system . Amadeus Data Processing GmbH Geschäftsführer: Dr. Wolfgang Krips Sitz der Gesellschaft: Erding HR München 48 199 Berghamer Strasse 6 85435 Erding Germany

adrienthebo commented 9 years ago

Yeah, let's go with wiping the existing state and starting afresh; I'm pretty baffled that running a git fsck --full would have more errors than --no-full.

Out of curiosity, are /etc/puppet and /opt/r10k on the same mount?

audimann commented 9 years ago

Hi Adrien,

These are in fact two different filesystems:

[root@btxv0100 /etc/puppet/environments]# df -h /etc/puppet/environments /opt/r10k Filesystem Size Used Avail Use% Mounted on /dev/mapper/vg00-puppetenv 25G 12G 12G 50% /etc/puppet/environments /dev/mapper/vg00-opt 3.9G 653M 3.0G 18% /opt

Today I removed /opt/r10k as well /etc/puppet/environments/* and I reran r10k deploy environment -v > /tmp/r10kdeploy 2>&1 The file is attached Two errors are suspicious to me, they look like: [R10K::TaskRunner - ERROR] Task #R10K::Task::Environment::Deploy:0x7f3c98b51c20 failed while running: Command git clone --reference /opt/r10k/http---myuser-mypassword@rndwww.nce.amadeus.net-git-scm-dzb-puppet-modules.git http://myuser:mypassword@rndwww.nce.amadeus.net/git/scm/dzb/puppet-modules.git /etc/puppet/environments/feature_DSS_123_Core_File_Permissions exited with 128: fatal: protocol error: bad line length character: <!DO fatal: The remote end hung up unexpectedly Not sure if this has anything to do with the problem

Anyway, when I look at /etc/puppet/environments I find: So, all but 3 branches are again big because of the packfile: [root@btxv0100 /etc/puppet/environments]# du -hs /etc/puppet/environments/* 154M /etc/puppet/environments/ap1 154M /etc/puppet/environments/btprd 154M /etc/puppet/environments/bugfix_dss_107 154M /etc/puppet/environments/bugfix_DSS_108_snmp_conf 154M /etc/puppet/environments/bugfix_DSS_112_error_hpacucli_cpqacuxe 154M /etc/puppet/environments/bugfix_DSS_113_couchbase_logfiles_perm 154M /etc/puppet/environments/bugfix_DSS_116_couchbase_app_value 154M /etc/puppet/environments/bugfix_DSS_136_firecell_location 154M /etc/puppet/environments/bugfix_DSS_137_XFS_missing 154M /etc/puppet/environments/bugfix_DSS_138_remove_reboot 154M /etc/puppet/environments/bugfix_DSS_140_lvm_regex_fix 154M /etc/puppet/environments/bugfix_DSS_152_mapr_lvm 154M /etc/puppet/environments/bugfix_DSS_153_remove_version_check 154M /etc/puppet/environments/bugfix_DSS_154_remove_stringify_facts_false_line 154M /etc/puppet/environments/bugfix_DSS_155_var_log_messages_not_compressed 154M /etc/puppet/environments/bugfix_DSS_159_rhel66_upgrade 154M /etc/puppet/environments/bugfix_DSS_225_firecell_timeout 4.3M /etc/puppet/environments/bugfix_DSS_230_cmdb_url_update 154M /etc/puppet/environments/bugfix_DSS_234_wrong_couchbase_permission 154M /etc/puppet/environments/bugfix_DSS_235_transparenthugepages_not_working 154M /etc/puppet/environments/bugfix_epel 154M /etc/puppet/environments/bugfix_mapr_storage_disable 154M /etc/puppet/environments/bugfix_mapr_storage_disable2 154M /etc/puppet/environments/cb 154M /etc/puppet/environments/dev 154M /etc/puppet/environments/dr 4.3M /etc/puppet/environments/feature_CDP_770_bashrc_change 154M /etc/puppet/environments/feature_docker_tests 3.8M /etc/puppet/environments/feature_DSS_111_filepermssions 154M /etc/puppet/environments/feature_DSS_114_couchbase_login_shell 154M /etc/puppet/environments/feature_DSS_115_cb_backup_restore_wrapper 154M /etc/puppet/environments/feature_DSS_118_firmware_version_fact 4.1M /etc/puppet/environments/feature_DSS_132_puppet_couchbase_logfilepermissions 154M /etc/puppet/environments/feature_DSS_134_puppet_bios_release_fact 154M /etc/puppet/environments/feature_DSS_135_topodb_module 154M /etc/puppet/environments/feature_DSS_144_couchbase3 154M /etc/puppet/environments/feature_DSS_224_test 154M /etc/puppet/environments/feature_DSS_229_controlm 154M /etc/puppet/environments/feature_DSS_231_mdb_logrotation 154M /etc/puppet/environments/feature_DSS_232_network_config 154M /etc/puppet/environments/feature_DSS_233_lom 154M /etc/puppet/environments/feature_DSS_237_couchbase_in_DRsite 154M /etc/puppet/environments/feature_DSS_33_mongodb_qradar 154M /etc/puppet/environments/feature_DSS_62_mongo 154M /etc/puppet/environments/feature_DSS_89_hostsfile_dist 154M /etc/puppet/environments/feature_DSS_96_patrol_user 155M /etc/puppet/environments/feature_DSS_99_Graphite 154M /etc/puppet/environments/feature_logserver 155M /etc/puppet/environments/feature_mapr 155M /etc/puppet/environments/feature_oozie 154M /etc/puppet/environments/fvt 154M /etc/puppet/environments/hotfix_DSS_128_bash_vulnerability_patch_installation 154M /etc/puppet/environments/hotfix_DSS_128_bash_vulnerability_patch_installation_dw 154M /etc/puppet/environments/mig 154M /etc/puppet/environments/mongodb_prd 154M /etc/puppet/environments/pdt 154M /etc/puppet/environments/ppt 154M /etc/puppet/environments/prd 154M /etc/puppet/environments/qrt 154M /etc/puppet/environments/release_1_0_0 154M /etc/puppet/environments/release_1_0_1 154M /etc/puppet/environments/release_1_0_10 154M /etc/puppet/environments/release_1_0_11 154M /etc/puppet/environments/release_1_0_12 154M /etc/puppet/environments/release_1_0_2 154M /etc/puppet/environments/release_1_0_3 154M /etc/puppet/environments/release_1_0_4 154M /etc/puppet/environments/release_1_0_5 154M /etc/puppet/environments/release_1_0_6 154M /etc/puppet/environments/release_1_0_7 154M /etc/puppet/environments/release_1_0_8 154M /etc/puppet/environments/release_1_0_9 154M /etc/puppet/environments/rf2 154M /etc/puppet/environments/rfq 154M /etc/puppet/environments/skl 154M /etc/puppet/environments/stg 154M /etc/puppet/environments/sup 154M /etc/puppet/environments/trunk 154M /etc/puppet/environments/tst 154M /etc/puppet/environments/uat [root@btxv0100 /etc/puppet/environments]#

Strangely enough, 3 environments are small as expected and have no packfile ! Most, but not all pack files have the same md5sum: [root@btxv0100 /etc/puppet/environments]# md5sum /.git/objects/pack/.pack 72f657834e712bc940d8f16d5e393ae0 ap1/.git/objects/pack/pack-0e6d1d7d0cf076865048e15f39b325c757a88ba6.pack 72f657834e712bc940d8f16d5e393ae0 btprd/.git/objects/pack/pack-0e6d1d7d0cf076865048e15f39b325c757a88ba6.pack 72f657834e712bc940d8f16d5e393ae0 bugfix_dss_107/.git/objects/pack/pack-0e6d1d7d0cf076865048e15f39b325c757a88ba6.pack 72f657834e712bc940d8f16d5e393ae0 bugfix_DSS_108_snmp_conf/.git/objects/pack/pack-0e6d1d7d0cf076865048e15f39b325c757a88ba6.pack ad05663d00eac6aa510f239052b2aaf9 bugfix_DSS_112_error_hpacucli_cpqacuxe/.git/objects/pack/pack-0e6d1d7d0cf076865048e15f39b325c757a88ba6.pack 72f657834e712bc940d8f16d5e393ae0 bugfix_DSS_113_couchbase_logfiles_perm/.git/objects/pack/pack-0e6d1d7d0cf076865048e15f39b325c757a88ba6.pack ad05663d00eac6aa510f239052b2aaf9 bugfix_DSS_116_couchbase_app_value/.git/objects/pack/pack-0e6d1d7d0cf076865048e15f39b325c757a88ba6.pack ad05663d00eac6aa510f239052b2aaf9 bugfix_DSS_136_firecell_location/.git/objects/pack/pack-0e6d1d7d0cf076865048e15f39b325c757a88ba6.pack 72f657834e712bc940d8f16d5e393ae0 bugfix_DSS_137_XFS_missing/.git/objects/pack/pack-0e6d1d7d0cf076865048e15f39b325c757a88ba6.pack 72f657834e712bc940d8f16d5e393ae0 bugfix_DSS_138_remove_reboot/.git/objects/pack/pack-0e6d1d7d0cf076865048e15f39b325c757a88ba6.pack 72f657834e712bc940d8f16d5e393ae0 bugfix_DSS_140_lvm_regex_fix/.git/objects/pack/pack-0e6d1d7d0cf076865048e15f39b325c757a88ba6.pack ad05663d00eac6aa510f239052b2aaf9 bugfix_DSS_152_mapr_lvm/.git/objects/pack/pack-0e6d1d7d0cf076865048e15f39b325c757a88ba6.pack ad05663d00eac6aa510f239052b2aaf9 bugfix_DSS_153_remove_version_check/.git/objects/pack/pack-0e6d1d7d0cf076865048e15f39b325c757a88ba6.pack 72f657834e712bc940d8f16d5e393ae0 bugfix_DSS_154_remove_stringify_facts_false_line/.git/objects/pack/pack-0e6d1d7d0cf076865048e15f39b325c757a88ba6.pack ad05663d00eac6aa510f239052b2aaf9 bugfix_DSS_155_var_log_messages_not_compressed/.git/objects/pack/pack-0e6d1d7d0cf076865048e15f39b325c757a88ba6.pack ad05663d00eac6aa510f239052b2aaf9 bugfix_DSS_159_rhel66_upgrade/.git/objects/pack/pack-0e6d1d7d0cf076865048e15f39b325c757a88ba6.pack ad05663d00eac6aa510f239052b2aaf9 bugfix_DSS_225_firecell_timeout/.git/objects/pack/pack-0e6d1d7d0cf076865048e15f39b325c757a88ba6.pack 72f657834e712bc940d8f16d5e393ae0 bugfix_DSS_234_wrong_couchbase_permission/.git/objects/pack/pack-0e6d1d7d0cf076865048e15f39b325c757a88ba6.pack 72f657834e712bc940d8f16d5e393ae0 bugfix_DSS_235_transparenthugepages_not_working/.git/objects/pack/pack-0e6d1d7d0cf076865048e15f39b325c757a88ba6.pack 72f657834e712bc940d8f16d5e393ae0 bugfix_epel/.git/objects/pack/pack-0e6d1d7d0cf076865048e15f39b325c757a88ba6.pack 72f657834e712bc940d8f16d5e393ae0 bugfix_mapr_storage_disable2/.git/objects/pack/pack-0e6d1d7d0cf076865048e15f39b325c757a88ba6.pack 72f657834e712bc940d8f16d5e393ae0 bugfix_mapr_storage_disable/.git/objects/pack/pack-0e6d1d7d0cf076865048e15f39b325c757a88ba6.pack 72f657834e712bc940d8f16d5e393ae0 cb/.git/objects/pack/pack-0e6d1d7d0cf076865048e15f39b325c757a88ba6.pack 72f657834e712bc940d8f16d5e393ae0 dev/.git/objects/pack/pack-0e6d1d7d0cf076865048e15f39b325c757a88ba6.pack 72f657834e712bc940d8f16d5e393ae0 dr/.git/objects/pack/pack-0e6d1d7d0cf076865048e15f39b325c757a88ba6.pack 72f657834e712bc940d8f16d5e393ae0 feature_docker_tests/.git/objects/pack/pack-0e6d1d7d0cf076865048e15f39b325c757a88ba6.pack 72f657834e712bc940d8f16d5e393ae0 feature_DSS_114_couchbase_login_shell/.git/objects/pack/pack-0e6d1d7d0cf076865048e15f39b325c757a88ba6.pack 72f657834e712bc940d8f16d5e393ae0 feature_DSS_115_cb_backup_restore_wrapper/.git/objects/pack/pack-0e6d1d7d0cf076865048e15f39b325c757a88ba6.pack 72f657834e712bc940d8f16d5e393ae0 feature_DSS_118_firmware_version_fact/.git/objects/pack/pack-0e6d1d7d0cf076865048e15f39b325c757a88ba6.pack 72f657834e712bc940d8f16d5e393ae0 feature_DSS_134_puppet_bios_release_fact/.git/objects/pack/pack-0e6d1d7d0cf076865048e15f39b325c757a88ba6.pack 72f657834e712bc940d8f16d5e393ae0 feature_DSS_135_topodb_module/.git/objects/pack/pack-0e6d1d7d0cf076865048e15f39b325c757a88ba6.pack 72f657834e712bc940d8f16d5e393ae0 feature_DSS_144_couchbase3/.git/objects/pack/pack-0e6d1d7d0cf076865048e15f39b325c757a88ba6.pack 72f657834e712bc940d8f16d5e393ae0 feature_DSS_224_test/.git/objects/pack/pack-0e6d1d7d0cf076865048e15f39b325c757a88ba6.pack 72f657834e712bc940d8f16d5e393ae0 feature_DSS_229_controlm/.git/objects/pack/pack-0e6d1d7d0cf076865048e15f39b325c757a88ba6.pack 72f657834e712bc940d8f16d5e393ae0 feature_DSS_231_mdb_logrotation/.git/objects/pack/pack-0e6d1d7d0cf076865048e15f39b325c757a88ba6.pack 72f657834e712bc940d8f16d5e393ae0 feature_DSS_232_network_config/.git/objects/pack/pack-0e6d1d7d0cf076865048e15f39b325c757a88ba6.pack 72f657834e712bc940d8f16d5e393ae0 feature_DSS_233_lom/.git/objects/pack/pack-0e6d1d7d0cf076865048e15f39b325c757a88ba6.pack 72f657834e712bc940d8f16d5e393ae0 feature_DSS_237_couchbase_in_DRsite/.git/objects/pack/pack-0e6d1d7d0cf076865048e15f39b325c757a88ba6.pack 72f657834e712bc940d8f16d5e393ae0 feature_DSS_33_mongodb_qradar/.git/objects/pack/pack-0e6d1d7d0cf076865048e15f39b325c757a88ba6.pack 72f657834e712bc940d8f16d5e393ae0 feature_DSS_62_mongo/.git/objects/pack/pack-0e6d1d7d0cf076865048e15f39b325c757a88ba6.pack 72f657834e712bc940d8f16d5e393ae0 feature_DSS_89_hostsfile_dist/.git/objects/pack/pack-0e6d1d7d0cf076865048e15f39b325c757a88ba6.pack 72f657834e712bc940d8f16d5e393ae0 feature_DSS_96_patrol_user/.git/objects/pack/pack-0e6d1d7d0cf076865048e15f39b325c757a88ba6.pack 72f657834e712bc940d8f16d5e393ae0 feature_DSS_99_Graphite/.git/objects/pack/pack-0e6d1d7d0cf076865048e15f39b325c757a88ba6.pack 72f657834e712bc940d8f16d5e393ae0 feature_logserver/.git/objects/pack/pack-0e6d1d7d0cf076865048e15f39b325c757a88ba6.pack 72f657834e712bc940d8f16d5e393ae0 feature_mapr/.git/objects/pack/pack-0e6d1d7d0cf076865048e15f39b325c757a88ba6.pack 72f657834e712bc940d8f16d5e393ae0 feature_oozie/.git/objects/pack/pack-0e6d1d7d0cf076865048e15f39b325c757a88ba6.pack 72f657834e712bc940d8f16d5e393ae0 fvt/.git/objects/pack/pack-0e6d1d7d0cf076865048e15f39b325c757a88ba6.pack 72f657834e712bc940d8f16d5e393ae0 hotfix_DSS_128_bash_vulnerability_patch_installation_dw/.git/objects/pack/pack-0e6d1d7d0cf076865048e15f39b325c757a88ba6.pack 72f657834e712bc940d8f16d5e393ae0 hotfix_DSS_128_bash_vulnerability_patch_installation/.git/objects/pack/pack-0e6d1d7d0cf076865048e15f39b325c757a88ba6.pack 72f657834e712bc940d8f16d5e393ae0 mig/.git/objects/pack/pack-0e6d1d7d0cf076865048e15f39b325c757a88ba6.pack 72f657834e712bc940d8f16d5e393ae0 mongodb_prd/.git/objects/pack/pack-0e6d1d7d0cf076865048e15f39b325c757a88ba6.pack 72f657834e712bc940d8f16d5e393ae0 pdt/.git/objects/pack/pack-0e6d1d7d0cf076865048e15f39b325c757a88ba6.pack 72f657834e712bc940d8f16d5e393ae0 ppt/.git/objects/pack/pack-0e6d1d7d0cf076865048e15f39b325c757a88ba6.pack 72f657834e712bc940d8f16d5e393ae0 prd/.git/objects/pack/pack-0e6d1d7d0cf076865048e15f39b325c757a88ba6.pack 72f657834e712bc940d8f16d5e393ae0 qrt/.git/objects/pack/pack-0e6d1d7d0cf076865048e15f39b325c757a88ba6.pack 72f657834e712bc940d8f16d5e393ae0 release_1_0_0/.git/objects/pack/pack-0e6d1d7d0cf076865048e15f39b325c757a88ba6.pack 72f657834e712bc940d8f16d5e393ae0 release_1_0_10/.git/objects/pack/pack-0e6d1d7d0cf076865048e15f39b325c757a88ba6.pack 72f657834e712bc940d8f16d5e393ae0 release_1_0_11/.git/objects/pack/pack-0e6d1d7d0cf076865048e15f39b325c757a88ba6.pack 72f657834e712bc940d8f16d5e393ae0 release_1_0_12/.git/objects/pack/pack-0e6d1d7d0cf076865048e15f39b325c757a88ba6.pack 72f657834e712bc940d8f16d5e393ae0 release_1_0_1/.git/objects/pack/pack-0e6d1d7d0cf076865048e15f39b325c757a88ba6.pack 72f657834e712bc940d8f16d5e393ae0 release_1_0_2/.git/objects/pack/pack-0e6d1d7d0cf076865048e15f39b325c757a88ba6.pack 72f657834e712bc940d8f16d5e393ae0 release_1_0_3/.git/objects/pack/pack-0e6d1d7d0cf076865048e15f39b325c757a88ba6.pack 72f657834e712bc940d8f16d5e393ae0 release_1_0_4/.git/objects/pack/pack-0e6d1d7d0cf076865048e15f39b325c757a88ba6.pack 72f657834e712bc940d8f16d5e393ae0 release_1_0_5/.git/objects/pack/pack-0e6d1d7d0cf076865048e15f39b325c757a88ba6.pack 72f657834e712bc940d8f16d5e393ae0 release_1_0_6/.git/objects/pack/pack-0e6d1d7d0cf076865048e15f39b325c757a88ba6.pack 72f657834e712bc940d8f16d5e393ae0 release_1_0_7/.git/objects/pack/pack-0e6d1d7d0cf076865048e15f39b325c757a88ba6.pack 72f657834e712bc940d8f16d5e393ae0 release_1_0_8/.git/objects/pack/pack-0e6d1d7d0cf076865048e15f39b325c757a88ba6.pack 72f657834e712bc940d8f16d5e393ae0 release_1_0_9/.git/objects/pack/pack-0e6d1d7d0cf076865048e15f39b325c757a88ba6.pack 72f657834e712bc940d8f16d5e393ae0 rf2/.git/objects/pack/pack-0e6d1d7d0cf076865048e15f39b325c757a88ba6.pack 72f657834e712bc940d8f16d5e393ae0 rfq/.git/objects/pack/pack-0e6d1d7d0cf076865048e15f39b325c757a88ba6.pack 72f657834e712bc940d8f16d5e393ae0 skl/.git/objects/pack/pack-0e6d1d7d0cf076865048e15f39b325c757a88ba6.pack 72f657834e712bc940d8f16d5e393ae0 stg/.git/objects/pack/pack-0e6d1d7d0cf076865048e15f39b325c757a88ba6.pack 72f657834e712bc940d8f16d5e393ae0 sup/.git/objects/pack/pack-0e6d1d7d0cf076865048e15f39b325c757a88ba6.pack 72f657834e712bc940d8f16d5e393ae0 trunk/.git/objects/pack/pack-0e6d1d7d0cf076865048e15f39b325c757a88ba6.pack 72f657834e712bc940d8f16d5e393ae0 tst/.git/objects/pack/pack-0e6d1d7d0cf076865048e15f39b325c757a88ba6.pack 72f657834e712bc940d8f16d5e393ae0 uat/.git/objects/pack/pack-0e6d1d7d0cf076865048e15f39b325c757a88ba6.pack

Which is different from the /opt/r10k copy: md5sum /opt/r10k/http---myuser-mypassword\@rndwww.nce.amadeus.net-git-scm-dzb-puppet-modules.git/objects/pack/pack-be8ba06bc48a5ef1a8eb1d901d2e90269a1e0cc6.pack 99b8ed70df9ffade4ee50b048a811136 /opt/r10k/http---myuser-mypassword@rndwww.nce.amadeus.net-git-scm-dzb-puppet-modules.git/objects/pack/pack-be8ba06bc48a5ef1a8eb1d901d2e90269a1e0cc6.packmailto:/opt/r10k/http---myuser-mypassword@rndwww.nce.amadeus.net-git-scm-dzb-puppet-modules.git/objects/pack/pack-be8ba06bc48a5ef1a8eb1d901d2e90269a1e0cc6.pack

I am puzzled.

Thanks again!

Michael

From: Adrien Thebo [mailto:notifications@github.com] Sent: 22 July 2015 21:41 To: puppetlabs/r10k r10k@noreply.github.com Cc: Michael EHRIG michael.ehrig@amadeus.com Subject: Re: [r10k] Question regarding git and pack files with r10k (#426)

Yeah, let's go with wiping the existing state and starting afresh; I'm pretty baffled that running a git fsck --full would have more errors than --no-full.

Out of curiosity, are /etc/puppet and /opt/r10k on the same mount?

— Reply to this email directly or view it on GitHubhttps://github.com/puppetlabs/r10k/issues/426#issuecomment-123839964.

IMPORTANT - CONFIDENTIALITY NOTICE - This e-mail is intended only for the use of the individual or entity shown above as addressees . It may contain information which is privileged, confidential or otherwise protected from disclosure under applicable laws . If the reader of this transmission is not the intended recipient, you are hereby notified that any dissemination, printing, distribution, copying, disclosure or the taking of any action in reliance on the contents of this information is strictly prohibited. If you have received this transmission in error, please immediately notify us by reply e-mail or using the address below and delete the message and any attachments from your system . Amadeus Data Processing GmbH Geschäftsführer: Dr. Wolfgang Krips Sitz der Gesellschaft: Erding HR München 48 199 Berghamer Strasse 6 85435 Erding Germany

#

command: r10k deploy environment -v > /tmp/r10kdeploy 2>&1

# [R10K::Task::Deployment::DeployEnvironments - INFO] Loading environments from all sources [R10K::Deployment::Environment - WARN] Environment "bugfix/DSS-108-snmp_conf" contained non-word characters; sanitizing to "bugfix_DSS_108_snmp_conf" [R10K::Deployment::Environment - WARN] Environment "bugfix/DSS-112-error-hpacucli-cpqacuxe" contained non-word characters; sanitizing to "bugfix_DSS_112_error_hpacucli_cpqacuxe" [R10K::Deployment::Environment - WARN] Environment "bugfix/DSS-113-couchbase-logfiles-perm" contained non-word characters; sanitizing to "bugfix_DSS_113_couchbase_logfiles_perm" [R10K::Deployment::Environment - WARN] Environment "bugfix/DSS-116-couchbase_app_value" contained non-word characters; sanitizing to "bugfix_DSS_116_couchbase_app_value" [R10K::Deployment::Environment - WARN] Environment "bugfix/DSS-136-firecell-location" contained non-word characters; sanitizing to "bugfix_DSS_136_firecell_location" [R10K::Deployment::Environment - WARN] Environment "bugfix/DSS-137-XFS-missing" contained non-word characters; sanitizing to "bugfix_DSS_137_XFS_missing" [R10K::Deployment::Environment - WARN] Environment "bugfix/DSS-138_remove_reboot" contained non-word characters; sanitizing to "bugfix_DSS_138_remove_reboot" [R10K::Deployment::Environment - WARN] Environment "bugfix/DSS-140-lvm-regex-fix" contained non-word characters; sanitizing to "bugfix_DSS_140_lvm_regex_fix" [R10K::Deployment::Environment - WARN] Environment "bugfix/DSS-152-mapr-lvm" contained non-word characters; sanitizing to "bugfix_DSS_152_mapr_lvm" [R10K::Deployment::Environment - WARN] Environment "bugfix/DSS-153-remove-version-check" contained non-word characters; sanitizing to "bugfix_DSS_153_remove_version_check" [R10K::Deployment::Environment - WARN] Environment "bugfix/DSS-154-remove-stringify_facts-false-line" contained non-word characters; sanitizing to "bugfix_DSS_154_remove_stringify_facts_false_line" [R10K::Deployment::Environment - WARN] Environment "bugfix/DSS-155-var-log-messages-not-compressed" contained non-word characters; sanitizing to "bugfix_DSS_155_var_log_messages_not_compressed" [R10K::Deployment::Environment - WARN] Environment "bugfix/DSS-159-rhel66-upgrade" contained non-word characters; sanitizing to "bugfix_DSS_159_rhel66_upgrade" [R10K::Deployment::Environment - WARN] Environment "bugfix/DSS-225-firecell-timeout" contained non-word characters; sanitizing to "bugfix_DSS_225_firecell_timeout" [R10K::Deployment::Environment - WARN] Environment "bugfix/DSS-230-cmdb-url-update" contained non-word characters; sanitizing to "bugfix_DSS_230_cmdb_url_update" [R10K::Deployment::Environment - WARN] Environment "bugfix/DSS-234-wrong-couchbase-permission" contained non-word characters; sanitizing to "bugfix_DSS_234_wrong_couchbase_permission" [R10K::Deployment::Environment - WARN] Environment "bugfix/DSS-235-transparenthugepages-not-working" contained non-word characters; sanitizing to "bugfix_DSS_235_transparenthugepages_not_working" [R10K::Deployment::Environment - WARN] Environment "bugfix/dss-107" contained non-word characters; sanitizing to "bugfix_dss_107" [R10K::Deployment::Environment - WARN] Environment "bugfix/epel" contained non-word characters; sanitizing to "bugfix_epel" [R10K::Deployment::Environment - WARN] Environment "bugfix/mapr-storage-disable" contained non-word characters; sanitizing to "bugfix_mapr_storage_disable" [R10K::Deployment::Environment - WARN] Environment "bugfix/mapr-storage-disable2" contained non-word characters; sanitizing to "bugfix_mapr_storage_disable2" [R10K::Deployment::Environment - WARN] Environment "feature/CDP-770-bashrc-change" contained non-word characters; sanitizing to "feature_CDP_770_bashrc_change" [R10K::Deployment::Environment - WARN] Environment "feature/DSS-111-filepermssions" contained non-word characters; sanitizing to "feature_DSS_111_filepermssions" [R10K::Deployment::Environment - WARN] Environment "feature/DSS-114-couchbase-login-shell" contained non-word characters; sanitizing to "feature_DSS_114_couchbase_login_shell" [R10K::Deployment::Environment - WARN] Environment "feature/DSS-115-cb-backup-restore-wrapper" contained non-word characters; sanitizing to "feature_DSS_115_cb_backup_restore_wrapper" [R10K::Deployment::Environment - WARN] Environment "feature/DSS-118-firmware-version-fact" contained non-word characters; sanitizing to "feature_DSS_118_firmware_version_fact" [R10K::Deployment::Environment - WARN] Environment "feature/DSS-123-Core-File-Permissions" contained non-word characters; sanitizing to "feature_DSS_123_Core_File_Permissions" [R10K::Deployment::Environment - WARN] Environment "feature/DSS-132-puppet-acl" contained non-word characters; sanitizing to "feature_DSS_132_puppet_acl" [R10K::Deployment::Environment - WARN] Environment "feature/DSS-132-puppet-couchbase-logfilepermissions" contained non-word characters; sanitizing to "feature_DSS_132_puppet_couchbase_logfilepermissions" [R10K::Deployment::Environment - WARN] Environment "feature/DSS-134-puppet-bios-release-fact" contained non-word characters; sanitizing to "feature_DSS_134_puppet_bios_release_fact" [R10K::Deployment::Environment - WARN] Environment "feature/DSS-135-topodb-module" contained non-word characters; sanitizing to "feature_DSS_135_topodb_module" [R10K::Deployment::Environment - WARN] Environment "feature/DSS-144_couchbase3" contained non-word characters; sanitizing to "feature_DSS_144_couchbase3" [R10K::Deployment::Environment - WARN] Environment "feature/DSS-224-test" contained non-word characters; sanitizing to "feature_DSS_224_test" [R10K::Deployment::Environment - WARN] Environment "feature/DSS-229-controlm" contained non-word characters; sanitizing to "feature_DSS_229_controlm" [R10K::Deployment::Environment - WARN] Environment "feature/DSS-231-mdb-logrotation" contained non-word characters; sanitizing to "feature_DSS_231_mdb_logrotation" [R10K::Deployment::Environment - WARN] Environment "feature/DSS-232-network-config" contained non-word characters; sanitizing to "feature_DSS_232_network_config" [R10K::Deployment::Environment - WARN] Environment "feature/DSS-233-lom" contained non-word characters; sanitizing to "feature_DSS_233_lom" [R10K::Deployment::Environment - WARN] Environment "feature/DSS-237-couchbase-in-DRsite" contained non-word characters; sanitizing to "feature_DSS_237_couchbase_in_DRsite" [R10K::Deployment::Environment - WARN] Environment "feature/DSS-33-mongodb-qradar" contained non-word characters; sanitizing to "feature_DSS_33_mongodb_qradar" [R10K::Deployment::Environment - WARN] Environment "feature/DSS-62-mongo" contained non-word characters; sanitizing to "feature_DSS_62_mongo" [R10K::Deployment::Environment - WARN] Environment "feature/DSS-89-hostsfile-dist" contained non-word characters; sanitizing to "feature_DSS_89_hostsfile_dist" [R10K::Deployment::Environment - WARN] Environment "feature/DSS-99-Graphite" contained non-word characters; sanitizing to "feature_DSS_99_Graphite" [R10K::Deployment::Environment - WARN] Environment "feature/DSS_96_patrol_user" contained non-word characters; sanitizing to "feature_DSS_96_patrol_user" [R10K::Deployment::Environment - WARN] Environment "feature/docker_tests" contained non-word characters; sanitizing to "feature_docker_tests" [R10K::Deployment::Environment - WARN] Environment "feature/logserver" contained non-word characters; sanitizing to "feature_logserver" [R10K::Deployment::Environment - WARN] Environment "feature/mapr" contained non-word characters; sanitizing to "feature_mapr" [R10K::Deployment::Environment - WARN] Environment "feature/oozie" contained non-word characters; sanitizing to "feature_oozie" [R10K::Deployment::Environment - WARN] Environment "hotfix/DSS-128-bash-vulnerability-patch-installation" contained non-word characters; sanitizing to "hotfix_DSS_128_bash_vulnerability_patch_installation" [R10K::Deployment::Environment - WARN] Environment "hotfix/DSS-128-bash-vulnerability-patch-installation-dw" contained non-word characters; sanitizing to "hotfix_DSS_128_bash_vulnerability_patch_installation_dw" [R10K::Deployment::Environment - WARN] Environment "mongodb-prd" contained non-word characters; sanitizing to "mongodb_prd" [R10K::Deployment::Environment - WARN] Environment "release/1.0.0" contained non-word characters; sanitizing to "release_1_0_0" [R10K::Deployment::Environment - WARN] Environment "release/1.0.1" contained non-word characters; sanitizing to "release_1_0_1" [R10K::Deployment::Environment - WARN] Environment "release/1.0.10" contained non-word characters; sanitizing to "release_1_0_10" [R10K::Deployment::Environment - WARN] Environment "release/1.0.11" contained non-word characters; sanitizing to "release_1_0_11" [R10K::Deployment::Environment - WARN] Environment "release/1.0.12" contained non-word characters; sanitizing to "release_1_0_12" [R10K::Deployment::Environment - WARN] Environment "release/1.0.2" contained non-word characters; sanitizing to "release_1_0_2" [R10K::Deployment::Environment - WARN] Environment "release/1.0.3" contained non-word characters; sanitizing to "release_1_0_3" [R10K::Deployment::Environment - WARN] Environment "release/1.0.4" contained non-word characters; sanitizing to "release_1_0_4" [R10K::Deployment::Environment - WARN] Environment "release/1.0.5" contained non-word characters; sanitizing to "release_1_0_5" [R10K::Deployment::Environment - WARN] Environment "release/1.0.6" contained non-word characters; sanitizing to "release_1_0_6" [R10K::Deployment::Environment - WARN] Environment "release/1.0.7" contained non-word characters; sanitizing to "release_1_0_7" [R10K::Deployment::Environment - WARN] Environment "release/1.0.8" contained non-word characters; sanitizing to "release_1_0_8" [R10K::Deployment::Environment - WARN] Environment "release/1.0.9" contained non-word characters; sanitizing to "release_1_0_9" [R10K::Task::Environment::Deploy - NOTICE] Deploying environment ap1 [R10K::Task::Environment::Deploy - NOTICE] Deploying environment btprd [R10K::Task::Environment::Deploy - NOTICE] Deploying environment bugfix_DSS_108_snmp_conf [R10K::Task::Environment::Deploy - NOTICE] Deploying environment bugfix_DSS_112_error_hpacucli_cpqacuxe [R10K::Task::Environment::Deploy - NOTICE] Deploying environment bugfix_DSS_113_couchbase_logfiles_perm [R10K::Task::Environment::Deploy - NOTICE] Deploying environment bugfix_DSS_116_couchbase_app_value [R10K::Task::Environment::Deploy - NOTICE] Deploying environment bugfix_DSS_136_firecell_location [R10K::Task::Environment::Deploy - NOTICE] Deploying environment bugfix_DSS_137_XFS_missing [R10K::Task::Environment::Deploy - NOTICE] Deploying environment bugfix_DSS_138_remove_reboot [R10K::Task::Environment::Deploy - NOTICE] Deploying environment bugfix_DSS_140_lvm_regex_fix [R10K::Task::Environment::Deploy - NOTICE] Deploying environment bugfix_DSS_152_mapr_lvm [R10K::Task::Environment::Deploy - NOTICE] Deploying environment bugfix_DSS_153_remove_version_check [R10K::Task::Environment::Deploy - NOTICE] Deploying environment bugfix_DSS_154_remove_stringify_facts_false_line [R10K::Task::Environment::Deploy - NOTICE] Deploying environment bugfix_DSS_155_var_log_messages_not_compressed [R10K::Task::Environment::Deploy - NOTICE] Deploying environment bugfix_DSS_159_rhel66_upgrade [R10K::Task::Environment::Deploy - NOTICE] Deploying environment bugfix_DSS_225_firecell_timeout [R10K::Task::Environment::Deploy - NOTICE] Deploying environment bugfix_DSS_230_cmdb_url_update [R10K::Task::Environment::Deploy - NOTICE] Deploying environment bugfix_DSS_234_wrong_couchbase_permission [R10K::Task::Environment::Deploy - NOTICE] Deploying environment bugfix_DSS_235_transparenthugepages_not_working [R10K::Task::Environment::Deploy - NOTICE] Deploying environment bugfix_dss_107 [R10K::Task::Environment::Deploy - NOTICE] Deploying environment bugfix_epel [R10K::Task::Environment::Deploy - NOTICE] Deploying environment bugfix_mapr_storage_disable [R10K::Task::Environment::Deploy - NOTICE] Deploying environment bugfix_mapr_storage_disable2 [R10K::Task::Environment::Deploy - NOTICE] Deploying environment cb [R10K::Task::Environment::Deploy - NOTICE] Deploying environment dev [R10K::Task::Environment::Deploy - NOTICE] Deploying environment dr [R10K::Task::Environment::Deploy - NOTICE] Deploying environment feature_CDP_770_bashrc_change [R10K::Task::Environment::Deploy - NOTICE] Deploying environment feature_DSS_111_filepermssions [R10K::Task::Environment::Deploy - NOTICE] Deploying environment feature_DSS_114_couchbase_login_shell [R10K::Task::Environment::Deploy - NOTICE] Deploying environment feature_DSS_115_cb_backup_restore_wrapper [R10K::Task::Environment::Deploy - NOTICE] Deploying environment feature_DSS_118_firmware_version_fact [R10K::Task::Environment::Deploy - NOTICE] Deploying environment feature_DSS_123_Core_File_Permissions [R10K::TaskRunner - ERROR] Task #R10K::Task::Environment::Deploy:0x7f3c98b51c20 failed while running: Command git clone --reference /opt/r10k/http---myuser-mypassword@rndwww.nce.amadeus.net-git-scm-dzb-puppet-modules.git http://myuser:mypassword@rndwww.nce.amadeus.net/git/scm/dzb/puppet-modules.git /etc/puppet/environments/feature_DSS_123_Core_File_Permissions exited with 128: fatal: protocol error: bad line length character: <!DO fatal: The remote end hung up unexpectedly

[R10K::Task::Environment::Deploy - NOTICE] Deploying environment feature_DSS_132_puppet_acl [R10K::TaskRunner - ERROR] Task #R10K::Task::Environment::Deploy:0x7f3c98b51c70 failed while running: Command git clone --reference /opt/r10k/http---myuser-mypassword@rndwww.nce.amadeus.net-git-scm-dzb-puppet-modules.git http://myuser:mypassword@rndwww.nce.amadeus.net/git/scm/dzb/puppet-modules.git /etc/puppet/environments/feature_DSS_132_puppet_acl exited with 128: fatal: protocol error: bad line length character: <!DO fatal: The remote end hung up unexpectedly

[R10K::Task::Environment::Deploy - NOTICE] Deploying environment feature_DSS_132_puppet_couchbase_logfilepermissions [R10K::Task::Environment::Deploy - NOTICE] Deploying environment feature_DSS_134_puppet_bios_release_fact [R10K::Task::Environment::Deploy - NOTICE] Deploying environment feature_DSS_135_topodb_module [R10K::Task::Environment::Deploy - NOTICE] Deploying environment feature_DSS_144_couchbase3 [R10K::Task::Environment::Deploy - NOTICE] Deploying environment feature_DSS_224_test [R10K::Task::Environment::Deploy - NOTICE] Deploying environment feature_DSS_229_controlm [R10K::Task::Environment::Deploy - NOTICE] Deploying environment feature_DSS_231_mdb_logrotation [R10K::Task::Environment::Deploy - NOTICE] Deploying environment feature_DSS_232_network_config [R10K::Task::Environment::Deploy - NOTICE] Deploying environment feature_DSS_233_lom [R10K::Task::Environment::Deploy - NOTICE] Deploying environment feature_DSS_237_couchbase_in_DRsite [R10K::Task::Environment::Deploy - NOTICE] Deploying environment feature_DSS_33_mongodb_qradar [R10K::Task::Environment::Deploy - NOTICE] Deploying environment feature_DSS_62_mongo [R10K::Task::Environment::Deploy - NOTICE] Deploying environment feature_DSS_89_hostsfile_dist [R10K::Task::Environment::Deploy - NOTICE] Deploying environment feature_DSS_99_Graphite [R10K::Task::Environment::Deploy - NOTICE] Deploying environment feature_DSS_96_patrol_user [R10K::Task::Environment::Deploy - NOTICE] Deploying environment feature_docker_tests [R10K::Task::Environment::Deploy - NOTICE] Deploying environment feature_logserver [R10K::Task::Environment::Deploy - NOTICE] Deploying environment feature_mapr [R10K::Task::Environment::Deploy - NOTICE] Deploying environment feature_oozie [R10K::Task::Environment::Deploy - NOTICE] Deploying environment fvt [R10K::Task::Environment::Deploy - NOTICE] Deploying environment hotfix_DSS_128_bash_vulnerability_patch_installation [R10K::Task::Environment::Deploy - NOTICE] Deploying environment hotfix_DSS_128_bash_vulnerability_patch_installation_dw [R10K::Task::Environment::Deploy - NOTICE] Deploying environment mig [R10K::Task::Environment::Deploy - NOTICE] Deploying environment mongodb_prd [R10K::Task::Environment::Deploy - NOTICE] Deploying environment pdt [R10K::Task::Environment::Deploy - NOTICE] Deploying environment ppt [R10K::Task::Environment::Deploy - NOTICE] Deploying environment prd [R10K::Task::Environment::Deploy - NOTICE] Deploying environment qrt [R10K::Task::Environment::Deploy - NOTICE] Deploying environment release_1_0_0 [R10K::Task::Environment::Deploy - NOTICE] Deploying environment release_1_0_1 [R10K::Task::Environment::Deploy - NOTICE] Deploying environment release_1_0_10 [R10K::Task::Environment::Deploy - NOTICE] Deploying environment release_1_0_11 [R10K::Task::Environment::Deploy - NOTICE] Deploying environment release_1_0_12 [R10K::Task::Environment::Deploy - NOTICE] Deploying environment release_1_0_2 [R10K::Task::Environment::Deploy - NOTICE] Deploying environment release_1_0_3 [R10K::Task::Environment::Deploy - NOTICE] Deploying environment release_1_0_4 [R10K::Task::Environment::Deploy - NOTICE] Deploying environment release_1_0_5 [R10K::Task::Environment::Deploy - NOTICE] Deploying environment release_1_0_6 [R10K::Task::Environment::Deploy - NOTICE] Deploying environment release_1_0_7 [R10K::Task::Environment::Deploy - NOTICE] Deploying environment release_1_0_8 [R10K::Task::Environment::Deploy - NOTICE] Deploying environment release_1_0_9 [R10K::Task::Environment::Deploy - NOTICE] Deploying environment rf2 [R10K::Task::Environment::Deploy - NOTICE] Deploying environment rfq [R10K::Task::Environment::Deploy - NOTICE] Deploying environment skl [R10K::Task::Environment::Deploy - NOTICE] Deploying environment stg [R10K::Task::Environment::Deploy - NOTICE] Deploying environment sup [R10K::Task::Environment::Deploy - NOTICE] Deploying environment trunk [R10K::Task::Environment::Deploy - NOTICE] Deploying environment tst [R10K::Task::Environment::Deploy - NOTICE] Deploying environment uat [R10K::Task::Deployment::PurgeEnvironments - INFO] Purging stale environments from /etc/puppet/environments

adrienthebo commented 9 years ago

This error:

protocol error: bad line length character: <!DO

is extremely interesting. What are you using to host your Git repositories?

adrienthebo commented 9 years ago

The reason I ask is that a cursory Googling of the error turns up this:

http://stackoverflow.com/questions/7152285/issue-with-git-hosted-on-iis-with-bonobo

I'm wondering if somehow your packfiles are somehow getting mangled during the download process.

audimann commented 9 years ago

Hi Adrien,

Very interesting, indeed. We’re using Atlassian – Stash to host our git repo.

Thanks Michael

From: Adrien Thebo [mailto:notifications@github.com] Sent: 13 August 2015 23:49 To: puppetlabs/r10k r10k@noreply.github.com Cc: Michael EHRIG michael.ehrig@amadeus.com Subject: Re: [r10k] Question regarding git and pack files with r10k (#426)

The reason I ask is that a cursory Googling of the error turns up this:

http://stackoverflow.com/questions/7152285/issue-with-git-hosted-on-iis-with-bonobo

I'm wondering if somehow your packfiles are somehow getting mangled during the download process.

— Reply to this email directly or view it on GitHubhttps://github.com/puppetlabs/r10k/issues/426#issuecomment-130857526.

IMPORTANT - CONFIDENTIALITY NOTICE - This e-mail is intended only for the use of the individual or entity shown above as addressees . It may contain information which is privileged, confidential or otherwise protected from disclosure under applicable laws . If the reader of this transmission is not the intended recipient, you are hereby notified that any dissemination, printing, distribution, copying, disclosure or the taking of any action in reliance on the contents of this information is strictly prohibited. If you have received this transmission in error, please immediately notify us by reply e-mail or using the address below and delete the message and any attachments from your system . Amadeus Data Processing GmbH Geschäftsführer: Dr. Wolfgang Krips Sitz der Gesellschaft: Erding HR München 48 199 Berghamer Strasse 6 85435 Erding Germany

adrienthebo commented 9 years ago

At this point I'm not really sure what to say - it looks like Git itself is the component that's malfunctioning. If you're seeing things like malformed length errors when you're trying to do a git clone, then there is something deeply wrong with your Git infrastructure. In addition if you're getting packfiles with the same filename but different checksums, something is deeply, deeply wrong with how Git is behaving. These things simply shouldn't be possible.

I didn't realize that you were having issues with the protocol errors that you indicated above, which is interesting. I would like to dive into this a bit more and get a better view of what's going on. To do this I would like to see what happens when you do a full r10k run. You can create a separate r10k.yaml to do this and have a fresh cachedir and basedir so that this environment is entirely pristine and not affect your production environments. But however you choose to do it, could you run r10k with a fresh cachedir and basedir and attach the logs of running r10k with -v debug2 ? If this has sensitive information you can send the log to adrien@puppetlabs.com, and if you want to encrypt it my GPG short ID is 6F1C4519 and my fingerprint is " 14E2 2854 D611 F131 7D51 02EC 1CEB 563E 6F1C 4519".

github-actions[bot] commented 3 years ago

This issue has been marked stale because it has had no activity for 60 days. The Puppet Team is actively prioritizing existing bugs and new features, if this issue is still important to you please comment and we will add this to our backlog to complete. Otherwise, it will be closed in 7 days.