search

xorpaul / g10k

my r10k fork in Go
Apache License 2.0
127 stars 51 forks source link

data race with v #114

Closed sfdc-afraley closed 5 years ago

sfdc-afraley commented 6 years ago

I upgraded from v0.4.4 to v0.5.3 and started seeing the following. If I clear the cache it works on the first run, but once the cache is populated I get this.

"/opt/puppetlabs/puppet/bin/g10k" -config "/etc/puppetlabs/g10k/g10k_config.yaml" -verbose -maxworker 10 -maxextractworker 10

WARNING: DATA RACE
Write at 0x00c0001c8950 by main goroutine:
  main.resolvePuppetfile()
      /home/andpaul/dev/go/src/github.com/xorpaul/g10k/puppetfile.go:228 +0x1a35
  main.resolvePuppetEnvironment()
      /home/andpaul/dev/go/src/github.com/xorpaul/g10k/puppetfile.go:148 +0x3db
  main.main()
      /home/andpaul/dev/go/src/github.com/xorpaul/g10k/g10k.go:235 +0x145a

Previous read at 0x00c0001c8950 by goroutine 281:
  main.resolvePuppetfile.func3()
      /home/andpaul/dev/go/src/github.com/xorpaul/g10k/puppetfile.go:330 +0x7fd

Goroutine 281 (running) created at:
  main.resolvePuppetfile()
      /home/andpaul/dev/go/src/github.com/xorpaul/g10k/puppetfile.go:276 +0x174f
  main.resolvePuppetEnvironment()
      /home/andpaul/dev/go/src/github.com/xorpaul/g10k/puppetfile.go:148 +0x3db
  main.main()
      /home/andpaul/dev/go/src/github.com/xorpaul/g10k/g10k.go:235 +0x145a
xorpaul commented 6 years ago

I don't see anything suspicious in the line numbers from the your output.

Also I can't reproduce it with my test g10k config.

How many Forge and Git modules are you trying to sync? Does it work if you only update a single branch with -branch?

Could you do a g10k run with the -debug CLI parameter and show me the output? (You should filter out your private Git server or sensitive output)

Please use this g10k version, so I'm 100% sure I'm looking at the correct lines: g10k-race.zip

deferraz commented 5 years ago

same problem here, i tried with the binary zipped and still got the data race:

==================
WARNING: DATA RACE
Read at 0x00c0004f2100 by goroutine 64:
  main.resolvePuppetfile.func3()
      /home/andpaul/dev/go/src/github.com/xorpaul/g10k/puppetfile.go:330 +0x7fd

Previous write at 0x00c0004f2100 by main goroutine:
  main.resolvePuppetfile()
      /home/andpaul/dev/go/src/github.com/xorpaul/g10k/puppetfile.go:228 +0x1a35
  main.resolvePuppetEnvironment()
      /home/andpaul/dev/go/src/github.com/xorpaul/g10k/puppetfile.go:148 +0x3db
  main.main()
      /home/andpaul/dev/go/src/github.com/xorpaul/g10k/g10k.go:232 +0x9f4

Goroutine 64 (running) created at:
  main.resolvePuppetfile()
      /home/andpaul/dev/go/src/github.com/xorpaul/g10k/puppetfile.go:276 +0x174f
  main.resolvePuppetEnvironment()
      /home/andpaul/dev/go/src/github.com/xorpaul/g10k/puppetfile.go:148 +0x3db
  main.main()
      /home/andpaul/dev/go/src/github.com/xorpaul/g10k/g10k.go:232 +0x9f4

For my ambient, I am specifying only the branch production with the -branch option with the same problem

xorpaul commented 5 years ago

Could you do a g10k run with the -debug CLI parameter and show me the output? (You should filter out your private Git server or other sensitive output)

deferraz commented 5 years ago 
2019/01/10 10:36:49 DEBUG main(): Using as config file: /etc/puppetlabs/r10k/r10k.yaml
2019/01/10 10:36:49 DEBUG readConfigfile(): Trying to read g10k config file: /etc/puppetlabs/r10k/r10k.yaml
2019/01/10 10:36:49 DEBUG checkDirAndCreate(): Using as cachedir from g10k config /etc/puppetlabs/r10k/r10k.yaml: /var/cache/r10k/
2019/01/10 10:36:49 DEBUG checkDirAndCreate(): Using as cachedir: /var/cache/r10k/
2019/01/10 10:36:49 DEBUG checkDirAndCreate(): Using as cachedir/forge: /var/cache/r10k/forge/
2019/01/10 10:36:49 DEBUG checkDirAndCreate(): Using as cachedir/modules: /var/cache/r10k/modules/
2019/01/10 10:36:49 DEBUG checkDirAndCreate(): Using as cachedir/environments: /var/cache/r10k/environments/
2019/01/10 10:36:49 DEBUG checkDirAndCreate(): Using as basedir for source ***: /etc/puppetlabs/code/environments/
2019/01/10 10:36:49 DEBUG checkDirAndCreate(): Using as basedir for source ***: /etc/puppetlabs/code/environments/
2019/01/10 10:36:49 DEBUG Puppet environment: *** ({Remote:git@xxxxxxxxxxxxxxx Basedir:/etc/puppetlabs/code/environments/ Prefix:true PrivateKey:/opt/puppetlabs/server/data/puppetserver/.ssh/id_rsa ForceForgeVersions:false WarnMissingBranch:true ExitIfUnreachable:false AutoCorrectEnvironmentNames:correct_and_warn})
2019/01/10 10:36:49 DEBUG checkDirAndCreate(): Using as basedir for source ***: /etc/puppetlabs/code/environments/
2019/01/10 10:36:49 DEBUG Puppet environment: *** ({Remote:git@xxxxxxxxxxxxxxxxx Basedir:/etc/puppetlabs/code/environments/ Prefix:true PrivateKey:/opt/puppetlabs/server/data/puppetserver/.ssh/id_rsa ForceForgeVersions:false WarnMissingBranch:true ExitIfUnreachable:false AutoCorrectEnvironmentNames:correct_and_warn})
2019/01/10 10:36:49 DEBUG checkDirAndCreate(): Using as basedir: /etc/puppetlabs/code/environments/
2019/01/10 10:36:49 DEBUG Puppet environment: *** ({Remote:git@xxxxxxxxxxxxxxxxxx Basedir:/etc/puppetlabs/code/environments/ Prefix:true PrivateKey:/opt/puppetlabs/server/data/puppetserver/.ssh/id_rsa ForceForgeVersions:false WarnMissingBranch:true ExitIfUnreachable:false AutoCorrectEnvironmentNames:correct_and_warn})
2019/01/10 10:36:49 DEBUG checkDirAndCreate(): Using as basedir for source ***: /etc/puppetlabs/code/environments/
2019/01/10 10:36:49 DEBUG checkDirAndCreate(): Using as basedir: /etc/puppetlabs/code/environments/
2019/01/10 10:36:49 DEBUG checkDirAndCreate(): Using as basedir: /etc/puppetlabs/code/environments/
2019/01/10 10:36:49 DEBUG executeCommand(): Executing ssh-agent bash -c 'ssh-add /opt/puppetlabs/server/data/puppetserver/.ssh/id_rsa; git clone --mirror git@xxxxxxxxxxxxxx /var/cache/r10k/environments/***.git'
2019/01/10 10:36:49 DEBUG executeCommand(): Executing ssh-agent bash -c 'ssh-add /opt/puppetlabs/server/data/puppetserver/.ssh/id_rsa; git --git-dir /var/cache/r10k/environments/***.git remote update --prune'
2019/01/10 10:36:49 DEBUG checkDirAndCreate(): Using as basedir for source ***: /etc/puppetlabs/code/environments/
2019/01/10 10:36:49 DEBUG Puppet environment: *** ({Remote:git@xxxxxxxxxxxxxxxxxxxxx Basedir:/etc/puppetlabs/code/environments/ Prefix:true PrivateKey:/opt/puppetlabs/server/data/puppetserver/.ssh/id_rsa ForceForgeVersions:false WarnMissingBranch:true ExitIfUnreachable:false AutoCorrectEnvironmentNames:correct_and_warn})
2019/01/10 10:36:49 DEBUG executeCommand(): Executing ssh-agent bash -c 'ssh-add /opt/puppetlabs/server/data/puppetserver/.ssh/id_rsa; git --git-dir /var/cache/r10k/environments/***.git remote update --prune'
2019/01/10 10:36:49 DEBUG Puppet environment: *** ({Remote:git@xxxxxxxxxxxxxxxxxx Basedir:/etc/puppetlabs/code/environments/ Prefix:true PrivateKey:/opt/puppetlabs/server/data/puppetserver/.ssh/id_rsa ForceForgeVersions:false WarnMissingBranch:true ExitIfUnreachable:false AutoCorrectEnvironmentNames:correct_and_warn})
2019/01/10 10:36:49 DEBUG checkDirAndCreate(): Using as basedir: /etc/puppetlabs/code/environments/
2019/01/10 10:36:49 DEBUG checkDirAndCreate(): Using as basedir: /etc/puppetlabs/code/environments/
2019/01/10 10:36:49 DEBUG checkDirAndCreate(): Using as basedir for source ***: /etc/puppetlabs/code/environments/
2019/01/10 10:36:49 DEBUG executeCommand(): Executing ssh-agent bash -c 'ssh-add /opt/puppetlabs/server/data/puppetserver/.ssh/id_rsa; git clone --mirror git@xxxxxxxxxxxxxxxxxx /var/cache/r10k/environments/***.git'
2019/01/10 10:36:49 DEBUG executeCommand(): Executing ssh-agent bash -c 'ssh-add /opt/puppetlabs/server/data/puppetserver/.ssh/id_rsa; git clone --mirror git@xxxxxxxxxxxxxxxxxx /var/cache/r10k/environments/***.git'
2019/01/10 10:36:49 DEBUG Puppet environment: *** ({Remote:git@xxxxxxxxxxxxx Basedir:/etc/puppetlabs/code/environments/ Prefix:true PrivateKey:/opt/puppetlabs/server/data/puppetserver/.ssh/id_rsa ForceForgeVersions:false WarnMissingBranch:true ExitIfUnreachable:false AutoCorrectEnvironmentNames:correct_and_warn})
2019/01/10 10:36:49 DEBUG checkDirAndCreate(): Using as basedir: /etc/puppetlabs/code/environments/
2019/01/10 10:36:49 DEBUG executeCommand(): Executing ssh-agent bash -c 'ssh-add /opt/puppetlabs/server/data/puppetserver/.ssh/id_rsa; git clone --mirror git@xxxxxxxxxxxxxxx /var/cache/r10k/environments/***.git'
2019/01/10 10:36:49 DEBUG executeCommand(): Executing ssh-agent bash -c 'ssh-add /opt/puppetlabs/server/data/puppetserver/.ssh/id_rsa; git clone --mirror git@xxxxxxxxxxxxxx /var/cache/r10k/environments/***.git' took 0.06063s
WARN: git repository git@xxxxxxxxxxxxxxx does not exist or is unreachable at this moment!
WARN: Trying to use cache for git@xxxxxxxxxxxxx git repository
WARN: git repository git@xxxxxxxxxxxxxxxx does not exist or is unreachable at this moment!
WARNING: Could not resolve git repository in source '***' (git@xxxxxxxxxxxxx:***/***-***.git)
2019/01/10 10:36:50 Executing ssh-agent bash -c 'ssh-add /opt/puppetlabs/server/data/puppetserver/.ssh/id_rsa; git --git-dir /var/cache/r10k/environments/***.git remote update --prune' took 0.53374s
2019/01/10 10:36:50 DEBUG executeCommand(): Executing git --git-dir /var/cache/r10k/environments/***.git branch
2019/01/10 10:36:50 Executing git --git-dir /var/cache/r10k/environments/***.git branch took 0.00239s
2019/01/10 10:36:50 DEBUG Skipping branch master
2019/01/10 10:36:50 DEBUG 1(): Resolving branch: production
Renaming branch production to production
2019/01/10 10:36:50 DEBUG executeCommand(): Executing git --git-dir /var/cache/r10k/environments/***.git rev-parse --verify 'production^{object}'
2019/01/10 10:36:50 DEBUG executeCommand(): Executing ssh-agent bash -c 'ssh-add /opt/puppetlabs/server/data/puppetserver/.ssh/id_rsa; git clone --mirror git@xxxxxxxxxxxxx:***/***-***.git /var/cache/r10k/environments/***.git' took 0.53748s
WARN: git repository git@xxxxxxxxxxxxx:***/***-***.git does not exist or is unreachable at this moment!
WARN: Trying to use cache for git@xxxxxxxxxxxxx:***/***-***.git git repository
WARN: git repository git@xxxxxxxxxxxxx:***/***-***.git does not exist or is unreachable at this moment!
WARNING: Could not resolve git repository in source '***' (git@xxxxxxxxxxxxx:***/***-***.git)
2019/01/10 10:36:50 Executing git --git-dir /var/cache/r10k/environments/***.git rev-parse --verify 'production^{object}' took 0.00330s
2019/01/10 10:36:50 DEBUG readPuppetfile(): Trying to parse: /etc/puppetlabs/code/environments/***_production/Puppetfile
2019/01/10 10:36:50 DEBUG preparePuppetfile(): adding line:mod '***prj',
2019/01/10 10:36:50 DEBUG preparePuppetfile(): adding line::git    => 'git@xxxxxxxxxxxxx:***/***-***.git',
2019/01/10 10:36:50 DEBUG preparePuppetfile(): adding line::branch => 'production'
2019/01/10 10:36:50 DEBUG readPuppetfile(): Setting :ignore_unreachable for Git module ***prj
2019/01/10 10:36:50 DEBUG executeCommand(): Executing ssh-agent bash -c 'ssh-add /opt/puppetlabs/server/data/puppetserver/.ssh/id_rsa; git clone --mirror git@xxxxxxxxxxxxx:***/***-***.git /var/cache/r10k/environments/***.git' took 0.55134s
WARN: git repository git@xxxxxxxxxxxxx:***/***-***.git does not exist or is unreachable at this moment!
WARN: Trying to use cache for git@xxxxxxxxxxxxx:***/***-***.git git repository
WARN: git repository git@xxxxxxxxxxxxx:***/***-***.git does not exist or is unreachable at this moment!
WARNING: Could not resolve git repository in source '***' (git@xxxxxxxxxxxxx:***/***-***.git)
2019/01/10 10:36:50 DEBUG executeCommand(): Executing ssh-agent bash -c 'ssh-add /opt/puppetlabs/server/data/puppetserver/.ssh/id_rsa; git clone --mirror git@xxxxxxxxxxxxx:***/***-***.git /var/cache/r10k/environments/***.git' took 0.57991s
WARN: git repository git@xxxxxxxxxxxxx:***/***-***.git does not exist or is unreachable at this moment!
WARN: Trying to use cache for git@xxxxxxxxxxxxx:***/***-***.git git repository
WARN: git repository git@xxxxxxxxxxxxx:***/***-***.git does not exist or is unreachable at this moment!
WARNING: Could not resolve git repository in source '***' (git@xxxxxxxxxxxxx:***/***-***.git)
2019/01/10 10:36:50 Executing ssh-agent bash -c 'ssh-add /opt/puppetlabs/server/data/puppetserver/.ssh/id_rsa; git --git-dir /var/cache/r10k/environments/***.git remote update --prune' took 0.58078s
2019/01/10 10:36:50 DEBUG executeCommand(): Executing git --git-dir /var/cache/r10k/environments/***.git branch
2019/01/10 10:36:50 Executing git --git-dir /var/cache/r10k/environments/***.git branch took 0.00155s
2019/01/10 10:36:50 DEBUG Skipping branch master
2019/01/10 10:36:50 DEBUG 1(): Resolving branch: production
Renaming branch production to production
2019/01/10 10:36:50 DEBUG executeCommand(): Executing git --git-dir /var/cache/r10k/environments/***.git rev-parse --verify 'production^{object}'
2019/01/10 10:36:50 Executing git --git-dir /var/cache/r10k/environments/***.git rev-parse --verify 'production^{object}' took 0.00381s
2019/01/10 10:36:50 DEBUG readPuppetfile(): Trying to parse: /etc/puppetlabs/code/environments/***_production/Puppetfile
2019/01/10 10:36:50 DEBUG preparePuppetfile(): adding line:mod 'puppetlabs-stdlib', '4.25.1'
2019/01/10 10:36:50 DEBUG preparePuppetfile(): adding line:mod 'puppetlabs-concat', '4.2.1'
2019/01/10 10:36:50 DEBUG preparePuppetfile(): adding line:mod 'puppetlabs-firewall', '1.12.0'
2019/01/10 10:36:50 DEBUG preparePuppetfile(): adding line:mod 'puppetlabs-apt', '6.0.0'
2019/01/10 10:36:50 DEBUG preparePuppetfile(): adding line:mod 'puppet-nginx', '0.13.0'
2019/01/10 10:36:50 DEBUG preparePuppetfile(): adding line:mod 'hfm-tinyproxy', '1.2.1'
2019/01/10 10:36:50 DEBUG preparePuppetfile(): adding line:mod 'puppetprj',
2019/01/10 10:36:50 DEBUG preparePuppetfile(): adding line::git    => 'git@xxxxxxxxxxxxx:***/***-***.git',
2019/01/10 10:36:50 DEBUG preparePuppetfile(): adding line::branch => 'production'
2019/01/10 10:36:50 DEBUG readPuppetfile(): setting forge module puppetlabs/stdlib to version 4.25.1
2019/01/10 10:36:50 DEBUG readPuppetfile(): setting forge module puppetlabs/concat to version 4.2.1
2019/01/10 10:36:50 DEBUG readPuppetfile(): setting forge module puppetlabs/firewall to version 1.12.0
2019/01/10 10:36:50 DEBUG readPuppetfile(): setting forge module puppetlabs/apt to version 6.0.0
2019/01/10 10:36:50 DEBUG readPuppetfile(): setting forge module puppet/nginx to version 0.13.0
2019/01/10 10:36:50 DEBUG readPuppetfile(): setting forge module hfm/tinyproxy to version 1.2.1
2019/01/10 10:36:50 DEBUG readPuppetfile(): Setting :ignore_unreachable for Git module ***prj
2019/01/10 10:36:50 DEBUG resolvePuppetfile(): Resolving ***_production
2019/01/10 10:36:50 DEBUG resolvePuppetfile(): Resolving ***_production
2019/01/10 10:36:50 DEBUG resolveGitRepositories(): Resolving 2 Git modules with 50 workers
2019/01/10 10:36:50 DEBUG resolveForgeModules(): Resolving 6 Forge modules with 50 workers
2019/01/10 10:36:50 DEBUG resolveGitRepositories(): git repo url git@***.git
2019/01/10 10:36:50 DEBUG resolveGitRepositories(): git repo url git@***.git
2019/01/10 10:36:50 DEBUG resolveForgeModules(): Trying to get forge module puppetlabs/stdlib-4.25.1 with Forge base url  and CacheTtl set to 0s
2019/01/10 10:36:50 DEBUG git repo url git@xxxxxxxxxxxxx:***/***-***.git with ssh key /opt/puppetlabs/server/data/puppetserver/.ssh/id_rsa
2019/01/10 10:36:50 DEBUG doModuleInstallOrNothing(): Using cache for puppetlabs-stdlib in version 4.25.1 because /var/cache/r10k/forge/puppetlabs-stdlib-4.25.1 exists
2019/01/10 10:36:50 DEBUG executeCommand(): Executing ssh-agent bash -c 'ssh-add /opt/puppetlabs/server/data/puppetserver/.ssh/id_rsa; git --git-dir /var/cache/r10k/modules/git@xxxxxxxxxxxxx-***_***-***.git remote update --prune'
2019/01/10 10:36:50 DEBUG git repo url git@xxxxxxxxxxxxx:***/***-***.git with ssh key /opt/puppetlabs/server/data/puppetserver/.ssh/id_rsa
2019/01/10 10:36:50 DEBUG resolveForgeModules(): Trying to get forge module puppetlabs/concat-4.2.1 with Forge base url  and CacheTtl set to 0s
2019/01/10 10:36:50 DEBUG executeCommand(): Executing ssh-agent bash -c 'ssh-add /opt/puppetlabs/server/data/puppetserver/.ssh/id_rsa; git --git-dir /var/cache/r10k/modules/git@xxxxxxxxxxxxx-***_***-***.git remote update --prune'
2019/01/10 10:36:50 DEBUG doModuleInstallOrNothing(): Using cache for puppetlabs-concat in version 4.2.1 because /var/cache/r10k/forge/puppetlabs-concat-4.2.1 exists
2019/01/10 10:36:50 DEBUG resolveForgeModules(): Trying to get forge module puppetlabs/firewall-1.12.0 with Forge base url  and CacheTtl set to 0s
2019/01/10 10:36:50 DEBUG doModuleInstallOrNothing(): Using cache for puppetlabs-firewall in version 1.12.0 because /var/cache/r10k/forge/puppetlabs-firewall-1.12.0 exists
2019/01/10 10:36:50 DEBUG resolveForgeModules(): Trying to get forge module puppetlabs/apt-6.0.0 with Forge base url  and CacheTtl set to 0s
2019/01/10 10:36:50 DEBUG resolveForgeModules(): Trying to get forge module puppet/nginx-0.13.0 with Forge base url  and CacheTtl set to 0s
2019/01/10 10:36:50 DEBUG doModuleInstallOrNothing(): Using cache for puppetlabs-apt in version 6.0.0 because /var/cache/r10k/forge/puppetlabs-apt-6.0.0 exists
2019/01/10 10:36:50 DEBUG doModuleInstallOrNothing(): Using cache for puppet-nginx in version 0.13.0 because /var/cache/r10k/forge/puppet-nginx-0.13.0 exists
2019/01/10 10:36:50 DEBUG resolveForgeModules(): Trying to get forge module hfm/tinyproxy-1.2.1 with Forge base url  and CacheTtl set to 0s
2019/01/10 10:36:50 DEBUG doModuleInstallOrNothing(): Using cache for hfm-tinyproxy in version 1.2.1 because /var/cache/r10k/forge/hfm-tinyproxy-1.2.1 exists
2019/01/10 10:36:50 DEBUG timeTrack(): resolveForgeModules() took 0.00566s
2019/01/10 10:36:50 Executing ssh-agent bash -c 'ssh-add /opt/puppetlabs/server/data/puppetserver/.ssh/id_rsa; git --git-dir /var/cache/r10k/modules/git@xxxxxxxxxxxxx-puppet_***-***.git remote update --prune' took 0.52207s
2019/01/10 10:36:50 Executing ssh-agent bash -c 'ssh-add /opt/puppetlabs/server/data/puppetserver/.ssh/id_rsa; git --git-dir /var/cache/r10k/modules/git@xxxxxxxxxxxxx-***_***-***.git remote update --prune' took 0.52191s
2019/01/10 10:36:50 DEBUG timeTrack(): resolveGitRepositories() took 0.52623s
2019/01/10 10:36:50 DEBUG resolvePuppetfile(): Syncing ***_production with workDir /etc/puppetlabs/code/environments/***_production/
2019/01/10 10:36:50 DEBUG checkDirAndCreate(): Using as basedir 2 for source ***: /etc/puppetlabs/code/environments/***_production/
2019/01/10 10:36:50 DEBUG resolvePuppetfile(): Syncing ****_production with workDir /etc/puppetlabs/code/environments/****_production/
2019/01/10 10:36:50 DEBUG checkDirAndCreate(): Using as basedir 2 for source ***: /etc/puppetlabs/code/environments/****_production/
==================
WARNING: DATA RACE
Read at 0x00c0002581b0 by goroutine 64:
  main.resolvePuppetfile.func3()
      /home/andpaul/dev/go/src/github.com/xorpaul/g10k/puppetfile.go:330 +0x7fd

Previous write at 0x00c0002581b0 by main goroutine:
  main.resolvePuppetfile()
      /home/andpaul/dev/go/src/github.com/xorpaul/g10k/puppetfile.go:228 +0x1a35
  main.resolvePuppetEnvironment()
      /home/andpaul/dev/go/src/github.com/xorpaul/g10k/puppetfile.go:148 +0x3db
  main.main()
      /home/andpaul/dev/go/src/github.com/xorpaul/g10k/g10k.go:232 +0x9f4

Goroutine 64 (running) created at:
  main.resolvePuppetfile()
      /home/andpaul/dev/go/src/github.com/xorpaul/g10k/puppetfile.go:276 +0x174f
  main.resolvePuppetEnvironment()
      /home/andpaul/dev/go/src/github.com/xorpaul/g10k/puppetfile.go:148 +0x3db
  main.main()
      /home/andpaul/dev/go/src/github.com/xorpaul/g10k/g10k.go:232 +0x9f4
==================
2019/01/10 10:36:50 DEBUG executeCommand(): Executing git --git-dir /var/cache/r10k/modules/git@xxxxxxxxxxxxx-***_***-***.git rev-parse --verify 'production^{object}'
2019/01/10 10:36:50 DEBUG executeCommand(): Executing git --git-dir /var/cache/r10k/modules/git@xxxxxxxxxxxxx-***_***-***.git rev-parse --verify 'production^{object}'
2019/01/10 10:36:50 DEBUG readModuleMetadata(): Found in file /etc/puppetlabs/code/environments/****_production/modules/firewall/metadata.json name: puppetlabs-firewall version: 1.12.0 author: Puppet Labs
2019/01/10 10:36:50 DEBUG syncForgeToModuleDir(): Nothing to do, existing Forge module: /etc/puppetlabs/code/environments/***_production/modules/firewall/ has the same version 1.12.0 as the to be synced version: 1.12.0
2019/01/10 10:36:50 DEBUG readModuleMetadata(): Found in file /etc/puppetlabs/code/environments/***_production/modules/apt/metadata.json name: puppetlabs-apt version: 6.2.1 author: puppetlabs
Need to sync, because existing Forge module: /etc/puppetlabs/code/environments/***_production/modules/apt/ has version 6.2.1 and the to be synced version is: 6.0.0
2019/01/10 10:36:50 DEBUG readModuleMetadata(): Found in file /etc/puppetlabs/code/environments/***_production/modules/nginx/metadata.json name: puppet-nginx version: 0.13.0 author: Vox Pupuli
2019/01/10 10:36:50 DEBUG syncForgeToModuleDir(): Nothing to do, existing Forge module: /etc/puppetlabs/code/environments/***_production/modules/nginx/ has the same version 0.13.0 as the to be synced version: 0.13.0
2019/01/10 10:36:50 DEBUG createOrPurgeDir(): Trying to remove: /etc/puppetlabs/code/environments/***_production/modules/apt/ called from  targetDir for module apt
2019/01/10 10:36:50 DEBUG readModuleMetadata(): Found in file /etc/puppetlabs/code/environments/***_production/modules/tinyproxy/metadata.json name: hfm-tinyproxy version: 1.2.1 author: hfm
2019/01/10 10:36:50 DEBUG syncForgeToModuleDir(): Nothing to do, existing Forge module: /etc/puppetlabs/code/environments/***_production/modules/tinyproxy/ has the same version 1.2.1 as the to be synced version: 1.2.1
2019/01/10 10:36:50 DEBUG readModuleMetadata(): Found in file /etc/puppetlabs/code/environments/***_production/modules/stdlib/metadata.json name: puppetlabs-stdlib version: 5.1.0 author: puppetlabs
2019/01/10 10:36:50 DEBUG readModuleMetadata(): Found in file /etc/puppetlabs/code/environments/***_production/modules/concat/metadata.json name: puppetlabs-concat version: 5.1.0 author: puppetlabs
Need to sync, because existing Forge module: /etc/puppetlabs/code/environments/***_production/modules/stdlib/ has version 5.1.0 and the to be synced version is: 4.25.1
Need to sync, because existing Forge module: /etc/puppetlabs/code/environments/***_production/modules/concat/ has version 5.1.0 and the to be synced version is: 4.2.1
2019/01/10 10:36:50 DEBUG createOrPurgeDir(): Trying to remove: /etc/puppetlabs/code/environments/***_production/modules/stdlib/ called from  targetDir for module stdlib
2019/01/10 10:36:50 DEBUG createOrPurgeDir(): Trying to remove: /etc/puppetlabs/code/environments/***_production/modules/concat/ called from  targetDir for module concat
2019/01/10 10:36:50 Executing git --git-dir /var/cache/r10k/modules/git@xxxxxxxxxxxxx-***_***-***.git rev-parse --verify 'production^{object}' took 0.00394s
2019/01/10 10:36:50 DEBUG not removing moduleDirectory /etc/puppetlabs/code/environments/***_production/modules/***prj/ because it's a subdirectory to existingDir /etc/puppetlabs/code/environments/***_production/modules/***prj
2019/01/10 10:36:50 Executing git --git-dir /var/cache/r10k/modules/git@xxxxxxxxxxxxx-***_***-***.git rev-parse --verify 'production^{object}' took 0.00295s
2019/01/10 10:36:50 DEBUG not removing moduleDirectory /etc/puppetlabs/code/environments/***_production/modules/***prj/ because it's a subdirectory to existingDir /etc/puppetlabs/code/environments/***_production/modules/***prj
2019/01/10 10:36:50 DEBUG createOrPurgeDir(): Trying to create dir: /etc/puppetlabs/code/environments/***_production/modules/concat/ called from  targetDir for module concat
Need to sync /etc/puppetlabs/code/environments/***_production/modules/concat/
2019/01/10 10:36:50 DEBUG checkDirAndCreate(): Using as as targetDir for module concat: /etc/puppetlabs/code/environments/***_production/modules/concat/
2019/01/10 10:36:50 DEBUG syncForgeToModuleDir(): syncForgeToModuleDir() filepath.Walk'ing directory /var/cache/r10k/forge/puppetlabs-concat-4.2.1/
2019/01/10 10:36:50 DEBUG createOrPurgeDir(): Trying to create dir: /etc/puppetlabs/code/environments/***_production/modules/apt/ called from  targetDir for module apt
Need to sync /etc/puppetlabs/code/environments/***_production/modules/apt/
2019/01/10 10:36:50 DEBUG checkDirAndCreate(): Using as as targetDir for module apt: /etc/puppetlabs/code/environments/***_production/modules/apt/
2019/01/10 10:36:50 DEBUG syncForgeToModuleDir(): syncForgeToModuleDir() filepath.Walk'ing directory /var/cache/r10k/forge/puppetlabs-apt-6.0.0/
2019/01/10 10:36:50 Populating /etc/puppetlabs/code/environments/***_production/modules/concat/ took 0.00785s
2019/01/10 10:36:50 Populating /etc/puppetlabs/code/environments/***_production/modules/apt/ took 0.00899s
2019/01/10 10:36:50 DEBUG createOrPurgeDir(): Trying to create dir: /etc/puppetlabs/code/environments/***_production/modules/stdlib/ called from  targetDir for module stdlib
Need to sync /etc/puppetlabs/code/environments/***_production/modules/stdlib/
2019/01/10 10:36:50 DEBUG checkDirAndCreate(): Using as as targetDir for module stdlib: /etc/puppetlabs/code/environments/***_production/modules/stdlib/
2019/01/10 10:36:50 DEBUG syncForgeToModuleDir(): syncForgeToModuleDir() filepath.Walk'ing directory /var/cache/r10k/forge/puppetlabs-stdlib-4.25.1/
2019/01/10 10:36:50 Populating /etc/puppetlabs/code/environments/***_production/modules/stdlib/ took 0.04278s
2019/01/10 10:36:50 DEBUG resolvePuppetfile(): Removing unmanaged file /etc/puppetlabs/code/environments/***_production/modules/resolv_conf
2019/01/10 10:36:50 DEBUG resolvePuppetfile(): Removing unmanaged file /etc/puppetlabs/code/environments/***_production/modules/ssh
2019/01/10 10:36:50 DEBUG resolvePuppetfile(): Removing unmanaged file /etc/puppetlabs/code/environments/***_production/modules/ssh
2019/01/10 10:36:50 DEBUG resolvePuppetfile(): Removing unmanaged file /etc/puppetlabs/code/environments/***_production/modules/ntp
2019/01/10 10:36:50 DEBUG resolvePuppetfile(): Removing unmanaged file /etc/puppetlabs/code/environments/***_production/modules/resolv_conf
2019/01/10 10:36:50 DEBUG resolvePuppetfile(): Removing unmanaged file /etc/puppetlabs/code/environments/***_production/modules/inifile
2019/01/10 10:36:50 DEBUG resolvePuppetfile(): Removing unmanaged file /etc/puppetlabs/code/environments/***_production/modules/apt
2019/01/10 10:36:50 DEBUG resolvePuppetfile(): Removing unmanaged file /etc/puppetlabs/code/environments/***_production/modules/inifile
2019/01/10 10:36:50 DEBUG resolvePuppetfile(): Removing unmanaged file /etc/puppetlabs/code/environments/***_production/modules/chronyd
2019/01/10 10:36:50 DEBUG resolvePuppetfile(): Removing unmanaged file /etc/puppetlabs/code/environments/***_production/modules/lwbase
2019/01/10 10:36:50 DEBUG resolvePuppetfile(): Removing unmanaged file /etc/puppetlabs/code/environments/***_production/modules/translate
2019/01/10 10:36:50 DEBUG resolvePuppetfile(): Removing unmanaged file /etc/puppetlabs/code/environments/***_production/modules/chronyd
2019/01/10 10:36:50 DEBUG resolvePuppetfile(): Removing unmanaged file /etc/puppetlabs/code/environments/***_production/modules/lwbase
2019/01/10 10:36:50 DEBUG resolvePuppetfile(): Removing unmanaged file /etc/puppetlabs/code/environments/***_production/modules/stdlib
2019/01/10 10:36:50 DEBUG resolvePuppetfile(): Removing unmanaged file /etc/puppetlabs/code/environments/***_production/modules/translate
2019/01/10 10:36:50 DEBUG resolvePuppetfile(): Removing unmanaged file /etc/puppetlabs/code/environments/***_production/modules/ntp
2019/01/10 10:36:50 DEBUG resolvePuppetfile(): Removing unmanaged file /etc/puppetlabs/code/environments/***_production/modules/concat
2019/01/10 10:36:50 DEBUG main(): Forge response JSON parsing took 0.0000 seconds
2019/01/10 10:36:50 DEBUG main(): Forge modules metadata.json parsing took 0.0002 seconds
Synced /etc/puppetlabs/r10k/r10k.yaml with branch production with 4 git repositories and 6 Forge modules in 1.3s with git (0.5s sync, I/O 0.0s) and Forge (0.0s query+download, I/O 0.1s) using 50 resolv and 20 extract workers
2019/01/10 10:36:50 DEBUG executeCommand(): Executing /usr/bin/*****
2019/01/10 10:36:52 Executing /usr/bin/**** took 1.20547s
2019/01/10 10:36:52 DEBUG checkForAndExecutePostrunCommand(): postrun command '****' terminated with exit code 0
Found 1 data race(s)
xorpaul commented 5 years ago

Please try out v0.5.7 https://github.com/xorpaul/g10k/releases/tag/v0.5.7

I think I've found the cause of the data race. (https://github.com/xorpaul/g10k/commit/297a67a0c64b083ccab6740748c5220cec995cac)

deferraz commented 5 years ago

I had no data race with v0.5.7 @xorpaul , thanks!

@sfdc-afraley you can test with the same release? on my puppet instance, the problem has been solved

xorpaul commented 5 years ago

I'm closing this as it doesn't seem to happen anymore.

Otherwise please reopen.