search

coderanger / kitchen-sync

Test Kitchen transport plugin to speed up tests.
Apache License 2.0
81 stars 24 forks source link

Kitchen-sync broken with test-kitchen >= 1.4 #8

Closed pelletiermaxime closed 9 years ago

pelletiermaxime commented 9 years ago

Hi, first of all, I really like this gem because of all the time it saves me when using test-kitchen, so thank you. However, it seems all the changes in the transports in test-kitchen 1.4 broke it. Is it possible to fix it or are the changes too big ?

iiro commented 9 years ago

Noticed the same... :/ Pretty much prevents using Test-Kitchen - the upload is so terrible slow :/

coderanger commented 9 years ago

This should work again with TK 1.4, just make sure you add:

transport:
  name: sftp

Still working on some fixes upstream before I do a big announce.

pelletiermaxime commented 9 years ago

Thanks !

iiro commented 9 years ago

@coderanger excellent work dude! :)

@pelletiermaxime did you get faster sync times?

I configured the sftp transport to be used - but still:

   [SFTP] Time taken to upload /var/folders/hv/xqb2mh7n13d_gzbrvpfcjr300000gp/T/xxxxx-example-frontend-xxxxxxx-ubuntu-1404-sandbox-20150530-49271-1bosxax/cookbooks to username@ec2-ip-address.eu-west-1.compute.amazonaws.com<{:user_known_hosts_file=>"/dev/null", :paranoid=>false, :port=>22, :compression=>"zlib", :compression_level=>6, :keepalive=>true, :keepalive_interval=>60, :timeout=>10, :keys_only=>true, :keys=>["/Users/username.ssh/keyfile"], :user=>"ubuntu"}>:/tmp/kitchen/cookbooks: 292.76 sec

   Chef Client finished, 30/93 resources updated in 38.66922325 seconds
   Finished converging <xxxxxx-example-frontend-xxxxxx-ubuntu-1404> (7m57.79s).

It's crazy slow (this is the first run).

The second run is normal (around 10 seconds).

coderanger commented 9 years ago

The first transfer is only marginally faster. The benefits of the SFTP transport are not re-transfering files that haven't changed. It should at least use more parallel transfers for the initial dump, but that's rarely the bounding factor. You can try forking the repo and upping the max transfers. Now that transport plugins are a real thing I would like to make that an option, but all I have so far is a quick-n-dirty port of the old code hence not advertising it yet.

iiro commented 9 years ago

Ah ok.

Any ideas or plans to support the rsync transport some time...? It has been blazing fast (no surprise of course).

coderanger commented 9 years ago

After some upstream patches get merged I'll port that over too.

iiro commented 9 years ago

Ok, excellent. Sorry for the pressure. You are awesome :)

iiro commented 8 years ago

@coderanger any updates? You have still some upstream patches waiting to be merged...? Just asking because being stuck in ChefDK 0.4.0 still... :)