A first try at using Quicksilver

[MidGe48]

This has the makings of a good and useful project. Thank you for sharing.

I cloned the git repo on my machine and have started trying to get it working. I experienced the following:

1. Not using "Digital Ocean" I had to comment out the import and the coda for it in the Vagrantfile as it was complaining about the missing plugin.
2. Does every passwords (webmaster, users, db, sites) need to be hashed? If so is the very simple 'echo "password" | sha512sum' sufficient? Using the recommended method in the readme.md actually changes the shadow file on the host, and changes the hashed value each time it is run.
3. I am unable to complete the job as I get "ssh: connect to host 127.0.0.1 port 22: Connection refused Couldn't read packet: Connection reset by peer" on the first task (install python-software-properties) in Ansible. Has that something to do with the VM being connected thru NAT, perhaps?. I can connect without problem thru ssh on the command line, so the issue is somewhere in the config, I think.

I could not go any further despite my dozens of attempts. Any clues about what the problem might be?

Thanks again for sharing.

[rdickert]

Thanks for trying it out! I think you are very close to getting it to work - sorry about the limitations in the documentation. Some answers:

1. No Digital Ocean: This is the correct thing to do.

2. Passwords:

   The linux user account passwords (webmaster, developer, user) must be hashed. Here are directions that worked for me. This method works on linux and produces a usable hash (i.e., I can log in with the password I chose on the vm). Let me know if you are able to get it working or need better detail. Unfortunately, my first attempts to use these packages on a mac did not work out - if you find a way that works, let me know!

   The db passwords (root and site) and the smtp password are plaintext.

3. SSH failure: I'm assuming that you are using VirtualBox here. Vagrant maps ssh to port 2222 on the vm. It appears your script is trying to reach port 22 (the standard ssh port) instead. If so, it is actually your local machine, not the vm, that is declining to accept SSH on port 22. If Vagrant is not able to connect, check your settings in the ansible_hosts file and make sure it says

   [vagrant]
   127.0.0.1:2222

   That should point Ansible to the right place. If you ever choose to ssh in without using vagrant ssh, you'll need to use ssh -p 2222 myaccount@localhost.

Let me know how these work for you. On my machine, the full script is taking something like 13-15 minutes for a distro install (it will take longer if you use a large archive). There may be ways to cut this down by a decent amount in the future. I will plan to update the instructions based on your feedback. Thanks again for trying the project!

[rdickert]

Here's a preview of the password documentation - a little more detail than I left before:

A Note on passwords and keyfiles

The Linux passwords (webmaster, developer, user) in the config file must be hashed, not plaintext. The following directions work for generating a password hash in Ubuntu and will likely work for any Debian system (source).

• Run sudo apt-get install whois. This will add the mkpasswd command.
• Run mkpasswd -m sha-512 MyPAsSwOrD $(openssl rand -base64| - d '+=' | head -c 16) (substitute your desired password for MyPAsSwOrD above)
• Copy and paste the output into the password field for the user. (Instructions for OSX and Windows would be helpful - please contribute!)

Leaving the password blank does not seem to allow a non-password protected account to be logged-in via SSH or su, but I don't know for a fact at the accounts are safe. Please report security issues! ssh_key_file should be a path to a .pub file and will allow users log in via ssh without a password. This may actually be easier than passwords, and its more secure and nice to use.

The db and email passwords should be in plaintext.

[MidGe48]

Thank you for this.

I have it all working now.

An issue was that I needed to stop the ssh demon on my host as it was hijacking the login attempts (since the VM is on a NAT connection).

Also, mkpasswd modifies the shadow file on the host (not desirable, imo) and may have had issues with my passwords containing some special characters ['#' and '$' in my case] according to mkpasswd docs. So, may I suggest that rather than using

 "Run |mkpasswd -m sha-512 MyPAsSwOrD $(openssl rand -base64| - d 

'+=' | head -c 16)| (substitute your desired password for MyPAsSwOrD above)",

you use small python one-liner:

 "python -c 'import crypt; print crypt.crypt("MyPAsSwOrD", 

"$6$a_random_salt")' "

which return a hash suitable for shadow without altering the host one and does not seem to have a problem with special characters. Needless to say that it did work for me.

Thanks again for your sharing and your help.

If I can contribute in anyway, let me know. I am very willing, and have the time, to try out any release candidate, for instance.

The issue is with the Ansible part of Quicksilver. The Vagrant part works fine, the VM in VirtualBox gets created and is running. On 12/06/13 05:36, rdickert wrote:

Thanks for trying it out! I think you are very close to getting it to work - sorry about the limitations in the documentation. Some answers:

1.

No Digital Ocean: This is the correct thing to do.

2.

Passwords:

The linux user account passwords (webmaster, developer, user) must
be hashed. Here are directions that worked for me
<http://serverfault.com/questions/330069/how-to-create-an-sha-512-hashed-password-for-shadow/479036#479036>.
This method works on linux and produces a usable hash (i.e., I can
log in with the password I chose on the vm). Let me know if you
are able to get it working or need better detail. Unfortunately,
my first attempts to use these packages on a mac did not work out
- if you find a way that works, let me know!

The db passwords (root and site) and the smtp password are plaintext.

3.

SSH failure: I'm assuming that you are using VirtualBox here.
Vagrant maps ssh to port 2222 on the vm. It appears your script is
trying to reach port 22 (the standard ssh port) instead. If so, it
is actually your local machine, not the vm, that is declining to
accept SSH on port 22. If Vagrant is not able to connect, check
your settings in the |ansible_hosts| file and make sure it says

|[vagrant]
127.0.0.1:2222
|

That should point Ansible to the right place. If you ever choose
to ssh in without using |vagrant ssh|, you'll need to use |ssh -p
2222 myaccount@localhost|.

Let me know how these work for you. On my machine, the full script is taking something like 13-15 minutes for a distro install (it will take longer if you use a large archive). There may be ways to cut this down by a decent amount in the future. I will plan to update the instructions based on your feedback. Thanks again for trying the project!

— Reply to this email directly or view it on GitHub https://github.com/rdickert/project-quicksilver/issues/1#issuecomment-19294062.

[rdickert]

Very cool. I'll try out your one-liner - it looks better than what I have and should be cross-platform.

Your help would be very welcome. Certainly, testing new releases would be a great help. Perhaps there should be some kind of testing protocol - testing Ansible is not as straightforward as tools higher in the stack. Also, if there is any area of the project where you think you have something to add, let me know. Some candidates:

• Tuning - new variables, better defaults. There are currently no database tuning variables. Should we add key buffer size, memory settings, etc.?
• Ansible script improvements (not all are idempotent, and some speed improvements are possible)
• Additional modules or enhancements: Varnish would be a candidate, although I didn't find it necessary in my situation. Redis could be available if the user wanted to switch from Memcached. Would small sites ever use Solr? If the project is missing something you want to use, there are likely other people who want it too (but let's not add things "just in case").

Let me know if want feedback on any ideas, or just submit a pull request.

[rdickert]

OK, so I looked at the Python one-liner. I take it you are supposed to substitute an actual salt for a_random_salt? If so, this is missing the salt-generation, which I think we need to have. Also, this script is surprisingly not cross-platform as I hoped. Based on this Stack Exchange q/a,, here's another option that may work: python -c 'from passlib.hash import sha512_crypt;print sha512_crypt.encrypt("t3$T1")' where "t3$T1" is your plaintext password (here I'm trying to test some various characters). It seems to insert an extra "$rounds=60000", but there appears to be a salt after that (here's the documentation from passlib that confirms this), and the server accepts the example I gave (haven't tested further). It works on both Ubuntu and Mac OS (I had to run sudo easy_install passlib on the Mac). What do you think?

[MidGe48]

OK, I am now trying the DigitalOcean option and I came across the following problem.

/opt/vagrant/embedded/gems/gems/vagrant-1.2.2/lib/vagrant/batch_action.rb:63:in `block (2 levels) in run' INFO interface: error: The secure connection to the Digital Ocean API has failed. Please ensure that your local certificates directory is defined in the provider config.

config.vm.provider :digital_ocean do |vm|
  vm.ca_path = "/path/to/ssl/ca/cert.crt"
end

This is generally caused by the OpenSSL configuration associated with the Ruby install being unaware of the system specific ca certs. The secure connection to the Digital Ocean API has failed. Please ensure that your local certificates directory is defined in the provider config.

config.vm.provider :digital_ocean do |vm|
  vm.ca_path = "/path/to/ssl/ca/cert.crt"
end

This is generally caused by the OpenSSL configuration associated with the Ruby install being unaware of the system specific ca certs.

If I change the Vagrantfile as suggested it gives me a

/xxx/xxx/xxx/MyOcean/Vagrantfile:26:in block (2 levels) in <top (required)>': undefined local variable or methodvm' for main:Object (NameError)

So it looks like the suggestion in the earlier error does not work. It seems not to be a missing certificate but an issue with suggested code. Unfortunately I cannot find reference to vm.ca_path on the Vagrant site.

Also, I am wondering which certificate ought to be used here, mine, the droplet one or a DigitalOcean one? But I guess that will be the next issue :)

Any clues?

[MidGe48]

Regarding the passlib.hash algorithm, it is a better solution than my suggestion as each call to encrypt does generate a new salt thus ensuring that each hashed password use a different salt.

[rdickert]

On the encryption: Thanks, I'll go with that then in the documentation.

On the /path/to/ssl/ca/cert.crt problem: My guess is that something is not set up right with the digital_ocean plugin. I followed his instructions and got it done, but I remember spending a little time on it. Assuming that you successfully installed curl-ca-bundle, you might want to double check that you are running the bash command export SSL_CERT_FILE=/usr/local/opt/curl-ca-bundle/share/ca-bundle.crt every time your shell starts up. Typing echo $SSL_CERT_FILE should return the path from the command above. Note that this is on a Mac - it's possible that the path could possibly be different on different OSes. Good luck, and let me know if this doesn't get you anywhere.

[MidGe48]

re cert.crt: Curl does not supply ca-bundle any longer but I did build one from the pem files for Mozilla. That may be the issue however as $SSL_CERT_FILE points at my built ca-bundle. So I will keep on looking at this. BTW, I am working on a Debian derivative OS (CrunchBang), so there are some small differences.

In the case above though, it seems that the complain is about the vm.ca_path variable name itself(the suggestion in the error message) not being recognized and not about its value!?

Anyway, I will keep on lookin. I am surely learning a lot in the process. :)

[rdickert]

Sorry I'm not more help on this. Please keep me posted, especially if there is something that can help other users. I'd like to get the install process to be simpler and bulletproof. I'd assume any issues you are having would be an issue for other linux users.

[MidGe48]

Just to keep you informed, I think the issues are all related to openssh and the quick adhoc way of solving them seems to be by rebooting at every opportunity. This is unfortunately not a very satisfactory solution for me but I need a much more in-depth understanding of openssh before I come up with the correct way. So, I will keep at it. I am surely gaining a good understanding of ansible whilst I am doing this and I believe I am solving two major issues I have with my various and very dynamic os installs.

[rdickert]

Agreed, that's an unfortunate "solution," but it's at least narrowing things down. The main barrier to this tool chain seems to be installation and integration. Hopefully, some of this will improve as these tools mature, but if you have any further thoughts on either the packages/config or just best practices for setup, I'd love to add them in.