Open andrewclegg opened 10 years ago
Sorry, I just realized #25 is a duplicate. I also tried this from my 64-bit Windows 7 machine (running a 32 bit VM), and had a similar problem. In both cases I was using VirtualBox 4.3.10.
Okay, I read that the issue might be with 4.3.10, so I upgraded to 4.3.12. Below is the issue I got this time. I can log in (from my 32 bit Linux machine) but there is no scipy, and therefore no oct2py to test. Also, Octave was not installed (but I could get that using apt-get
).
==> charmed34: stdin: is not a tty
==> charmed34: salt-minion stop/waiting
==> charmed34: Checking numpy is intact...
==> charmed34: Imported numpy 1.8.1 OK
==> charmed34: Checking scipy is intact...
==> charmed34: Traceback (most recent call last):
==> charmed34: File "/root/bin/sanity_check.py", line 29, in <module>
==> charmed34: import scipy
==> charmed34: ImportError: No module named 'scipy'
The following SSH command responded with a non-zero exit status.
Vagrant assumes that this means the command failed!
chmod +x /tmp/vagrant-shell && /tmp/vagrant-shell
Stdout from the command:
salt-minion stop/waiting
Checking numpy is intact...
Imported numpy 1.8.1 OK
Checking scipy is intact...
Stderr from the command:
stdin: is not a tty
Traceback (most recent call last):
File "/root/bin/sanity_check.py", line 29, in <module>
import scipy
ImportError: No module named 'scipy'
From what I can tell none of the apt_pkgs in python34.sls are being installed, so all of the libraries that are affected are not installed. I verified that neither g++ nor Octave is installed, as well as libatlas and libhdf5 (using sudo updatedb
and locate libatlas
).
Thanks for helping out! Could you paste your whole log/charmed34/minion
and log/charmed34/pip.log
into gists please, and I'll take a look.
The files were too big for gist, it refused to paste them. I put them on dropbox: https://www.dropbox.com/l/JWipIftmwmuZVxQLcLdRNt? https://www.dropbox.com/l/rUSZcBH756wNxSlUQZkxnt?
Actually I don't think this is an issue relating to VirtualBox or your host. I think you're using a slightly out-of-date version. The minion log is showing this error in the apt stage:
E: Version '2.7.8.dfsg-5.1ubuntu4.7' for 'libxml2-dev' was not found
But the version currently specified in salt/roots/pillar/python34.sls
is 2.7.8.dfsg-5.1ubuntu4.9.
Annoyingly, when they release a new bugfix, they sometimes remove the older ones from the Ubuntu repos... No idea why!
Please try
git pull
vagrant destroy
vagrant up
I would like to move to a model where you can an image with most of the stuff you need pre-installed, which would make installation much faster and avoid these issues, but I need to work out where I can host them etc.
Will do. Have you considered using Anaconda to avoid having to compile some of these libraries? I use Python 3.4 Anaconda as part of my travis build environment.
I wouldn't rule it out, but I'd be a bit hesitant without understanding how well it would work if you mixed Anaconda packages with packages installed from source.
What happens, for example, if the version of Fortran or Atlas or whatever which was used to compile an Anaconda binary package, differs from the version installed on the VM? This can cause problems if you mix binary packages installed from apt with source packages from elsewhere... Is Anaconda different somehow?
Also, I'm not sure how Anaconda packages are licensed, but if they restrict people's ability to share or republish modified VMs, that'd rule it out.
Valid points. You could host the compiled packages on binstar.
Booted up just fine, and I ran the oct2py test:
vagrant@charmed34:~$ python3.4 -c 'import oct2py; print(oct2py.__version__)'
1.3.0
vagrant@charmed34:~$ nosetests oct2py
...
----------------------------------------------------------------------
Ran 70 tests in 296.369s
OK
Great news! This is with a 32-bit Linux host, yes? If you have a chance, would you be able to repeat it on your Windows host too?
Yes it was. I'm at the Scipy conference this week, I'll try on Windows this weekend.
To get it working on Windows 64 bit host, 32 bit client, I installed Vagrant and Virtualbox, and made the same changes to VagrantFile:
config.vm.box = "hashicorp/precise32"
v.memory = get_env("CHARMER_RAM", 1024)
v.cpus = get_env("CHARMER_CPUS", 1)
I also needed ssh
for vagrant ssh
, so I used the version in Git for Windows. Note: if I had git
itself on my Path, it would mess with vagrant, so I had to move it out of the same directory as ssh
for testing. After doing that, the oct2py
tests pass. I also ran through the Hello World notebook in my browser (Chrome).
@blink1073 -- I forgot to say at the time, thanks for the tips! I will add this into the documentation. (It never occurred to me that there were computers out there without ssh on... My bad)
Hey.
I'm having issues getting Snake Charmer to run on my Windows machine (Windows 7 64bit).
At first I had the issue blink1073 had where git was causing issues so I removed git and it progressed fine.
I get the issue:
==> charmed34: Waiting for machine to boot. This may take a few minutes...
The guest machine entered an invalid state while waiting for it
to boot. Valid states are 'starting, running'. The machine is in the
'poweroff' state. Please verify everything is configured
properly and try again.
The full message (from vagrant up
to the error message) can be found at in this gist.
After the error message it says that it might work if I try with the virtualbox GUI open, I've tried this and getting the same error.
I will try re-installing Virtualbox and Vagrant (installed them both to try this) and let you know if that changes anything.
Wasn't sure if I should open a new issue or reply to this existing one regarding problems with Windows.
Cheers.
Keiron.
You could try setting the CHARMER_RAM and CHARMER_CPUS environment variables to be lower than their defaults - maybe they're just too high for your machine's physical hardware?
See the customization guide for details.
I am on my way out now but can look into it in more detail later or tomorrow, if that doesn't work.
On Sunday, 10 August 2014, Ffisegydd notifications@github.com wrote:
Hey.
I'm having issues getting Snake Charmer to run on my Windows machine (Windows 7 64bit).
At first I had the issue blink1073 had where git was causing issues so I removed git and it progressed fine.
I get the issue:
==> charmed34: Waiting for machine to boot. This may take a few minutes... The guest machine entered an invalid state while waiting for it to boot. Valid states are 'starting, running'. The machine is in the 'poweroff' state. Please verify everything is configured properly and try again.
The full message (from vagrant up to the error message) can be found at in this gist https://gist.github.com/Ffisegydd/3765e933bc2fafeb1165.
After the error message it says that it might work if I try with the virtualbox GUI open, I've tried this and getting the same error.
I will try re-installing Virtualbox and Vagrant (installed them both to try this) and let you know if that changes anything.
Wasn't sure if I should open a new issue or reply to this existing one regarding problems with Windows.
Cheers.
Keiron.
Reply to this email directly or view it on GitHub https://github.com/snake-charmer-devs/snake-charmer/issues/24#issuecomment-51712187 .
The latest build of VirtualBox is having serious issues which was causing my problem.
The VM is now booting but I'm having issues with it getting stuck on Calling state.highstate... (this may take a while)
. How long is a "while" in this case? I've left it for 20 minutes so far but no luck. I've restarted it with --provision
and will leave it going and see how long it takes.
That's normal, it'll take well over an hour to download and install all the packages! Maybe I should be more explicit about that :-)
Not sure exactly how long, currently, but that depends on your network connection, hardware etc.
Later builds will be a bit quicker (if you need to do them) as all the packages get cached outside the vm, in .cache, meaning they won't need to be downloaded again in general.
On Sunday, 10 August 2014, Keiron Pizzey notifications@github.com wrote:
The latest build of VirtualBox is having serious issues https://forums.virtualbox.org/viewtopic.php?f=6&t=62615 which was causing my problem.
The VM is now booting but I'm having issues with it getting stuck on Calling state.highstate... (this may take a while). How long is a "while" in this case? I've left it for 20 minutes so far but no look. I've restarted it with --provision and will leave it going and see how long it takes.
Reply to this email directly or view it on GitHub https://github.com/snake-charmer-devs/snake-charmer/issues/24#issuecomment-51720108 .
It takes me about an hour from a fresh install.
I've tried to install snake charmer on a Vista machine and only made it as far as launching charmed 34 from the Windows shell but it failed soon after updating vbguest plugin. I have no experience with Vagrant and don't really understand why is complaining about the loader.rb:xx and how to fix the issue.
C:\Users\bpajusco\Documents\snake-charmer>vagrant up charmed34
Installing the 'vagrant-vbguest' plugin. This can take a few minutes...
Installed the plugin 'vagrant-vbguest (0.10.0)'!
C:/Users/bpajusco/Documents/snake-charmer/Vagrantfile:152:in `': No such file o r directory - C:\Program Files\Git\cmd/git.EXE log --no-merges -n 1 (Errno::ENOE NT) from C:/Users/bpajusco/Documents/snake-charmer/Vagrantfile:152:in
block
in <top (required)>'
from C:/HashiCorp/Vagrant/embedded/gems/gems/vagrant-1.6.3/lib/vagrant/c
onfig/v2/loader.rb:37:in call' from C:/HashiCorp/Vagrant/embedded/gems/gems/vagrant-1.6.3/lib/vagrant/c onfig/v2/loader.rb:37:in
load'
from C:/HashiCorp/Vagrant/embedded/gems/gems/vagrant-1.6.3/lib/vagrant/c
onfig/loader.rb:103:in block (2 levels) in load' from C:/HashiCorp/Vagrant/embedded/gems/gems/vagrant-1.6.3/lib/vagrant/c onfig/loader.rb:97:in
each'
from C:/HashiCorp/Vagrant/embedded/gems/gems/vagrant-1.6.3/lib/vagrant/c
onfig/loader.rb:97:in block in load' from C:/HashiCorp/Vagrant/embedded/gems/gems/vagrant-1.6.3/lib/vagrant/c onfig/loader.rb:94:in
each'
from C:/HashiCorp/Vagrant/embedded/gems/gems/vagrant-1.6.3/lib/vagrant/c
onfig/loader.rb:94:in load' from C:/HashiCorp/Vagrant/embedded/gems/gems/vagrant-1.6.3/lib/vagrant/v agrantfile.rb:28:in
initialize'
from C:/HashiCorp/Vagrant/embedded/gems/gems/vagrant-1.6.3/lib/vagrant/e
nvironment.rb:574:in new' from C:/HashiCorp/Vagrant/embedded/gems/gems/vagrant-1.6.3/lib/vagrant/e nvironment.rb:574:in
vagrantfile'
from C:/HashiCorp/Vagrant/embedded/gems/gems/vagrant-1.6.3/lib/vagrant/e
nvironment.rb:359:in host' from C:/HashiCorp/Vagrant/embedded/gems/gems/vagrant-1.6.3/lib/vagrant/e nvironment.rb:168:in
block in action_runner'
from C:/HashiCorp/Vagrant/embedded/gems/gems/vagrant-1.6.3/lib/vagrant/a
ction/runner.rb:33:in call' from C:/HashiCorp/Vagrant/embedded/gems/gems/vagrant-1.6.3/lib/vagrant/a ction/runner.rb:33:in
run'
from C:/HashiCorp/Vagrant/embedded/gems/gems/vagrant-1.6.3/lib/vagrant/e
nvironment.rb:346:in hook' from C:/HashiCorp/Vagrant/embedded/gems/gems/vagrant-1.6.3/lib/vagrant/e nvironment.rb:556:in
unload'
from C:/HashiCorp/Vagrant/embedded/gems/gems/vagrant-1.6.3/bin/vagrant:1
69:in ensure in <main>' from C:/HashiCorp/Vagrant/embedded/gems/gems/vagrant-1.6.3/bin/vagrant:1 69:in
@bpajusco -- I think this is a problem with spaces in file paths -- I'll attempt to do a fix, and then get you to test it on Windows :-)
Okay @bpajusco, please clone a fresh copy from the master branch, and give it another go -- I think I've fixed it...
Thanks, your patch definitely did the trick but somehow I can't stop breaking your creature. Now the problem I have seems to be with authenticating ssh. Once again, I can't fathom why that is possible since I'm using ssh to logon to AWS servers. Here is the full log of my issue and note that I manually stop the process after a few failed attempts but let it go a lot longer the first time around:
C:\Users\bpajusco\Documents\snake-charmer>vagrant up charmed34 Bringing machine 'charmed34' up with 'virtualbox' provider... ==> charmed34: Checking if box 'hashicorp/precise64' is up to date... ==> charmed34: Setting the name of the VM: charmed34 ==> charmed34: Clearing any previously set network interfaces... ==> charmed34: Preparing network interfaces based on configuration... charmed34: Adapter 1: nat ==> charmed34: Forwarding ports... charmed34: 8834 => 8834 (adapter 1) charmed34: 22 => 2234 (adapter 1) charmed34: 9034 => 9034 (adapter 1) charmed34: 9134 => 9134 (adapter 1) charmed34: 9234 => 9234 (adapter 1) charmed34: 9334 => 9334 (adapter 1) charmed34: 9434 => 9434 (adapter 1) charmed34: 9534 => 9534 (adapter 1) charmed34: 9634 => 9634 (adapter 1) charmed34: 9734 => 9734 (adapter 1) charmed34: 9834 => 9834 (adapter 1) charmed34: 9934 => 9934 (adapter 1) charmed34: 22 => 2222 (adapter 1) ==> charmed34: Running 'pre-boot' VM customizations... ==> charmed34: Booting VM... ==> charmed34: Waiting for machine to boot. This may take a few minutes... charmed34: SSH address: 127.0.0.1:2222 charmed34: SSH username: vagrant charmed34: SSH auth method: private key charmed34: Warning: Connection timeout. Retrying... charmed34: Warning: Connection timeout. Retrying... ==> charmed34: Waiting for cleanup before exiting...
Also, after finding this problem I have to delete the machine otherwise it complains about a previous istance of charmed34 being still active.
Thanks, Bruno
This is usually due to the VM asking for some input at boot time (e.g. "which kernel version should I load" or "something went wrong, should I boot in safe mode" stuff).
Not sure why it would ask you that on a fresh VM though.
Can you edit the Vagrantfile and find the line that says:
charmed34.vm.provider "virtualbox" do |v|
and add this underneath it:
v.gui = true
That should boot up the VM with a GUI, so you can see any boot-time messages.
I got that idea from this thread:
http://stackoverflow.com/questions/22575261/vagrant-stuck-connection-timeout-retrying
Failing that, there's another suggestion here about BIOS settings that might apply:
Thanks. Looks like i have a virtualisation issue. I will have to either ditch Vista (which it's about time) or install snake charmer on another machine. Thanks for your help.
@bpajusco -- any chance you could summarize the symptoms and cause, so I can document it as a known issue for Vista users?
Sure. I think it was an issue specific to my machine and not with your code. When I tried to enable virtualisation at BIOS level (from the startup screen) I simply did not have the options available to enable vt-x. I then tried to run Ubuntu in parallel with Vista but even thath wasn't playing nicely and then ditched Vista altogether. As I said, I think it had more to do with the machine than any aspects of your code.
Ah, ok, it sounds like maybe you have an older BIOS (or CPU) that doesn't support vt-x at all...
On 19 August 2014 19:55, bpajusco notifications@github.com wrote:
Sure. I think it was an issue specific to my machine and not with your code. When I tried to enable virtualisation at BIOS level (from the startup screen) I simply did not have the options available to enable vt-x. I then tried to run Ubuntu in parallel with Vista but even thath wasn't playing nicely and then ditched Vista altogether. As I said, I think it had more to do with the machine than any aspects of your code.
Reply to this email directly or view it on GitHub https://github.com/snake-charmer-devs/snake-charmer/issues/24#issuecomment-52681737 .
Hey, so I am running
C:\Programming\snake-charmer>vagrant up charmed34
And I get the following ouput:
Bringing machine 'charmed34' up with 'virtualbox' provider... ==> charmed34: Checking if box 'hashicorp/precise64' is up to date... ==> charmed34: Setting the name of the VM: charmed34 There was an error while executing `VBoxManage`, a CLI used by Vagrant for controlling VirtualBox. The command and stderr is shown below. Command: ["modifyvm", "ee00cc26-4dba-4f7e-8764-bd3dfb884891", "--name", "charmed 34"] Stderr: VBoxManage.exe: error: Could not rename the directory 'C:\Users\Jakub\Vi rtualBox VMs\precise64_1412093545259_42575' to 'C:\Users\Jakub\VirtualBox VMs\ch armed34' to save the settings file (VERR_ALREADY_EXISTS) VBoxManage.exe: error: Details: code E_FAIL (0x80004005), component SessionMachi ne, interface IMachine, callee IUnknown VBoxManage.exe: error: Context: "SaveSettings()" at line 2788 of file VBoxManage ModifyVM.cpp
I have manually set VirtualBox as an environmental variable and I can run VBoxManage
commands from console just fine now, but the problem persist even after unregistering both vms listed.
C:\Programming\snake-charmer>VBoxManage list vms "precise64_1412092507610_54372" {9ce09a14-fe26-4b4e-a735-717c7776969c} "precise64_1412093545259_42575" {ee00cc26-4dba-4f7e-8764-bd3dfb884891} C:\Programming\snake-charmer>VBoxManage list vms
I just get a new machine ID each time I run it. So I tried manually deleting all the files in the noted destination and I got the following problem, which is where I got stuck:
C:\Programming\snake-charmer>vagrant up charmed34 Bringing machine 'charmed34' up with 'virtualbox' provider... ==> charmed34: Importing base box 'hashicorp/precise64'... ==> charmed34: Matching MAC address for NAT networking... ==> charmed34: Checking if box 'hashicorp/precise64' is up to date... ==> charmed34: Setting the name of the VM: charmed34 ==> charmed34: Clearing any previously set network interfaces... ==> charmed34: Preparing network interfaces based on configuration... charmed34: Adapter 1: nat ==> charmed34: Forwarding ports... charmed34: 8834 => 8834 (adapter 1) charmed34: 22 => 2234 (adapter 1) charmed34: 9034 => 9034 (adapter 1) charmed34: 9134 => 9134 (adapter 1) charmed34: 9234 => 9234 (adapter 1) charmed34: 9334 => 9334 (adapter 1) charmed34: 9434 => 9434 (adapter 1) charmed34: 9534 => 9534 (adapter 1) charmed34: 9634 => 9634 (adapter 1) charmed34: 9734 => 9734 (adapter 1) charmed34: 9834 => 9834 (adapter 1) charmed34: 9934 => 9934 (adapter 1) charmed34: 22 => 2222 (adapter 1) ==> charmed34: Running 'pre-boot' VM customizations... ==> charmed34: Booting VM... ==> charmed34: Waiting for machine to boot. This may take a few minutes... The guest machine entered an invalid state while waiting for it to boot. Valid states are 'starting, running'. The machine is in the 'paused' state. Please verify everything is configured properly and try again.
EDIT: Some other people facing the same problem suggested running vagrant provision
, here is my output:
C:\Programming\snake-charmer>vagrant provision ==> charmed34: VM is not currently running. Please, first bring it up with `vagr ant up` then run this command.
@jakubLangr - are you running the current version of virtualbox? I have seen a couple of discussions mentioning a conflict between VB and some antivirus packages on Windows, for example:
https://forums.virtualbox.org/viewtopic.php?f=6&t=62615
Apparently fixed in more recent versions. But even if you do have a more recent version, it may be worth temporarily disabling your AV to see if that works.
@andrewclegg: Yes, I downloaded the latest version:
C:\Programming\snake-charmer>VBoxManage --version 4.3.16r95972
My Anti-Virus does not give any notifications of blocking anything so I would not assume that is an issue. But I will investigate further.
Hmm... If it's not that, then I guess the next step is to build a completely new VM in a different directory:
vagrant init hashicorp/precise64
Then we can at least find out whether the difference is due to something in Snake Charmer's Vagrantfile, or something more fundamental.
@andrewclegg : I managed to enable gui and I restarted the machine, so I can now see that the vm is running somewhere, however, snakecharmer does not have access to it. I installed some ssh
client for Windows, but it might be possible that that is still the issue. Any easy way to test that?
This is the output of when I tried to initiate a new vm in a new directory:
C:\Programming\testvm>vagrant init precise64 A `Vagrantfile` has been placed in this directory. You are now ready to `vagrant up` your first virtual environment! Please read the comments in the Vagrantfile as well as documentation on `vagrantup.com` for more information on using Vagrant. fatal: Not a git repository (or any of the parent directories): .git C:\Programming\testvm>vagrant up Bringing machine 'default' up with 'virtualbox' provider... ==> default: Box 'precise64' could not be found. Attempting to find and install. .. default: Box Provider: virtualbox default: Box Version: >= 0 ==> default: Adding box 'precise64' (v0) for provider: virtualbox default: Downloading: precise64 default: An error occurred while downloading the remote file. The error message, if any, is reproduced below. Please fix this error and try again. Couldn't open file /Programming/testvm/precise64
You need to include the org name as well as the box name, I think:
vagrant init hashicorp/precise64
You had left out the "hashicorp/" part, so it was looking for a local box file called precise64.
Re. ssh, you need to make sure ssh.exe (from Putty or whatever ssh client you installed) is in your PATH. Not sure how to do that on Windows, to be honest!
Oh yes, sorry, I tried that before, but forgot to include it. I thought it was clashing with something so I wanted to try something different:
C:\Programming\testvm>vagrant init hashicorp/precise64 `Vagrantfile` already exists in this directory. Remove it before running `vagrant init`.
Re: SSH, I think it is added to PATH--I can run ssh help from command prompt, but is there anything else?
Sorry, that was quite silly: so I removed the Vagrant file and this is what I got:
C:\Programming\testvm>vagrant init hashicorp/precise64 A `Vagrantfile` has been placed in this directory. You are now ready to `vagrant up` your first virtual environment! Please read the comments in the Vagrantfile as well as documentation on `vagrantup.com` for more information on using Vagrant. fatal: Not a git repository (or any of the parent directories): .git C:\Programming\testvm>vagrant up Bringing machine 'default' up with 'virtualbox' provider... ==> default: Importing base box 'hashicorp/precise64'... ==> default: Matching MAC address for NAT networking... ==> default: Checking if box 'hashicorp/precise64' is up to date... ==> default: There was a problem while downloading the metadata for your box ==> default: to check for updates. This is not an error, since it is usually due ==> default: to temporary network problems. This is just a warning. The problem ==> default: encountered was: ==> default: ==> default: Could not resolve host: (nil); Host not found ==> default: ==> default: If you want to check for box updates, verify your network connectio n ==> default: is valid and try again. ==> default: Setting the name of the VM: testvm_default_1412159376532_81080 ==> default: Clearing any previously set network interfaces... ==> default: Preparing network interfaces based on configuration... default: Adapter 1: nat ==> default: Forwarding ports... default: 22 => 2222 (adapter 1) ==> default: Booting VM... ==> default: Waiting for machine to boot. This may take a few minutes... default: SSH address: 127.0.0.1:2222 default: SSH username: vagrant default: SSH auth method: private key default: Warning: Connection refused. Retrying... default: Warning: Connection refused. Retrying...
EDIT: I am sure that virtualization is enabled (from the above issues) and I have tried enabling gui, which gets stuck on the issue as per previous screen shot.
EDIT 2: Some SO thread suggested that it might be Hyper-V but I do not seem to have that installed on my system (hence Hyper-V is unlikely to be causing the issue) and I think that is the correct behavior for Windows 8.
Hmm, I think this is a bit beyond my ability to help with, I'm afraid! If Vagrant can't even bring up a fresh precise64
box then it sounds like there's an underlying issue with Vagrant or Virtualbox, rather than Snake Charmer. You might get further by posting this as a Vagrant issue. Sorry!
Just one last thought: since I can ssh in-GUI, is there some way of running the remainder of snake-charmer using the terminal in there? (see my previous screen shot)
Thanks!
@jakubLangr yeah, it's just a Linux machine, so you can use it from the command line to edit/run python scripts.
Thinking about it though... It looks like the port forwarding step in Vagrant has completed successfully. Have you checked to see whether you can access the IPython Notebook server on localhost:8834 (from the host)?
If you're lucky, maybe that's up and running fine, and it's only ssh that's broken...
If you have a chance to test Snake Charmer on any Windows machine, or on any 32-bit OS, please let us know in the comments below how you fared. Or, simply open new issues for any specific problems you encountered. Thanks :-)