scalingexcellence / scrapybook

Scrapy Book Code
http://scrapybook.com/
475 stars 209 forks source link

Could you please give a brief guide on how to setup the environment without vigrent. #38

Open lefeudelavie opened 6 years ago

lefeudelavie commented 6 years ago

Hi lookfwd,

I have seen in the readme , it's said after you add the "docker-compose.yml", so there is no need to install the Vagrant. But could you give me a brief guide on how to setup the env without Vagrant?

Sincerely, Lefeudelavie

lookfwd commented 6 years ago

Theoretically - you install docker from here. Then you checkout the repo, you go to the root directory and do docker-compose up. This should easily bring up all the containers. Then one should be able to ssh to them with ssh -i insecure_key root@hostname_of_the_container. @normangilmore - what was the easiest way to ssh to containers?

normangilmore commented 6 years ago

To get into containers, there are two options:

The mildly tedious way is docker ps to get the hex container id, then docker exec -it LONGHEXID bash, but the easy way is docker-compose exec web bash or docker-compose exec spark bash, substituting a container name listed as a services key in docker-compose.yml. The advantage of the first technique is it works from any directory, the second technique requires you to be in or under the directory containing docker-compose.yml.

Note that docker containers are not running sshd as a general rule, so you can't ssh into them. Containers are process wrappers, not virtual machines with a full set of services.

Hope this helps!

normangilmore commented 6 years ago

If and only if you are running Docker Toolbox (which is different than Docker for Mac or Docker for PC!), then you need to first ssh into the VirtualBox VM that docker is running on your behalf.

ssh docker@$(docker-machine ip) Your ports will be exposed on the internal docker-machine ip, which is usually something like 192.168.99.100.

If you want your ports exposed on localhost, you can do something like this: ssh -L 80:$(docker-machine ip):80 -L 8080:$(docker-machine ip):8080 docker@$(docker-machine ip), adjusting source and dest ports as needed. You will get a password request, this is for the docker-machine user account on the VM, the password is tcuser. Just leave that window open, and your ports will be tunneled so you can access with http://localhost:8080 or whatever port you mapped to, instead of http://192.168.99.100:8080/

lefeudelavie commented 6 years ago

@lookfwd @normangilmore Thanks for your prompt response.

I have tried to run the "docker-compose up" in the root, but unfortunately, I get an error as follows: D:\code\scrpaybook\scrapybook>docker-compose up Traceback (most recent call last): File "docker-compose", line 3, in File "compose\cli\main.py", line 68, in main File "compose\cli\main.py", line 118, in perform_command File "compose\cli\main.py", line 928, in up File "compose\project.py", line 414, in up File "compose\project.py", line 640, in warn_for_swarm_mode File "site-packages\docker\api\daemon.py", line 90, in info File "site-packages\docker\utils\decorators.py", line 46, in inner File "site-packages\docker\api\client.py", line 189, in _get File "site-packages\requests\sessions.py", line 488, in get File "site-packages\requests\sessions.py", line 475, in request File "site-packages\requests\sessions.py", line 596, in send File "site-packages\requests\adapters.py", line 423, in send File "site-packages\requests\packages\urllib3\connectionpool.py", line 595, in urlopen File "site-packages\requests\packages\urllib3\connectionpool.py", line 363, in _make_request File "httplib.py", line 1042, in request File "httplib.py", line 1082, in _send_request File "httplib.py", line 1038, in endheaders File "httplib.py", line 882, in _send_output File "httplib.py", line 844, in send File "site-packages\docker\transport\npipeconn.py", line 31, in connect File "site-packages\docker\transport\npipesocket.py", line 22, in wrapped File "site-packages\docker\transport\npipesocket.py", line 50, in connect pywintypes.error: (2, 'WaitNamedPipe', 'The system cannot find the file specified.') Failed to execute script docker-compose

My install steps are: 1, download the dockertoolbox and install the docker and virtualbox. 2, git clone the scrapybook code to a dir. 3, run "docker-compose up" in above-mentioned dir which have downloaded the code. And then the error .

Thanks!

normangilmore commented 6 years ago

I just installed DockerToolbox and Virtualbox on Windows 7, and after I ran VirtualBox.exe -msiparams NETWORKTYPE=NDIS5 to downgrade the network driver for Windows 7, I was able to docker-compose up in this repo.

Are you in the Docker Quickstart terminal?

What do you get for python --version?

normangilmore commented 6 years ago

Also, are you able to run docker run hello-world?

lefeudelavie commented 6 years ago

@normangilmore Thanks for your response. I am able to run "docker run hello-world" on my PC (windows 7 64 bits). I have installed two python, one is 2.7, another is 3.5.

python --version output as following: D:\code\scrapybook>python --version Python 2.7.13

lefeudelavie commented 6 years ago

@normangilmore And I also have tried to disable path for Python 2.7.13 and change it to Python 3.5, hence I can get Python 3.5 when run 'python --version'. But get the same error when run 'docker-compose up'

normangilmore commented 6 years ago

docker quickstart icon docker quickstart terminal

If you are in the Docker Quickstart Terminal, the proper env variables should be set, as shown in image. But you can set them with the command shown as well.

Enter this command to set them, then try it. eval $("C:\Program Files\Docker Toolbox\docker-machine.exe" env)

I found the suggestion here, https://github.com/shipping-docker/php-app/issues/19 docs here: https://docs.docker.com/machine/reference/env/

docker-compose may also want the newer Python I think.

lefeudelavie commented 6 years ago

@normangilmore I just checked the docker as you mentioned: image But this didn't solve my problem. I think the problem is caused by some other reason. I am trying to reinstall related sw to see if it work.

xhf79 commented 6 years ago

Hi, I don't want to establish a virtual envirment. And ,what I care about is, how to output the results of the code programms? I only want to see the results in IDLE or Ipython shell or in a file. But I really don't know how to do, what code should I append in the programm.

Gaxadyin commented 5 years ago

The box 'lookfwd/scrapybook' could not be found or could not be accessed in the remote catalog. If this is a private box on HashiCorp's Vagrant Cloud, please verify you're logged in via vagrant login. Also, please double-check the name. The expanded URL and error message are shown below:

URL: ["https://vagrantcloud.com/lookfwd/scrapybook"] Error: Failed to connect to vagrantcloud.com port 443: Timed out