[demo] Running locally gives Fatal error: [ERROR] File "/project/rector.php" was not found

[ruudk]

I'm using bin/run-from-docker.sh to start the project, then I go to http://localhost:8080/ and visit the website. I click demo and click Process. then I get:

Would be really great if the Readme can be fixed + tested on a new checkout so that everything works.

[ruudk]

/cc @JanMikes

[JanMikes]

Hi, i found this is not issue with docker setup itself but seems like you have outdated image locally.

Running docker pull rector/rector-secured:latest should fix your issue. Please confirm so we can close ;-)

I will as well update readme.

[ruudk]

$ docker pull rector/rector-secured:latest
latest: Pulling from rector/rector-secured
a076a628af6f: Already exists
02bab8795938: Already exists
657d9d2c68b9: Already exists
f47b5ee58e91: Already exists
83c81249613f: Pull complete
b6d2f87d40f1: Pull complete
7504890bf8c1: Pull complete
54458ea35e4e: Pull complete
25e05b97b27e: Pull complete
6ab89feb2696: Pull complete
8578d3b27b67: Pull complete
6a3bc4a67e1d: Pull complete
9048cf9d4477: Pull complete
79638ad4d518: Pull complete
181b13a23f10: Pull complete
fca83b9cdfb8: Pull complete
8d0c5e17e963: Pull complete
4790f14dfaf1: Pull complete
4201aaa55b52: Pull complete
2351b3cd7f63: Pull complete
09269212ebbb: Pull complete
93e4bd2a22a5: Pull complete
c5a36b4f7f63: Pull complete
Digest: sha256:8eac8cb60df7454f6b0472f9118981f7709505eb00248ed7e8f31c452a9bf6cc
Status: Downloaded newer image for rector/rector-secured:latest
docker.io/rector/rector-secured:latest
~/o/getrector.org (error-nl2br|✔) $ bin/run-from-docker.sh
Starting getrectororg_js-watch_1 ... done
Starting getrectororg_mysql_1    ... done
Starting getrectororg_adminer_1  ... done
Starting getrectororg_web_1      ... done
Attaching to getrectororg_adminer_1, getrectororg_js-watch_1, getrectororg_mysql_1, getrectororg_web_1
// ...

But still the same issue:

[JanMikes]

Damn, i am unable to reproduce this. I started fresh VM with only ubuntu + git + docker. git clone this repo, docker-compose up and run demo and everything works :-/

[ruudk]

Could you maybe please update the Readme so that I can follow those steps? I'll do a fresh checkout too and see if that works.

[JanMikes]

Already in progress in #251

[ruudk]

Same issue, Fatal error: [ERROR] File "/project/rector.php" was not found

[ruudk]

Tried everything. Ran docker-compose stop, docker-compose down, docker-compose rm, docker ps = empty, checked out repo in fresh location, it just doesn't work.

Btw, I'm on macOS.

[ruudk]

also docker-compose pull and docker pull rector/rector-secured:latest

[JanMikes]

Seems like for some reason config file is not created correctly or not passed via volume into demo running docker container: packages/demo/src/Process/RectorProcessRunner.php:61

I have some hints how to debug and find out where it breaks:

1) Comment out packages/demo/src/Process/RectorProcessRunner.php:58 and check if config file is present (default path is %kernel.project_dir%/var/demo )

You should see something like:

root@b11dc1a858c7:/var/www/getrector.org/var# ls
cache  demo  log
root@b11dc1a858c7:/var/www/getrector.org/var# ls demo
uhp5ntg7efoq7va2s18d
root@b11dc1a858c7:/var/www/getrector.org/var# ls demo/uhp5ntg7efoq7va2s18d/
rector.php  rector_analyzed_file.php

[JanMikes]

I forget to mention that you have to run demo in order to get temp files generated.

If files rector.php rector_analyzed_file.php are there, next step is:

docker run \
    --volume /var/www/getrector.org/var/demo/uhp5ntg7efoq7va2s18d:/project \   # tweak path to your temporary directory, here is mine 
    rector/rector-secured:latest \
    process /project/rector_analyzed_file.php \
    --autoload-file /project/rector_analyzed_file.php \
    --output-format json \
    --config /project/rector.php

My output is:

Warning: proc_open() has been disabled for security reasons in /rector/vendor/symfony/console/Terminal.php on line 162
Warning: proc_open() has been disabled for security reasons in /rector/vendor/symfony/console/Terminal.php on line 162
{
    "meta": {
        "version": "dev-master@2085291",
        "config": "/project/rector.php"
    },
    "totals": {
        "changed_files": 0,
        "removed_and_added_files_count": 0,
        "removed_node_count": 0,
        "errors": 0
    }
}

If files are missing, i will provide another debugging steps.

I am using ubuntu VM on macos. Give me few mins, i will try directly on mac too.

[ruudk]

When I do docker-compose exec web bash I can see the cache:

root@867f3a4e1c74:/var/www/getrector.org/var/demo# pwd
/var/www/getrector.org/var/demo
root@867f3a4e1c74:/var/www/getrector.org/var/demo# ls -la
total 0
drwxr-xr-x 3 root     root  96 Feb  5 18:10 .
drwxr-xr-x 5 www-data root 160 Feb  5 18:10 ..
drwxr-xr-x 4 root     root 128 Feb  5 18:10 j2p7h2h1bkyk43r3ahye
root@867f3a4e1c74:/var/www/getrector.org/var/demo# cd j2p7h2h1bkyk43r3ahye/
root@867f3a4e1c74:/var/www/getrector.org/var/demo/j2p7h2h1bkyk43r3ahye# ls -la
total 8
drwxr-xr-x 4 root root 128 Feb  5 18:10 .
drwxr-xr-x 3 root root  96 Feb  5 18:10 ..
-rw-rw-rw- 1 root root 582 Feb  5 18:10 rector.php
-rw-rw-rw- 1 root root 147 Feb  5 18:10 rector_analyzed_file.php

but this is not visible on my host system (macos).

I think the run-demo.sh that is opened, tries to mount from my host, while the demo cache is not there.

[JanMikes]

 but this is not visible on my host system (macos). I think the run-demo.sh that is opened, tries to mount from my host, while the demo cache is not there.

You are right, so we have to find out why those files are not available on your host machine.

[ruudk]

Maybe it's related to :delegated?

[ruudk]

Or ``` tmpfs:

• /var/www/getrector.org/var/cache

[JanMikes]

Maybe it's related to :delegated?

Please give it a try, remove and re-run docker-compose up

Or tmpfs: - /var/www/getrector.org/var/cache?

Should not be the case, because /var/cache vs /var/demo - different directories.

[ruudk]

Tried without :delegated. Does not solve it.

I think we should create a new volume that is located on var/demo that is also used by run-demo.sh.

BTW I'm on Docker version 20.10.2, build 2291f61

[JanMikes]

I am still waiting until my docker-for-mac finishes docker-compose up, that is why i use VM for docker, uncomparable performance :smile:

I believe it should not be required as previously it worked on mac and ubuntu too, but let's give it a try. I will prepare branch for testing this.

[JanMikes]

Please try this command:

docker run --rm -v $(pwd):/tmp alpine:latest ash -c "echo 'test' > /tmp/test-file.txt"

In your current directory, file test-file.txt should appear. It is emulating nearly the same behaviour of what is demo doing - create file in mounted volume and it should appear on host.

[ruudk]

It does appear!

[ruudk]

But I execute this from my host machine.... Does that matter?

[JanMikes]

Ok, now delete the file and run this:

docker run --entrypoint="" --rm  -v /var/run/docker.sock:/var/run/docker.sock -v $(pwd):$(pwd) getrector_org bash -c \
 "docker run --rm -v $(pwd):/tmp alpine:latest ash -c \"echo 'test' > /tmp/test-file.txt\""

This is exactly 1:1 the same what is doing the demo. Docker in docker and volume to volume 😄

[ruudk]

I have it working. Let me create a patch so that you can see what I did.

[JanMikes]

Ok, finally after 20 min of docker spinning on my mac, i confirm i can reproduce the issue. Seems like it is MacOS specific. But now as i can repro it should not be hard to find a fix.

I have it working. Let me create a patch so that you can see what I did.

Or you got the fix already? 😄

[JanMikes]

But it is weird as files are appearing on my host when running demo, but somehow seems like volume is failing.

[ruudk]

Give me a few minutes to verify my patch :)

[ruudk]

From 50994dcc7e8445f5ee51f8095067d4e506ed34cf Mon Sep 17 00:00:00 2001
From: Ruud Kamphuis <ruudk@users.noreply.github.com>
Date: Fri, 5 Feb 2021 19:46:21 +0100
Subject: [PATCH] test

---
 .env                                              | 2 +-
 bin/run-demo.sh                                   | 6 +++---
 docker-compose.dist.yml                           | 5 +++++
 packages/demo/src/Process/RectorProcessRunner.php | 2 +-
 4 files changed, 10 insertions(+), 5 deletions(-)

diff --git a/.env b/.env
index 19232ec..2edd27f 100644
--- a/.env
+++ b/.env
@@ -24,7 +24,7 @@ DATABASE_CHARSET=utf8

 DEBUG_COOKIE=

-HOST_DEMO_DIR=/var/www/getrector.org/var/demo
+HOST_DEMO_DIR=getrectororg_demo

 ###> sentry/sentry-symfony ###
 SENTRY_DSN=
diff --git a/bin/run-demo.sh b/bin/run-demo.sh
index 94a3252..1d3e591 100755
--- a/bin/run-demo.sh
+++ b/bin/run-demo.sh
@@ -33,10 +33,10 @@ docker run \
     --name ${NAME} \
     --volume ${VOLUME_SOURCE}:/project \
     ${DOCKER_IMAGE} \
-    process /project/rector_analyzed_file.php \
-    --autoload-file /project/rector_analyzed_file.php \
+    process /project/${NAME}/rector_analyzed_file.php \
+    --autoload-file /project/${NAME}/rector_analyzed_file.php \
     --output-format json \
-    --config /project/rector.php
+    --config /project/${NAME}/rector.php

 RECTOR_RUN_EXIT_CODE=$?

diff --git a/docker-compose.dist.yml b/docker-compose.dist.yml
index ab5bd53..2240b12 100644
--- a/docker-compose.dist.yml
+++ b/docker-compose.dist.yml
@@ -17,6 +17,9 @@ services:
         ports:
             - "8080:80"
         volumes:
+            -   type: volume
+                source: demo
+                target: /var/www/getrector.org/var/demo
             - /var/run/docker.sock:/var/run/docker.sock
             - .:/var/www/getrector.org:delegated
         tmpfs:
@@ -49,3 +52,5 @@ services:
         restart: unless-stopped
         ports:
             - 8081:8080
+volumes:
+    demo:
diff --git a/packages/demo/src/Process/RectorProcessRunner.php b/packages/demo/src/Process/RectorProcessRunner.php
index 063292a..4f281e5 100644
--- a/packages/demo/src/Process/RectorProcessRunner.php
+++ b/packages/demo/src/Process/RectorProcessRunner.php
@@ -97,7 +97,7 @@ final class RectorProcessRunner

     private function createProcess(string $identifier): Process
     {
-        $volumeSourcePath = $this->hostDemoDir . '/' . $identifier;
+        $volumeSourcePath = $this->hostDemoDir;

         return new Process([
             // Because user www-data runs docker owned by root, we need to use sudo

This works :)

[ruudk]

I think this is the way to go, as it uses a volume to exchange the data between containers.

[JanMikes]

Oh my... you found it, it is all about HOST_DEMO_DIR env var that has to be changed for you to work (my local path matched it so that is why it worked in my vm).

I am afraid i can not accept your patch as it seems it might break production but i already have in my mind how to fix it globally.

Need short break for now, in few hours you can expect fix in master.

Thank you very much for spending your time and ivestigating. You seem to be first one running demo form locally (apart of rector team) 😁

[ruudk]

🎉🥳

No rush! Enjoy the weekend and I'm gonna try to finish my PR's next week!

[ruudk]

Btw, one thing I noticed that is annoying, is that run-from-docker.sh copies the compose dist once. So next runs you need to delete it. Why having this at all? Keep it simple, just one default yaml. It makes it easier to make changes and test them.

[JanMikes]

I understand and totally agree 👍

run-from-docker.sh no longer exists in master and directly docker-compose up should be used instead.

[JanMikes]

Hi @ruudk please would you be so nice and try checkout branch issue-249/dynamic-host-demo-dir-resolution and try if it works for you?

I think this is more elegant and less robust fix 😄 though i am afraid one day, when any developer on windows will want to run this repo 😄

[ruudk]

Works 🎉

[JanMikes]

To explain, why i did not want to use your patch - in production it would mean that there is named volume and all files will be stored same directory and it is accessible in demo docker container - imagine if in parallel 20 demos run, you could access files in your code from other demos run, maybe i am too obsessed with security 😄

[ruudk]

I understand , thanks for fixing it ☺️

[ruudk]

@JanMikes Same issue again.... :( This is really annoying...

docker pull rector/rector-secured
Digest: sha256:83d429790dca4b0028ffac47ac02b6eabf6fc261a8ca57f192ece87a9ed5b9a4
Status: Downloaded newer image for rector/rector-secured:latest

[JanMikes]

Hi, please create docker-compose.override.yml:

version: "3.7"
services:
    web:
        environment:
            HOST_DEMO_DIR: "/my/absolute/location/to-project-on-host-machine"

We had to revert as it did not work on some machines :-/

[JanMikes]

I will update readme with this specific situation.

[ruudk]

Let me first test it.

[ruudk]

Still doesn't work.

I think we should do this the right way, and use volumes as I suggested.

[ruudk]

And for production it should work differently.

[JanMikes]

Even after docker-compose down && docker-compose down not working?

[ruudk]

Nope.

[JanMikes]

More ideas: bin/console clear:cache

if not working ty .env.local -> HOST_DEMO_DIR="/var/www/getrector.org/var/demo" (tweak so it ends with /var/demo on your local directory)

[ruudk]

Are you able to test this on macOS yourself? Or can we maybe chat (Slack?) about this so that we can go quick back and forth?

[JanMikes]

Later today, im on a meetings now 😢

I will be happy if you reach me on http://pehapkari.slack.com slack, janmikes username.

[ruudk]

How do I join? https://pehapkari.slack.com/join/shared_invite/zt-i7fssvmv-BVx7Qdp6IE20v5g0Z2ka9w#/ not valid

[JanMikes]

https://join.slack.com/t/pehapkari/shared_invite/zt-m3c6y1ya-XvJ4dXppLmAsT_YH5hZSwQ