Improve CGI performance

jamesaoverton commented 2 years ago

There is currently about a 1 second delay when executing the simplest CGI script. When that is added to startup time for Python and Flask, the CGI pages seem annoyingly slow. It's not clear what the source of the delay is, since DROID is snappy for all the non-CGI operations.

Simple CGI script:

#!/bin/sh

echo "Content-Type: text/html"
echo ""
echo "Hello!"

Possible sources of the delay:

Clojure conch? probably not
docker exec? does seem to have a delay of about 0.5 seconds
a hardcoded sleep? we didn't see one in a quick check
local vs. non-local mode?
could just be an accumulation of slowish steps

lmcmicu commented 2 years ago

Docker is definitely one of the bigger culprits here. One thing worth noting is that prior to running a CGI script, DROID needs to check if the script needs to be rebuilt and then it needs to check if it is executable. It does this by running make -q and test -x. Although these steps are necessary, I am thinking that it is not necessary to actually run these steps in the docker container. There is a precedent for this. git status commands, for instance, are not run in containers. See the following log:

2022-07-30 11:35:54.050-0400-DEBUG Processing request in view-file with params: {:project-name "lmcmicu-for-testing-with-droid", :branch-name "lmcmicu-extra-patch-5", :view-path "build/gondor-script.sh"}
2022-07-30 11:35:54.052-0400-DEBUG Running git status --short --branch --porcelain /home/mike/Knocean/droid/projects/lmcmicu-for-testing-with-droid/workspace/lmcmicu-extra-patch-5 without a container
2022-07-30 11:35:54.097-0400-DEBUG Running make -f Makefile -q build/gondor-script.sh /home/mike/Knocean/droid/projects/lmcmicu-for-testing-with-droid/workspace/lmcmicu-extra-patch-5 in container lmcmicu-for-testing-with-droid-lmcmicu-extra-patch-5
2022-07-30 11:35:54.098-0400-DEBUG Mapping /home/mike/Knocean/droid/projects/lmcmicu-for-testing-with-droid/workspace/lmcmicu-extra-patch-5 to /workspace/ and /home/mike/Knocean/droid/projects/lmcmicu-for-testing-with-droid/temp/lmcmicu-extra-patch-5 to /tmp/droid/ in container.
2022-07-30 11:35:54.099-0400-DEBUG Mapping /home/mike/Knocean/droid/projects/lmcmicu-for-testing-with-droid/workspace/lmcmicu-extra-patch-5 to /workspace/ and /home/mike/Knocean/droid/projects/lmcmicu-for-testing-with-droid/temp/lmcmicu-extra-patch-5 to /tmp/droid/ in container.
2022-07-30 11:35:54.355-0400-DEBUG Running test -x /home/mike/Knocean/droid/projects/lmcmicu-for-testing-with-droid/workspace/lmcmicu-extra-patch-5/build/gondor-script.sh in container lmcmicu-for-testing-with-droid-lmcmicu-extra-patch-5
2022-07-30 11:35:54.356-0400-DEBUG Mapping /home/mike/Knocean/droid/projects/lmcmicu-for-testing-with-droid/workspace/lmcmicu-extra-patch-5 to /workspace/ and /home/mike/Knocean/droid/projects/lmcmicu-for-testing-with-droid/temp/lmcmicu-extra-patch-5 to /tmp/droid/ in container.
2022-07-30 11:35:54.357-0400-DEBUG Mapping /home/mike/Knocean/droid/projects/lmcmicu-for-testing-with-droid/workspace/lmcmicu-extra-patch-5 to /workspace/ and /home/mike/Knocean/droid/projects/lmcmicu-for-testing-with-droid/temp/lmcmicu-extra-patch-5 to /tmp/droid/ in container.
2022-07-30 11:35:54.565-0400-DEBUG Running sh -c ./gondor-script.sh > /home/mike/Knocean/droid/projects/lmcmicu-for-testing-with-droid/temp/lmcmicu-extra-patch-5/gondor-script.sh.out.1659195354564 /home/mike/Knocean/droid/projects/lmcmicu-for-testing-with-droid/workspace/lmcmicu-extra-patch-5/build in container lmcmicu-for-testing-with-droid-lmcmicu-extra-patch-5
2022-07-30 11:35:54.566-0400-DEBUG Mapping /home/mike/Knocean/droid/projects/lmcmicu-for-testing-with-droid/workspace/lmcmicu-extra-patch-5 to /workspace/ and /home/mike/Knocean/droid/projects/lmcmicu-for-testing-with-droid/temp/lmcmicu-extra-patch-5 to /tmp/droid/ in container.
2022-07-30 11:35:54.567-0400-DEBUG Mapping /home/mike/Knocean/droid/projects/lmcmicu-for-testing-with-droid/workspace/lmcmicu-extra-patch-5 to /workspace/ and /home/mike/Knocean/droid/projects/lmcmicu-for-testing-with-droid/temp/lmcmicu-extra-patch-5 to /tmp/droid/ in container.
2022-07-30 11:35:54.570-0400-INFO Running CGI script /home/mike/Knocean/droid/projects/lmcmicu-for-testing-with-droid/workspace/lmcmicu-extra-patch-5/build/gondor-script.sh for at most 10000 milliseconds
2022-07-30 11:35:54.814-0400-DEBUG Timeout is set: 10000
2022-07-30 11:35:54.815-0400-INFO CGI script returned no status. Assuming 200.

Notice that the git status command (second line) is run "without a container" while the make -q and test -x commands are run within the container and take much longer to complete. In the log above they are taking longer than the CGI script itself (gondor-script.sh, which simply outputs the message "hello" as in the issue description).

So what I am proposing (to begin with, and then we can see if further optimisations are needed) is to only run "real" commands in containers, like gondor-script.sh in the example above. All "overhead commands", like make -q and test -x, should be run without a container.

Note that it's possible that there is some reason (which I'm not remembering) why we are running these commands in the container, but I suspect that this is not the case. At least this is true for make -q. In the case of test -x I guess a case could be made that it is possible for the file to have execute permission only outside the container but not inside (if the effective owner is different). But this seems to me like a corner case that I doubt we would ever encounter in practice.

jamesaoverton commented 2 years ago

I like this plan!

lmcmicu commented 2 years ago

Implemented. DROID seems much snappier now.

ontodev / droid

Improve CGI performance #121