Difficulty Running saw-remote-api

[michaelabernardo]

I'm having some issues getting saw-remote-api to consistently work in a Docker image. It seems like it may have to do with the volumes.

Dockerfile used for the Python client:

FROM python:3.9.2-buster

ENV POETRY_VERSION=1.1.5
ENV POETRY_VIRTUALENVS_CREATE=false

WORKDIR /home

RUN pip install "poetry==$POETRY_VERSION"

RUN git clone https://github.com/GaloisInc/saw-script.git

WORKDIR /home/saw-script

RUN git submodule update --init

WORKDIR /home/saw-script/saw-remote-api/python

RUN poetry install

(1) For the below docker-compose.yml, running docker compose run saw-client python tests/saw/test_salsa20.py works as expected:

services:
  saw-remote-api:
    image: "ghcr.io/galoisinc/saw-remote-api:nightly"
    ports:
      - "8080:8080"
    volumes:
      - /home/user/saw-script/saw-remote-api/python/tests/saw/test-files:/home/saw/tests/saw/test-files

  saw-client:
    build: .
    environment:
      SAW_SERVER_URL: "http://saw-remote-api:8080"

(2) For the below docker-compose.yml, running docker compose run saw-client python tests/saw/test_salsa20.py hangs:

services:
  saw-remote-api:
    image: "ghcr.io/galoisinc/saw-remote-api:nightly"
    ports:
      - "8080:8080"
    volumes:
      - /home/user/saw-script/saw-remote-api/python/tests/saw/test-files:/home/saw

  saw-client:
    build: .
    environment:
      SAW_SERVER_URL: "http://saw-remote-api:8080"

(Note: this same set-up used instead for cryptol-remote-api works just fine)

(3) For the below docker-compose.yml, running docker compose run saw-client python tests/saw/test_salsa20.py gets an error:

services:
  saw-remote-api:
    image: "ghcr.io/galoisinc/saw-remote-api:nightly"
    ports:
      - "8080:8080"
    volumes:
      - /home/user/saw-script/saw-remote-api/python/tests/saw/test-files:/home/saw/test-files

  saw-client:
    build: .
    environment:
      SAW_SERVER_URL: "http://saw-remote-api:8080"

Error:

======================================================================
ERROR: test_salsa20 (__main__.Salsa20EasyTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/saw-script/saw-remote-api/python/tests/saw/test_salsa20.py", line 133, in test_salsa20
    cryptol_load_file(cryname)
  File "/home/saw-script/saw-remote-api/python/saw_client/__init__.py", line 371, in cryptol_load_file
    __get_designated_connection().cryptol_load_file(filename).result()
  File "/usr/local/lib/python3.9/site-packages/argo_client/interaction.py", line 178, in result
    return self.process_result(self._result_and_state_and_out_err()[0])
  File "/usr/local/lib/python3.9/site-packages/argo_client/interaction.py", line 167, in _result_and_state_and_out_err
    raise self.process_error(exception)
saw_client.exceptions.CryptolError: error: user error (Cryptol error:
[error] can't find file: Salsa20.cry)

----------------------------------------------------------------------
Ran 1 test in 1.149s

FAILED (errors=1)
🔶 No goal results logged.

This causes issues, as well, when I try to run saw-remote-api as a service in GitHub Actions.

The below works fine for cryptol-remote-api:

    services:
      cryptol-remote-api:
        image: ghcr.io/galoisinc/cryptol-remote-api:nightly
        ports:
          - 8080:8080
        options: -v ${{ github.workspace }}:/home/cryptol
        env:
          CRYPTOL_SERVER_URL: http://0.0.0.0:8080

But the same configuration for saw-remote-api hangs, as in (2):

    services:
      saw-remote-api:
        image: ghcr.io/galoisinc/saw-remote-api:nightly
        ports:
          - 8080:8080
        options: -v ${{ github.workspace }}:/home/saw
        env:
          SAW_SERVER_URL: http://0.0.0.0:8080

[RyanGlScott]

I haven't had a chance to look into this deeply yet, but a couple of things I'm wondering about at a quick glance:

• Is there a reason that you use docker compose run saw-client python ... instead of docker compose run saw-client poetry run python ...? Intuitively, I'd expect this only to work if you use python in tandem with poetry run (perhaps there is more to this docker-compose setup than what I'm seeing, however).

• I'm not sure that I understand what you're trying to do with setup (2). If I'm reading the docker-compose setup correctly, you're trying to mount /home/user/saw-script/saw-remote-api/python/tests/saw/test-files to /home/saw (where /home/saw is the entrypoint to the saw-remote-api image), but then you're trying to run python tests/saw/test_salsa20.py within the image. This doesn't make sense to me, since test_salsa20.py would be located at /home/saw/test_salsa20.py due to the way you've set up your volumes. Wouldn't it make more sense to run python test_salsa.py in the image?

  Setup (3) seems to have a similar issue, where you are mounting the test directory to a different location but still invoking the same python tests/saw/test_salsa20.py command within the image, which no longer points to test_salsa20.py's location. I'm not sure if this explains the mysterious hang or not, but it's possible that it's related.

[michaelabernardo]

• Oops you're right, I didn't notice that! I pulled this together quickly for an example, but I've noticed the behavior no matter how I install the client (pip install saw-client, or cloning the repo and setting the appropriate PYTHONPATH)
• Yeah sorry I wasn't very clear. What I'm really trying to do is run a CI/CD pipeline where saw-remote-api is a service, and I can run several Python SAW scripts from across the entire repository. An example of how I have this working with cryptol-remote-api is here. When I try to do the same thing with saw-remote-api, it doesn't work (usually getting one of the errors in my original message). I'm probably just not using the volumes correctly like you said.

[RyanGlScott]

I'm not sure if you are using volumes correctly or not—I think it's quite possible that your bind mounts could be made to work. I do think it would be worth retrying these examples with different arguments to Python to see if changing the location of test_salsa20.py makes a difference.

[michaelabernardo]

In my CI/CD pipeline, I was able to get it working by doing -v ${{ github.workspace }}:${{ github.workspace }}.

I had been trying to do something like the recommended -v PATH_TO_FILES:/home/saw/files/ from the README, but I still haven't been able to get that to work universally. It's possible I'm making some other error, but doing it the other way works for me. Thanks for the help!