I have checked the following

This issue is about the Docker image, so the outputs shown below were run within a Securify container built by following the README.

[x] The souffle binary is available, output when running souffle:

============================================================================
souffle -- A datalog engine.
Usage: souffle [OPTION] FILE.
----------------------------------------------------------------------------
Options:
        -F<DIR>                 --fact-dir=<DIR>                        Specify directory for fact files.
        -I<DIR>                 --include-dir=<DIR>                     Specify directory for include files.
        -D<DIR>                 --output-dir=<DIR>                      Specify directory for output files (if <DIR> is -, stdout is used).
        -j<N>                   --jobs=<N>                              Run interpreter/compiler in parallel using N threads, N=auto for system default.
        -c                      --compile                               Generate C++ source code, compile to a binary executable, then run this executable.
        -g<FILE>                --generate=<FILE>                       Generate C++ source code for the given Datalog program and write it to <FILE>.
        -w                      --no-warn                               Disable warnings.
        -m<RELATIONS>           --magic-transform=<RELATIONS>           Enable magic set transformation changes on the given relations, use '*' for all.
        -o<FILE>                --dl-program=<FILE>                     Generate C++ source code, written to <FILE>, andcompile this to a binary executable (without executing it).
        -l                      --live-profile                          Enable live profiling.
        -p<FILE>                --profile=<FILE>                        Enable profiling, and write profile data to <FILE>.
        -r<FILE>                --debug-report=<FILE>                   Write HTML debug report to <FILE>.
        -t<EXPLAIN>             --provenance=<EXPLAIN>                  Enable provenance information via guided SLD.
        -d<type>                --data-structure=<type>                 Specify data structure (brie/btree/eqrel/rbtset/hashset).
        -e<[ file | mpi ]>      --engine=<[ file | mpi ]>               Specify communication engine for distributed execution.
                                --hostfile=<FILE>                       Specify --hostfile option for call to mpiexec when using mpi as execution engine.
        -v                      --verbose                               Verbose output.
        -h                      --help                                  Display this help message.
----------------------------------------------------------------------------
Version: 1.4.0
----------------------------------------------------------------------------
Copyright (c) 2016-18 The Souffle Developers.
Copyright (c) 2013-16 Oracle and/or its affiliates.
All rights reserved.
============================================================================

[x] Output of solc --version

solc, the solidity compiler commandline interface
Version: 0.4.24+commit.e67f0147.Linux.g++

[x] Output of solc --bin-runtime MyContract.sol

solc --bin-runtime project/example.sol outputs:

project/example.sol:5:9: Warning: Failure condition of 'send' ignored. Consider using 'transfer' instead.
        msg.sender.send(x);
        ^----------------^
project/example.sol:11:9: Warning: Failure condition of 'send' ignored. Consider using 'transfer' instead.
        msg.sender.send(y);
        ^----------------^
project/example.sol:15:18: Warning: Using contract member "balance" inherited from the address type is deprecated. Convert the contract to "address" type to access the member, for example use "address(contract).balance" instead.
        uint x = this.balance;
                 ^----------^
project/example.sol:16:9: Warning: Failure condition of 'send' ignored. Consider using 'transfer' instead.
        msg.sender.send(x);
        ^----------------^
project/example.sol:3:5: Warning: No visibility specified. Defaulting to "public".
    function safeTransfer1() {
    ^ (Relevant source part starts here and spans across multiple lines).
project/example.sol:4:18: Warning: "msg.value" used in non-payable function. Do you want to add the "payable" modifier to this function?
        uint x = msg.value;
                 ^-------^
project/example.sol:8:5: Warning: No visibility specified. Defaulting to "public".
    function safeTransfer2() {
    ^ (Relevant source part starts here and spans across multiple lines).
project/example.sol:14:5: Warning: No visibility specified. Defaulting to "public".
    function unsafeTransfer() {
    ^ (Relevant source part starts here and spans across multiple lines).

======= project/example.sol:MarketPlace =======
Binary of the runtime part:
608060405260043610610057576000357c0100000000000000000000000000000000000000000000000000000000900463ffffffff1680633155194c1461005c578063bc5a6c2014610073578063c7961f2a1461008a575b600080fd5b34801561006857600080fd5b506100716100a1565b005b34801561007f57600080fd5b506100886100df565b005b34801561009657600080fd5b5061009f610134565b005b60003490503373ffffffffffffffffffffffffffffffffffffffff166108fc829081150290604051600060405180830381858888f193505050505050565b60003073ffffffffffffffffffffffffffffffffffffffff163190503373ffffffffffffffffffffffffffffffffffffffff166108fc829081150290604051600060405180830381858888f193505050505050565b600080606491506064820290503373ffffffffffffffffffffffffffffffffffffffff166108fc829081150290604051600060405180830381858888f193505050505050505600a165627a7a72305820d909d9e9daefa07cd162b106f126c93011e07a987d21e0f19b567a6e1cadf06b0029

[x] I can reproduce this issue in a Docker container with the following command

N/A since this is in a Docker container

Steps to reproduce

Clone the repository
Follow the steps in the README to build and run the Docker image

Upon running docker run securify I receive the following output:

Traceback (most recent call last):
  File "/usr/lib/python3.6/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/usr/lib/python3.6/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/sec/scripts/pysolc.py", line 161, in <module>
    res = compile_project(sys.argv[1])
  File "/sec/scripts/pysolc.py", line 136, in compile_project
    return compile_solfiles(sources, path)
  File "/sec/scripts/pysolc.py", line 126, in compile_solfiles
    stdoutdata, _, _, _ = solc_wrapper(**compiler_kwargs)
  File "/root/.local/lib/python3.6/site-packages/solc/utils/string.py", line 85, in inner
    return force_obj_to_text(fn(*args, **kwargs))
  File "/root/.local/lib/python3.6/site-packages/solc/wrapper.py", line 159, in solc_wrapper
    stderr=subprocess.PIPE)
  File "/usr/lib/python3.6/subprocess.py", line 709, in __init__
    restore_signals, start_new_session)
  File "/usr/lib/python3.6/subprocess.py", line 1344, in _execute_child
    raise child_exception_type(errno_num, err_msg, err_filename)
FileNotFoundError: [Errno 2] No such file or directory: '/root/.py-solc/solc-v0.4.24/bin/solc':'/root/.py-solc/solc-v0.4.24/bin/solc'

Proposed solution

Assuming that the path it is looking for is the desired path of solc, I updated the # install solc part of Dockerfile to this:

# install solc
RUN mkdir -p ~/.py-solc/solc-v0.4.24/bin/

RUN wget https://github.com/ethereum/solidity/releases/download/v0.4.24/solc-static-linux -O ~/.py-solc/solc-v0.4.24/bin/solc &&\
  chmod u+x ~/.py-solc/solc-v0.4.24/bin/solc

After building the image I can now successfully run the demo using docker run securify. However, using docker run -v $(pwd)/contracts:/project securify to run Securify on my own files fails with the message Killed.

Google tells me this problem is due to the max size of the Java heap memory, which is set to 16GB in docker_run_securify. My machine only has 8GB of memory, and Docker is set to use a max of 4GB, so in docker_run_securify I changed:

securify_cmd="java -Xmx16G -jar /securify_jar/securify.jar -co"

securify_cmd="java -Xmx4G -jar /securify_jar/securify.jar -co"

Everything now seems to work properly. I could be missing something, but it seems Dockerfile and docker_run_securify should be updated with the above changes. Perhaps the max heap memory size can be optionally passed in as an input when running the container. I can submit a pull request with these changes if you'd like.

I believe the sloc version is meant to be installed based on the input solidity files. Therefore hardcoding a version into Dockerfile will not fix the issue if a contract with a non compatible solidity pragma is given as input.

Right now the automatic install of sloc is handled by scripts/install_sloc.py which installs the latest version (currently 0.4.25) which is too high for the example contract (pragma solidity 0.4.24;). I would rather propose to change the install script to add:

if not os.path.exists(binary):
    install_solc(f'v{solc_version}', platform='linux')

after the binary path is set. This should then install the correct sloc version based on the contracts provided if no install already exists.

Of course, this does not solve the Killed message.

Good point about scripts/install_solc.py. I had overlooked that, and just assumed it was using the hardcoded solc version installed in lines 34-36 of the original Dockerfile, shown here:

https://github.com/eth-sri/securify/blob/13d4784901e5e2d179ef504a30b9af1abe355451/Dockerfile#L34-L36

Now, though, why would we need the above lines in Dockerfile if solc is later installed using scripts/install_solc.py? I think those lines can be removed.

If I understood your proposal correctly, I believe you are suggesting to add those lines into pysolc.py, right after the binary path is determined on line 116, i.e. like this:

def compile_solfiles(files, proj_dir, solc_version=None, output_values=OUTPUT_VALUES):

    # original function code was here...

    if solc_version is None:
        solc_version = min(map(parse_version, files),
                           key=lambda x: _version_to_tuple(x))

    binary = os.path.join(Path.home(), f'.py-solc/solc-v{solc_version}/bin/solc')

    # proposing to add these two lines
    if not os.path.exists(binary):
        install_solc(f'v{solc_version}', platform='linux')

    # ...continue original function code

This seems like a good solution, since it allows the image to always use the proper solc version without rebuilding the image. It also means you can remove scripts/install_solc.py and the corresponding line in Dockerfile, since solc is now installed on each run.

This combination of changes (removing those Dockerfile lines and installing the proper solc version with each run) seem to work well in my brief testing.

A downside here is that by removing the above Dockerfile lines the image cannot find solc simply by running solc --version

Yes, that is where I would place the binary existence check.

I agree that the image not finding sloc simply by running sloc --version is a disadvantage, but considering the sloc executable (and version) provided by this is not the one used to compile the source code, I would actually say it is better to not have at all (it can create confusion). As to why this install is explicit in the Dockerfile in the first place, I have no idea.

Regarding removing the scripts/install_sloc.py altogether, I would argue that installing the latest version of sloc during the building of the docker image is not really problematic and caches at least one version of sloc in the image. This would allow faster analysis on most contracts with non-strict pragmas as the latest version of solc can be used. If this is removed, the existence check of the binary

if not os.path.join(binary):

is useless as no version of sloc will ever be cached between docker container runs. Therefore you would indeed need a fresh install of sloc on each run, which I believe not to be ideal.

I don't know much about Dockerfiles or docker in general, but maybe a smart solution would be to let the user decide on which additional versions of sloc he wants to have preinstalled in the docker image, hence not requiring an explicit install each run. I don't know if this is possible by just passing arguments to the Dockerfile? I think that would be a good (at least temporary) solution because the user should know which solidity versions he is most likely to use.

I'm no Docker expert either, but my understanding of the typical Docker use case makes caching a user-chosen solc between runs not very feasible.

To elaborate, a typical use case would be:

# Step 1: Pull an already built Securify image from https://hub.docker.com/ 
#   This image will either have a chosen version of `solc` pre-installed, or no
#   version at all, depending on the decision made 
docker pull eth-sri/securify

# Step 2: Run Securify on your contracts
docker run -v $(pwd)/folder_with_solidity_files:/project eth-sri/securify

Note on step 1: Currently there is no image for this project on Docker Hub, but it looks like it'll happen eventually in #3.

The command in step 2 simply runs docker_run_securify within the container. This would fetch the appropriate version of solc, if necessary, and then exits the container. A newly installed version of solc does not persist between runs of this command, as containers do not have persistent storage by default. On a related note, with additional changes, you could allow the user to pass in a solc version using an environment variable like this:

docker run -v $(pwd)/folder_with_solidity_files:/project -e SOLC_VERSION=0.4.23 eth-sri/securify

Then you could check for the existence of the SOLC_VERSION environment variable, and if it exists, download that version. But once the container exits, re-running the above command generates a new container and solc must again be installed, even if using the same non-default version as the last call.

Personally, I think it's ok even if no solc versions are installed by default, since the install is pretty quick. Including the latest solc version with the image certainly can't hurt though.

To ensure users know what solc version is being used, you could either (1) force users to specify versions with the environment variable, and/or (2) include the solc version used in the JSON output

Oh yes never mind I did not consider that the image would be pulled from Docker Hub instead of installed by the user directly. So I guess the pre-install on the image is not really necessary / very useful.

The problem was that we only installed the latest version of solc, so the version will change whenever there is a new release. We now install all versions, it means that the docker image is slower to build (which is why I wanted to avoid this), but it will do for now.

eth-sri / securify

Docker image does not run #31

I have checked the following

Steps to reproduce

Proposed solution