InsufficientDataBytes: Tried to read 32 bytes. Only got 0 bytes (This One Appears To Be Different)

ethereum / web3.py

A python interface for interacting with the Ethereum blockchain and ecosystem.

http://web3py.readthedocs.io

MIT License

5.02k stars 1.71k forks source link

InsufficientDataBytes: Tried to read 32 bytes. Only got 0 bytes (This One Appears To Be Different) #1257

Closed sam-ulrich1 closed 5 years ago

sam-ulrich1 commented 5 years ago

Web3 Version: 4.8.2 Solidity: 0.4.18, 0.5.3 (Both Installed Via Py-Solc) Geth Version: 1.7 (Built From Source), 1.8.23 (Binary), Master (as of 2/23/19) (Built From Source)

Python: 3.6
OS: Ubuntu 18.04
pip freeze output

aenum==2.1.2
asn1crypto==0.24.0
astor==0.7.1
attrdict==2.0.0
attrs==18.2.0
Automat==0.7.0
autopep8==1.4.3
base58==1.0.3
bitcoin==1.1.42
blessings==1.7
bpython==0.17.1
bumpversion==0.5.3
certifi==2018.11.29
cffi==1.11.5
chardet==3.0.4
Click==7.0
colorlog==4.0.2
constantly==15.1.0
coverage==4.5.2
coveralls==1.5.1
coz-bytecode==0.5.1
cryptography==2.4.2
curtsies==0.3.0
cycler==0.10.0
cytoolz==0.9.0.1
docopt==0.6.2
ecdsa==0.13
eth-abi==1.3.0 (I just updated this from 1.2.2 in hopes of the issue being solved)
eth-account==0.3.0
eth-hash==0.2.0
eth-keyfile==0.5.1
eth-keys==0.2.0b3
eth-rlp==0.1.2
eth-typing==2.0.0
eth-utils==1.3.0
Events==0.3
Flask==1.0.2
Flask-Cors==3.0.7
Flask-Limiter==1.0.1
Flask-REST==1.3
furl==2.0.0
gevent==1.4.0
greenlet==0.4.15
hexbytes==0.1.0
hyperlink==18.0.0
idna==2.8
incremental==17.5.0
inflection==0.2.0
itsdangerous==1.1.0
Jinja2==2.10
klein==17.10.0
limits==1.3
logzero==1.5.0
lru-dict==1.1.6
MarkupSafe==1.1.0
memory-profiler==0.55.0
mmh3==2.5.1
mock==2.0.0
Mongothon==0.8.0
mpmath==1.1.0
neo-boa==0.5.6
-e git+https://github.com/CityOfZion/neo-python.git@fe90f62e123d720d4281c79af0598d9df9e776fb#egg=neo_python
neo-python-rpc==0.2.1
neocore==0.5.6
orderedmultidict==1.0
parse==1.9.0
parsimonious==0.8.1
pbr==5.1.1
peewee==3.8.1
pexpect==4.6.0
pluggy==0.8.1
plyvel==1.0.5
prompt-toolkit==2.0.7
psutil==5.4.8
ptyprocess==0.6.0
py==1.7.0
py-solc==2.1.0
pycodestyle==2.4.0
pycparser==2.19
pycryptodome==3.7.2
Pygments==2.3.1
PyHamcrest==1.9.0
pymitter==0.2.3
pymongo==3.7.2
Pympler==0.6
pyparsing==2.3.1
python-dateutil==2.7.5
pytz==2018.9
requests==2.21.0
rlp==1.0.3
Schemer==0.2.11
scrypt==0.8.6
semantic-version==2.6.0
six==1.12.0
toolz==0.9.0
tqdm==4.29.1
Twisted==18.9.0
typing==3.6.6
urllib3==1.24.1
uuid==1.30
virtualenv==16.2.0
wcwidth==0.1.7
web3==4.8.2
websockets==6.0
Werkzeug==0.14.1
zope.interface==4.6.0

What was wrong?

I have had this specific contract deployed on Ropsten for just over a month but the exact same code has been deployed and used many times (at least 4 different successful deployments). About 4 days ago seemingly out of nowhere the error "InsufficientDataBytes: Tried to read 32 bytes. Only got 0 bytes" arose. The contract had been used for about 26 days consecutively. There was literally no change in the code base between the contract communication working and it no longer working. I have not found ANY reason as to why this happened.

I wanted to make sure this was not somehow a delayed result of a faulty smart contract so I have confirmed that the exact same issue happens with the example code below. Because I received the exact same issue with the example code I chose to show the example code for simplicity:

import json
import web3

from web3 import Web3
from solc import compile_source
from web3.contract import ConciseContract

contract_source_code = '''
pragma solidity ^0.4.18;

contract Greeter {
    string public greeting;

    function Greeter() public {
        greeting = 'Hello';
    }

    function setGreeting(string _greeting) public {
        greeting = _greeting;
    }

    function greet() view public returns (string) {
        return greeting;
    }
}
'''

compiled_sol = compile_source(contract_source_code)
contract_interface = compiled_sol['<stdin>:Greeter']

w3 = Web3(Web3.HTTPProvider('http://localhost:8545'))

w3.eth.defaultAccount = w3.eth.accounts[0]

w3.personal.unlockAccount(w3.personal.listAccounts[0], "password")

print(w3.eth.defaultAccount)

Greeter = w3.eth.contract(abi=contract_interface['abi'], bytecode=contract_interface['bin'])

tx_hash = Greeter.constructor().transact()

tx_receipt = w3.eth.waitForTransactionReceipt(tx_hash)

greeter = w3.eth.contract(
    address=tx_receipt.contractAddress,
    abi=contract_interface['abi'],
)

print('Default contract greeting: {}'.format(
    greeter.functions.greet().call()
))

print('Setting the greeting to Nihao...')
tx_hash = greeter.functions.setGreeting('Nihao').transact()

w3.eth.waitForTransactionReceipt(tx_hash)

print('Updated contract greeting: {}'.format(
    greeter.functions.greet().call()
))

reader = ConciseContract(greeter)
assert reader.greet() == "Nihao"

Here is the traceback:

Traceback (most recent call last):
Traceback (most recent call last):
  File "ENVPATH/lib/python3.6/site-packages/web3/contract.py", line 1374, in call_contract_function
    output_data = decode_abi(output_types, return_data)
  File "ENVPATH/lib/python3.6/site-packages/eth_abi/abi.py", line 96, in decode_abi
    return decoder(stream)
  File "ENVPATH/lib/python3.6/site-packages/eth_abi/decoding.py", line 118, in __call__
    return self.decode(stream)
  File "ENVPATH/lib/python3.6/site-packages/eth_utils/functional.py", line 46, in inner
    return callback(fn(*args, **kwargs))
  File "ENVPATH/lib/python3.6/site-packages/eth_abi/decoding.py", line 164, in decode
    yield decoder(stream)
  File "ENVPATH/lib/python3.6/site-packages/eth_abi/decoding.py", line 118, in __call__
    return self.decode(stream)
  File "ENVPATH/lib/python3.6/site-packages/eth_abi/decoding.py", line 133, in decode
    start_pos = decode_uint_256(stream)
  File "ENVPATH/lib/python3.6/site-packages/eth_abi/decoding.py", line 118, in __call__
    return self.decode(stream)
  File "ENVPATH/lib/python3.6/site-packages/eth_abi/decoding.py", line 186, in decode
    raw_data = self.read_data_from_stream(stream)
  File "ENVPATH/lib/python3.6/site-packages/eth_abi/decoding.py", line 296, in read_data_from_stream
    len(data),
eth_abi.exceptions.InsufficientDataBytes: Tried to read 32 bytes.  Only got 0 bytes

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "PROJECTPATH/testContract.py", line 67, in <module>
    greeter.functions.greet().call()
  File "ENVPATH/lib/python3.6/site-packages/web3/contract.py", line 1115, in call
    **self.kwargs
  File "ENVPATH/lib/python3.6/site-packages/web3/contract.py", line 1396, in call_contract_function
    raise BadFunctionCallOutput(msg) from e
web3.exceptions.BadFunctionCallOutput: Could not decode contract function call greet return data b'' for output_types ['string']

How can it be fixed?

I have no idea at the moment.

pipermerriam commented 5 years ago

after deployment but before interacting with the contract, please check the value of w3.eth.getCode(greeter.address) and verify that the output isn't empty, aka 0x. If this is the case then you are interacting with an empty contract (I.E. your deployment failed)

You can also check the value of tx_receipt['status'] for the receipt from the contract deployment. I don't recall if we normalize that to True/False or if it remains the canincal 0x1/0x0. If the value is False or 0x0 then the deploy transaction failed.

sam-ulrich1 commented 5 years ago

First of all, thank you for the time and help!

The following contract details are on Ropsten.

Here is the contract address: 0x6D1E498F5F94072E83313b2b096d57Bafe9ee617

Here is the deployment transaction: 0xf5f7f5ac54ce4c26d02a9b29bc816d45d4997b32ccec31abf14f0a9b7fee0948

Here is the ABI: [{"constant": false, "inputs": [{"name": "_greeting", "type": "string"}], "name": "setGreeting", "outputs": [], "payable": false, "stateMutability": "nonpayable", "type": "function"}, {"constant": true, "inputs": [], "name": "greet", "outputs": [{"name": "", "type": "string"}], "payable": false, "stateMutability": "view", "type": "function"}, {"constant": true, "inputs": [], "name": "greeting", "outputs": [{"name": "", "type": "string"}], "payable": false, "stateMutability": "view", "type": "function"}, {"inputs": [], "payable": false, "stateMutability": "nonpayable", "type": "constructor"}]

This was the result of w3.eth.getCode(greeter.address):

b"\x80@R4\x80\x15a\x00\x10W\x00\x80\xfd[P\x046\x10a\x00^W\x005|\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x90\x04\x80c\xa4\x13hb\x14a\x00cW\x80c\xcf\xae2\x17\x14a\x01\x1eW\x80c\xefi\x0c\xc0\x14a\x01\xa1W[\x00\x80\xfd[a\x01\x1c\x04\x806\x03 \x81\x10\x15a\x00yW\x00\x80\xfd[\x81\x01\x90\x80\x805\x90 \x01\x90d\x01\x00\x00\x00\x00\x81\x11\x15a\x00\x96W\x00\x80\xfd[\x82\x01\x83 \x82\x01\x11\x15a\x00\xa8W\x00\x80\xfd[\x805\x90 \x01\x91\x84\x01\x83\x02\x84\x01\x11d\x01\x00\x00\x00\x00\x83\x11\x17\x15a\x00\xcaW\x00\x80\xfd[\x91\x90\x80\x80\x1f\x01 \x80\x91\x04\x02\x01@Q\x90\x81\x01@R\x80\x93\x92\x91\x90\x81\x81R \x01\x83\x83\x80\x82\x847\x00\x81\x84\x01R\x1f\x19\x1f\x82\x01\x16\x90P\x80\x83\x01\x92PPPPPPP\x91\x92\x91\x92\x90PPPa\x02$V[\x00[a\x01&a\x02>V[@Q\x80\x80\x01\x82\x81\x03\x82R\x83\x81\x81Q\x81R \x01\x91P\x80Q\x90\x01\x90\x80\x83\x83\x00[\x83\x81\x10\x15a\x01fW\x80\x82\x01Q\x81\x84\x01R\x81\x01\x90Pa\x01KV[PPPP\x90P\x90\x81\x01\x90\x1f\x16\x80\x15a\x01\x93W\x80\x82\x03\x80Q\x01\x83 \x03a\x01\x00\n\x03\x19\x16\x81R\x01\x91P[P\x92PPP@Q\x80\x91\x03\x90\xf3[a\x01\xa9a\x02\xe0V[@Q\x80\x80 \x01\x82\x81\x03\x82R\x83\x81\x81Q\x81R\x01\x91P\x80Q\x90 \x01\x90\x80\x83\x83\x00[\x83\x81\x10\x15a\x01\xe9W\x80\x82\x01Q\x81\x84\x01R \x81\x01\x90Pa\x01\xceV[PPPP\x90P\x90\x81\x01\x90\x1f\x16\x80\x15a\x02\x16W\x80\x82\x03\x80Q\x01\x83\x03a\x01\x00\n\x03\x19\x16\x81R \x01\x91P[P\x92PPP@Q\x80\x91\x03\x90\xf3[\x80\x00\x90\x80Q\x90\x01\x90a\x02:\x92\x91\x90a\x03~V[PPV[```\x00\x80T\x01\x81\x01\x16\x15a\x01\x00\x02\x03\x16\x02\x90\x04\x80\x1f\x01 \x80\x91\x04\x02\x01@Q\x90\x81\x01@R\x80\x92\x91\x90\x81\x81R \x01\x82\x80T\x01\x81\x01\x16\x15a\x01\x00\x02\x03\x16\x02\x90\x04\x80\x15a\x02\xd6W\x80\x1f\x10a\x02\xabWa\x01\x00\x80\x83T\x04\x02\x83R\x91\x01\x91a\x02\xd6V[\x82\x01\x91\x90\x00R`\x00 \x90[\x81T\x81R\x90\x01\x01\x90\x01\x80\x83\x11a\x02\xb9W\x82\x90\x03\x1f\x16\x82\x01\x91[PPPPP\x90P\x90V[\x00\x80T\x01\x81\x01\x16\x15a\x01\x00\x02\x03\x16\x02\x90\x04\x80\x1f\x01 \x80\x91\x04\x02\x01@Q\x90\x81\x01@R\x80\x92\x91\x90\x81\x81R \x01\x82\x80T\x01\x81\x01\x16\x15a\x01\x00\x02\x03\x16\x02\x90\x04\x80\x15a\x03vW\x80\x1f\x10a\x03KWa\x01\x00\x80\x83T\x04\x02\x83R\x91\x01\x91a\x03vV[\x82\x01\x91\x90\x00R`\x00 \x90[\x81T\x81R\x90\x01\x01\x90\x01\x80\x83\x11a\x03YW\x82\x90\x03\x1f\x16\x82\x01\x91[PPPPP\x81V[\x82\x80T\x01\x81\x01\x16\x15a\x01\x00\x02\x03\x16\x02\x90\x04\x90\x00R`\x00 \x90\x1f\x01\x90\x04\x81\x01\x92\x82\x1f\x10a\x03\xbfW\x80Q\xff\x19\x16\x83\x80\x01\x17\x85Ua\x03\xedV[\x82\x80\x01\x01\x01\x85U\x82\x15a\x03\xedW\x91\x82\x01[\x82\x81\x11\x15a\x03\xecW\x82Q\x82U\x91\x01\x91\x90\x01\x01\x90a\x03\xd1V[[P\x90Pa\x03\xfa\x91\x90a\x03\xfeV[P\x90V[a\x04 \x91\x90[\x80\x82\x11\x15a\x04\x1cW\x00\x81\x00\x90UP`\x01\x01a\x04\x04V[P\x90V[\x90V\xfe\xa1ebzzr0X \x11\xc53\xe6v\x9a\x9e\xdb\xa6\xb5~e\x95C5\xed\xbd\x1e\x81\xce\x15\x03\xec\x05\x17-\xdc&'\xc9\xc8p\x00)"

tx_recipet['status'] returned: 1 I assume that that is the equivalent of True.

davesque commented 5 years ago

This isn't really a solution, but I tried to duplicate the problem with the same deployment script and package versions on ropsten and wasn't having any issues. So I'm led to believe that there was some issue with @sam-ulrich1 's particular deployment of that contract. But I don't know for sure.

sam-ulrich1 commented 5 years ago

@davesque Thanks! I know that doesn't solve my issue but it helps me narrow the problem down. After hearing that I'll start by purging all of my web3 related modules and reinstall from scratch. If that doesn't work I'll wipe geth, solidity, and my chain and reinstall from scratch. If none of that works I'll update here with more information.

sam-ulrich1 commented 5 years ago

I wiped all Ethereum based modules from my venv and reinstalled them. That did not solve the issue. I then wiped geth completely off my computer and built from source (once master and then again with 1.8). This produced an odd issue where contract deployments would show in my geth console but would not be relayed to the network. I have now wiped all the chain data from my computer and am resyncing in fast mode. I will update after the chain has synced and I can fully test this issue again.

sam-ulrich1 commented 5 years ago

Okay, I have traced back some weird behavior. I tripled the calls to the contract in contract.py and I found that the contract fails between 1 and 4 times but it always works after it has been called at a maximum 4 times in a row.

Update: I have found that if I sleep my program for 5-10 seconds after the Web3 Contract instance has been established the first call consistently fails but the following two calls succeed every time.

I think this could be a Web3.py issue because the failure on the first call and success in the following two calls is consistent as long as I sleep for 5 or more seconds.

P.S. I can't figure out how to format the code correctly

` def call_contract_function( web3, address, normalizers, function_identifier, transaction, block_id=None, contract_abi=None, fn_abi=None, *args, **kwargs): """ Helper function for interacting with a contract function using the 'eth_call' API. """ call_transaction = prepare_transaction( address, web3, fn_identifier=function_identifier, contract_abi=contract_abi, fn_abi=fn_abi, transaction=transaction, fn_args=args, fn_kwargs=kwargs, )

if block_id is None:
    print("block id none")
    return_data = web3.eth.call(call_transaction)
    print(return_data)
    return_data = web3.eth.call(call_transaction)
    print(return_data)
    return_data = web3.eth.call(call_transaction)
    print(return_data)
else:
    return_data = web3.eth.call(call_transaction, block_identifier=block_id)
    print(return_data)
    return_data = web3.eth.call(call_transaction, block_identifier=block_id)
    print(return_data)
    return_data = web3.eth.call(call_transaction, block_identifier=block_id)
    print(return_data)

if fn_abi is None:
    fn_abi = find_matching_fn_abi(contract_abi, function_identifier, args, kwargs)

output_types = get_abi_output_types(fn_abi)

try:
    output_data = decode_abi(output_types, return_data)
except DecodingError as e:
    # Provide a more helpful error message than the one provided by
    # eth-abi-utils
    is_missing_code_error = (
        return_data in ACCEPTABLE_EMPTY_STRINGS and
        web3.eth.getCode(address) in ACCEPTABLE_EMPTY_STRINGS
    )
    if is_missing_code_error:
        msg = (
            "Could not transact with/call contract function, is contract "
            "deployed correctly and chain synced?"
        )
    else:
        msg = (
            "Could not decode contract function call {} return data {} for "
            "output_types {}".format(
                function_identifier,
                return_data,
                output_types
            )
        )
    raise BadFunctionCallOutput(msg) from e

_normalizers = itertools.chain(
    BASE_RETURN_NORMALIZERS,
    normalizers,
)
normalized_data = map_abi_data(_normalizers, output_types, output_data)

if len(normalized_data) == 1:
    return normalized_data[0]
else:
    return normalized_data

sam-ulrich1 commented 5 years ago

I have found this for a temporary solution but It's obviously not suitable for the main net. This is from call_contract_function in contract.py

if block_id is None: return_data = web3.eth.call(call_transaction) while return_data == bytes("".encode()): return_data = web3.eth.call(call_transaction) else: return_data = web3.eth.call(call_transaction, block_identifier=block_id) while return_data == bytes("".encode()): return_data = web3.eth.call(call_transaction, block_identifier=block_id)

I thought this was a Web3 issue but it doesn't appear to be. I followed the call from the origin all the way to the requests module making the POST request. I found that successful and failed calls were identical. The only thing that I can imagine at this point is that it is a Geth issue. However, that doesn't sit right with me because the same issue occurs across multiple Geth versions. Is there anything you guys can think of that would cause such an odd issue?

anzhaozhong commented 5 years ago

i meet the problems too, and i find that occurred when the node concurrent.

dianalo commented 5 years ago

It probably doesn't help, but in my own setup (private chain, geth miner, web3.py contract deployment/calling code), I'm getting the same error, when calling a contract function with a return statement in it.

adamnmcc commented 5 years ago

i have had a similar issue when trying to run some code but managed to avert the issue when using direct IPC connection? i have a full Geth node running in docker, so i don't know if that is one contributing factor.

picomancer commented 5 years ago

I have some speculation about the cause of this problem. Solidity 0.5.x now requires the memory keyword for string return values. Is it possible that Solidity is putting the return value somewhere on the heap, but web3 or geth or whatever is expecting it on the stack? Anyone familiar with the EVM want to chime in about this possibility?

At any rate, I do have a minimal reproduction case. Here's short code that reliably triggers the problem for me:

#!/usr/bin/env python3

import argparse
import json
import sys
import time

from web3 import Web3

# This is the output of solc --combined-json abi,bin greeter.sol

greeter_json = json.loads(r"""
{"contracts":{"greeter.sol:Greeter":{"abi":"[{\"constant\":false,\"inputs\":[{\"name\":\"_greeting\",\"type\":\"string\"}],\"name\":\"setGreeting\",\"outputs\":[],\"payable\":false,\"stateMutability\":\"nonpayable\",\"type\":\"function\"},{\"constant\":true,\"inputs\":[],\"name\":\"greet\",\"outputs\":[{\"name\":\"\",\"type\":\"string\"}],\"payable\":false,\"stateMutability\":\"view\",\"type\":\"function\"},{\"constant\":true,\"inputs\":[],\"name\":\"greeting\",\"outputs\":[{\"name\":\"\",\"type\":\"string\"}],\"payable\":false,\"stateMutability\":\"view\",\"type\":\"function\"},{\"inputs\":[],\"payable\":false,\"stateMutability\":\"nonpayable\",\"type\":\"constructor\"}]","bin":"608060405234801561001057600080fd5b506040518060400160405280600581526020017f48656c6c6f0000000000000000000000000000000000000000000000000000008152506000908051906020019061005c929190610062565b50610107565b828054600181600116156101000203166002900490600052602060002090601f016020900481019282601f106100a357805160ff19168380011785556100d1565b828001600101855582156100d1579182015b828111156100d05782518255916020019190600101906100b5565b5b5090506100de91906100e2565b5090565b61010491905b808211156101005760008160009055506001016100e8565b5090565b90565b610432806101166000396000f3fe608060405234801561001057600080fd5b50600436106100415760003560e01c8063a413686214610046578063cfae321714610101578063ef690cc014610184575b600080fd5b6100ff6004803603602081101561005c57600080fd5b810190808035906020019064010000000081111561007957600080fd5b82018360208201111561008b57600080fd5b803590602001918460018302840111640100000000831117156100ad57600080fd5b91908080601f016020809104026020016040519081016040528093929190818152602001838380828437600081840152601f19601f820116905080830192505050505050509192919290505050610207565b005b610109610221565b6040518080602001828103825283818151815260200191508051906020019080838360005b8381101561014957808201518184015260208101905061012e565b50505050905090810190601f1680156101765780820380516001836020036101000a031916815260200191505b509250505060405180910390f35b61018c6102c3565b6040518080602001828103825283818151815260200191508051906020019080838360005b838110156101cc5780820151818401526020810190506101b1565b50505050905090810190601f1680156101f95780820380516001836020036101000a031916815260200191505b509250505060405180910390f35b806000908051906020019061021d929190610361565b5050565b606060008054600181600116156101000203166002900480601f0160208091040260200160405190810160405280929190818152602001828054600181600116156101000203166002900480156102b95780601f1061028e576101008083540402835291602001916102b9565b820191906000526020600020905b81548152906001019060200180831161029c57829003601f168201915b5050505050905090565b60008054600181600116156101000203166002900480601f0160208091040260200160405190810160405280929190818152602001828054600181600116156101000203166002900480156103595780601f1061032e57610100808354040283529160200191610359565b820191906000526020600020905b81548152906001019060200180831161033c57829003601f168201915b505050505081565b828054600181600116156101000203166002900490600052602060002090601f016020900481019282601f106103a257805160ff19168380011785556103d0565b828001600101855582156103d0579182015b828111156103cf5782518255916020019190600101906103b4565b5b5090506103dd91906103e1565b5090565b61040391905b808211156103ff5760008160009055506001016103e7565b5090565b9056fea165627a7a72305820814306c93ae542576424adc73baca03d7dd370f976456800cea5ac7610ebe28a0029"}},"version":"0.5.5+commit.47a71e8f.Linux.g++"}
""")

def main(argv):
    parser = argparse.ArgumentParser(description="Interpreter")
    parser.add_argument("--ipc", metavar="IPC", type=str, help="IPC to connect to node")
    args = parser.parse_args(argv)

    eth_provider = Web3.IPCProvider(args.ipc)
    w3 = Web3(eth_provider)

    abi      = greeter_json["contracts"]["greeter.sol:Greeter"]["abi"]
    bytecode = greeter_json["contracts"]["greeter.sol:Greeter"]["bin"]

    Greeter = w3.eth.contract(abi=abi, bytecode=bytecode)

    account = w3.eth.accounts[0]
    tx_hash = Greeter.constructor().transact({"from" : account})
    tx_receipt = w3.eth.waitForTransactionReceipt(tx_hash)

    greeter = w3.eth.contract(
       address=tx_receipt.contractAddress,
       abi=abi,
    )

    print('Default contract greeting: {}'.format(
        greeter.functions.greet().call()
    ))

if __name__ == "__main__":
    main(sys.argv[1:])

Put the code into minerror.py, flag it executable, then run it with ./minerror.py --ipc /path/to/geth.ipc

Here is the Solidity source:


// Greeter.sol copy-pasted from https://web3py.readthedocs.io/en/stable/contracts.html
// Adapted for Solidity 0.5.x

pragma solidity ^0.5.5;

contract Greeter {
    string public greeting;

    constructor() public {
        greeting = 'Hello';
    }

    function setGreeting(string memory _greeting) public {
        greeting = _greeting;
    }

    function greet() view public returns (string memory) {
        return greeting;
    }
}

I'm using a private testnet based on the latest version of geth, based on this guide. Based on the guide, I've used a custom --networkid value, I've rewritten geth's CalcDifficulty() function to simply return big.NewInt(1), and I have a custom genesis alloc.

Python is running in a virtualenv. Here's some more detailed information:

$ geth version
Geth
Version: 1.8.23-stable
Architecture: amd64
Protocol Versions: [63 62]
Network Id: 1
Go Version: go1.12
Operating System: linux
GOPATH=
GOROOT=/usr/lib/go-1.12

$ solc --version
solc, the solidity compiler commandline interface
Version: 0.5.5+commit.47a71e8f.Linux.g++

$ cat /etc/issue
Ubuntu 18.04.2 LTS \n \l

$ pip freeze
attrdict==2.0.1
certifi==2019.3.9
chardet==3.0.4
cytoolz==0.9.0.1
eth-abi==1.3.0
eth-account==0.3.0
eth-hash==0.2.0
eth-keyfile==0.5.1
eth-keys==0.2.1
eth-rlp==0.1.2
eth-typing==2.1.0
eth-utils==1.4.1
hexbytes==0.1.0
idna==2.8
lru-dict==1.1.6
parsimonious==0.8.1
pkg-resources==0.0.0
pycryptodome==3.7.3
requests==2.21.0
rlp==1.1.0
six==1.12.0
toolz==0.9.0
urllib3==1.24.1
web3==4.8.3
websockets==6.0

$ python3 --version
Python 3.6.7

$ go version
go version go1.12 linux/amd64

Fierozen commented 5 years ago

I too am getting the same error:

site-packages/eth_abi/decoding.py", line 296, in read_data_from_stream
    len(data),
eth_abi.exceptions.InsufficientDataBytes: Tried to read 32 bytes.  Only got 0 bytes

Similar conditions are:

Private network
geth running in Docker
Client Python 3.6
Solc version 0.5.5
happens running Greeter.sol during return in call of greet function

I'm going to start trying different changes that I can control. Eager to hear more from anyone.

Fierozen commented 5 years ago

Several examples are working for me now - including greeting.sol, after I installed py-solc-x and installed and used solc v0.4.25 rather than v0.5.5 or v0.5.6.

davesque commented 5 years ago

Not sure if this is the cause, but it's worth noting that py-solc only supports up to solc version 0.4.25: https://github.com/ethereum/py-solc/blob/master/solc/install.py#L14-L34

However, that might not be related because a higher solc version would likely just cause py-solc to crash and not work at all.

Fierozen commented 5 years ago

@davesque I was using py-solc-x, which has support for solc 5.x, and I was experiencing the error. It was only when I notched down the version of solc to v0.4.25 that everything started working again.

Unless, of course, py-solc-x support for v0.5.x itself has problems.....

davesque commented 5 years ago

@Fierozen Ahh, my mistake.

sam-ulrich1 commented 5 years ago

@davesque I can't speak for anyone else, but in my case, I had the same issue with solc 0.4.18, 0.4.15, and 0.5.3.

tgreco commented 5 years ago

What's the solution to this? I cannot interact with any contracts at the moment.

File "/usr/local/lib/python3.6/site-packages/eth_abi/decoding.py", line 296, in read_data_from_stream len(data), eth_abi.exceptions.InsufficientDataBytes: Tried to read 32 bytes. Only got 0 bytes

pipermerriam commented 5 years ago

So far I have not seen anything definitive to show this as a web3.py bug and it can be explained by any of the following:

interacting with the wrong address
interacting with an account with empty code (such as one from a failed deployment)

@kclowes and I talked about an update to this error message to provide more detailed information about what happened so that we can better try to diagnose, as well as a new section in the documentation which explicitely goes through the debugging steps one should go through when they encounter this. They are roughly:

verify that the address you are interacting with has code associated with it via: w3.eth.getCode
verify that you are interacting with the address you expect by checking contract.address property.
try calling manualy by constructing transaction manually and executing via w3.eth.call

sam-ulrich1 commented 5 years ago

@tgreco If you want I have forked the web3 repo and created a really ugly solution but it works. https://github.com/sam-ulrich1/web3.py

modagi commented 5 years ago

same situation in mainnet. I'm wondering if it is an issue about rpc module of geth. When I changed only the rpc directory to https://github.com/ethereum/go-ethereum/commit/f91312dbdbb9e04ef578946226e5d8069d5dfd5a, it worked fine(with minor modifying to other directories affected by the rpc change). So I suspect that this is a bug since https://github.com/ethereum/go-ethereum/commit/245f3146c26698193c4b479e7bc5825b058c444a.

Environment

$ cat /etc/issue
Ubuntu 18.04.1 LTS \n \l

$ geth version
Geth
Version: 1.9.0-unstable
Git Commit: 59e195324643e8f3a18396b529e3350e550fdecc
Architecture: amd64
Protocol Versions: [63 62]
Network Id: 1
Go Version: go1.10.4
Operating System: linux
GOPATH=
GOROOT=/usr/lib/go-1.10

$ python3 --version
Python 3.6.7

$ pip3 freeze
apturl==0.5.2
asn1crypto==0.24.0
attrdict==2.0.1
attrs==17.4.0
Brlapi==0.6.6
certifi==2019.3.9
chardet==3.0.4
command-not-found==0.3
cryptography==2.1.4
cupshelpers==1.0
cytoolz==0.9.0.1
defer==1.0.6
distro-info==0.18
eth-abi==1.3.0
eth-account==0.3.0
eth-hash==0.2.0
eth-keyfile==0.5.1
eth-keys==0.2.1
eth-rlp==0.1.2
eth-typing==2.1.0
eth-utils==1.4.1
hexbytes==0.1.0
httplib2==0.9.2
idna==2.8
keyring==10.6.0
keyrings.alt==3.0
language-selector==0.1
launchpadlib==1.10.6
lazr.restfulclient==0.13.5
lazr.uri==1.0.3
louis==3.5.0
lru-dict==1.1.6
macaroonbakery==1.1.3
Mako==1.0.7
MarkupSafe==1.0
netifaces==0.10.4
oauth==1.0.1
olefile==0.45.1
parsimonious==0.8.1
pexpect==4.2.1
Pillow==5.1.0
pluggy==0.6.0
protobuf==3.0.0
py==1.5.2
py-solc==3.2.0
pyahocorasick==1.4.0
pycairo==1.16.2
pycrypto==2.6.1
pycryptodome==3.8.0
pycups==1.9.73
pygobject==3.26.1
pymacaroons==0.13.0
PyNaCl==1.1.2
pyRFC3339==1.0
pytest==3.3.2
python-apt==1.6.3
python-debian==0.1.32
pytz==2018.3
pyxdg==0.25
PyYAML==3.12
reportlab==3.4.0
requests==2.21.0
requests-unixsocket==0.1.5
rlp==1.1.0
SecretStorage==2.3.1
semantic-version==2.6.0
sha3==0.2.1
simplejson==3.13.2
six==1.12.0
ssh-import-id==5.7
system-service==0.3
systemd-python==234
toolz==0.9.0
ubuntu-drivers-common==0.0.0
ufw==0.35
unattended-upgrades==0.1
urllib3==1.24.1
usb-creator==0.3.3
wadllib==1.3.2
web3==4.9.0
websockets==6.0
xkit==0.0.0
zope.interface==4.3.2

Test information

Call to deployed ERC20 token contracts fails. I tested some of top ERC20 tokens in Etherscan. The abi below is a typical ERC20 abi. For each contract, the results are the same using the verified abi from Etherscan.

address : "0xB8c77482e45F1F44dE1745F52C74426C631bDD52", "0xE41d2489571d322189246DaFA5ebDe1F4699F498", "0x6f259637dcD74C767781E37Bc6133cd6A68aa161"
calling method : name, symbol, decimals, totalSupply, owner
abi : [{"constant":true,"inputs":[],"name":"name","outputs":[{"name":"","type":"bytes32"}],"payable":false,"type":"function"},{"constant":false,"inputs":[{"name":"_spender","type":"address"},{"name":"_value","type":"uint256"}],"name":"approve","outputs":[{"name":"","type":"bool"}],"payable":false,"type":"function"},{"constant":true,"inputs":[],"name":"totalSupply","outputs":[{"name":"","type":"uint256"}],"payable":false,"type":"function"},{"constant":false,"inputs":[{"name":"_from","type":"address"},{"name":"_to","type":"address"},{"name":"_value","type":"uint256"}],"name":"transferFrom","outputs":[{"name":"","type":"bool"}],"payable":false,"type":"function"},{"constant":true,"inputs":[],"name":"decimals","outputs":[{"name":"","type":"bytes"}],"payable":false,"type":"function"},{"constant":true,"inputs":[{"name":"_owner","type":"address"}],"name":"balanceOf","outputs":[{"name":"","type":"uint256"}],"payable":false,"type":"function"},{"constant":true,"inputs":[],"name":"symbol","outputs":[{"name":"","type":"bytes32"}],"payable":false,"type":"function"},{"constant":false,"inputs":[{"name":"_to","type":"address"},{"name":"_value","type":"uint256"}],"name":"transfer","outputs":[{"name":"","type":"bool"}],"payable":false,"type":"function"},{"constant":true,"inputs":[{"name":"_owner","type":"address"},{"name":"_spender","type":"address"}],"name":"allowance","outputs":[{"name":"","type":"uint256"}],"payable":false,"type":"function"},{"inputs":[],"payable":false,"type":"constructor"},{"anonymous":false,"inputs":[{"indexed":true,"name":"_from","type":"address"},{"indexed":true,"name":"_to","type":"address"},{"indexed":false,"name":"_value","type":"uint256"}],"name":"Transfer","type":"event"},{"anonymous":false,"inputs":[{"indexed":true,"name":"_owner","type":"address"},{"indexed":true,"name":"_spender","type":"address"},{"indexed":false,"name":"_value","type":"uint256"}],"name":"Approval","type":"event"}]

Test code

# -*- coding: utf-8 -*-

from web3 import Web3, HTTPProvider
from web3.contract import ConciseContract

w3 = Web3(HTTPProvider("http://MY_NODE:8545"))

def test(contractAddress):
    abi = 'THE ONE DESCRIBED ABOVE'

    contractInstance = w3.eth.contract(address=contractAddress, abi=abi)
    reader = ConciseContract(contractInstance)

    print("name : " + reader.name())

test("0xB8c77482e45F1F44dE1745F52C74426C631bDD52") # BNB
test("0xE41d2489571d322189246DaFA5ebDe1F4699F498") # 0x protocol
test("0x6f259637dcD74C767781E37Bc6133cd6A68aa161") # Huobi

Result

Traceback (most recent call last):
  File "/home/modagi/.local/lib/python3.6/site-packages/web3/contract.py", line 1390, in call_contract_function
    output_data = decode_abi(output_types, return_data)
  File "/home/modagi/.local/lib/python3.6/site-packages/eth_abi/abi.py", line 96, in decode_abi
    return decoder(stream)
  File "/home/modagi/.local/lib/python3.6/site-packages/eth_abi/decoding.py", line 118, in __call__
    return self.decode(stream)
  File "/home/modagi/.local/lib/python3.6/site-packages/eth_utils/functional.py", line 46, in inner
    return callback(fn(*args, **kwargs))
  File "/home/modagi/.local/lib/python3.6/site-packages/eth_abi/decoding.py", line 164, in decode
    yield decoder(stream)
  File "/home/modagi/.local/lib/python3.6/site-packages/eth_abi/decoding.py", line 118, in __call__
    return self.decode(stream)
  File "/home/modagi/.local/lib/python3.6/site-packages/eth_abi/decoding.py", line 186, in decode
    raw_data = self.read_data_from_stream(stream)
  File "/home/modagi/.local/lib/python3.6/site-packages/eth_abi/decoding.py", line 296, in read_data_from_stream
    len(data),
eth_abi.exceptions.InsufficientDataBytes: Tried to read 32 bytes.  Only got 0 bytes

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "testerror.py", line 16, in <module>
    test("0xB8c77482e45F1F44dE1745F52C74426C631bDD52") # BNB
  File "testerror.py", line 14, in test
    print("name : " + reader.name())
  File "/home/modagi/.local/lib/python3.6/site-packages/web3/contract.py", line 894, in __call__
    return self.__prepared_function(*args, **kwargs)
  File "/home/modagi/.local/lib/python3.6/site-packages/web3/contract.py", line 907, in __prepared_function
    return getattr(self._function(*args), modifier)(modifier_dict)
  File "/home/modagi/.local/lib/python3.6/site-packages/web3/contract.py", line 1122, in call
    **self.kwargs
  File "/home/modagi/.local/lib/python3.6/site-packages/web3/contract.py", line 1412, in call_contract_function
    raise BadFunctionCallOutput(msg) from e
web3.exceptions.BadFunctionCallOutput: Could not decode contract function call name return data b'' for output_types ['bytes32']

While the contract call request is being processed by Run function in core/vm/interpreter.go, it appears to complete abnormally(https://github.com/ethereum/go-ethereum/blob/master/core/vm/interpreter.go#L191). The reason seems to be because the context is cancelled(https://github.com/ethereum/go-ethereum/blob/master/rpc/server.go#L102).

When I tried with the solution of @sam-ulrich1, sometimes it worked after thousands or even tens of thousands of iterations. But often it didn't succeed. As interpreter processing and context closing proceed simultaneously, contract call seems to complete normally only when interpreter processing is done first by chance.

sam-ulrich1 commented 5 years ago

@modagi Hey nice work on tracing this back. Could you open up an issue on the go-ethereum repo?

pipermerriam commented 5 years ago

Yeah, if the RPC call is returning an empty byte string (empty hex) when it's expected to return a value that sounds a lot like a go-ethereum issue. I'm curious if anyone has experienced this with parity or any other JSON-RPC provider (note that infura does use geth on it's backend so most any cloud provider would likely just be a manifestation of the same issue)

modagi commented 5 years ago

I tested the code below by connecting to my geth and parity node. All contract calls in geth fail like the above, and all succeeds in parity like below. I'll open up this issue on the geth repo.

NOTE : I changed ABI to something from https://github.com/ethereum/wiki/wiki/Contract-ERC20-ABI

Test code

# -*- coding: utf-8 -*-

from web3 import Web3, HTTPProvider
from web3.contract import ConciseContract

w3 = Web3(HTTPProvider("parity rpc endpoint"))

def test(contractAddress):
    abi = 'ABI in https://github.com/ethereum/wiki/wiki/Contract-ERC20-ABI'

    contractInstance = w3.eth.contract(address=contractAddress, abi=abi)
    reader = ConciseContract(contractInstance)

    print("contractAddress : ", contractAddress)
    print("name : ", reader.name())
    print("symbol : ", reader.symbol())
    print("decimals : ", str(reader.decimals()))
    print("totalSupply : ", str(reader.totalSupply()))
    print("owner : ", reader.owner())

test("0xB8c77482e45F1F44dE1745F52C74426C631bDD52")
test("0xD850942eF8811f2A866692A623011bDE52a462C1")
test("0xd26114cd6EE289AccF82350c8d8487fedB8A0C07")

Result of parity node

contractAddress :  0xB8c77482e45F1F44dE1745F52C74426C631bDD52
name :  BNB
symbol :  BNB
decimals :  18
totalSupply :  197192382000000000000000000
owner :  0x00C5E04176d95A286fccE0E68c683Ca0bfec8454
contractAddress :  0xD850942eF8811f2A866692A623011bDE52a462C1
name :  VeChain Token
symbol :  VEN
decimals :  18
totalSupply :  1000000000000000000000000000
owner :  None
contractAddress :  0xd26114cd6EE289AccF82350c8d8487fedB8A0C07
name :  OMGToken
symbol :  OMG
decimals :  18
totalSupply :  140245398245132780789239631
owner :  0x000000000000000000000000000000000000dEaD

modagi commented 5 years ago

https://github.com/ethereum/go-ethereum/issues/19186 looks like same issue.

pipermerriam commented 5 years ago

I'm going to close this issue since this is looking a lot like an issue with go-ethereum. Feel free to continue the discussion.

Fierozen commented 5 years ago

Here is what fixed it for me, for all solc versions, in both geth and parity: When invoking the compiler, the evmVersion must be set to the correct level depending on the Solidity version. In my case, the Docker image of geth must still use evmVersion : 'byzantium', while the parity version, as explained on their web site, is using the latest, 'petersburg'.

result = compile_standard({ \
    'language': 'Solidity',

    'sources': {
        fileName: {
            'content': contract_source,
        },
    },
    'settings': {
        'evmVersion': 'byzantium',
        'outputSelection': {
            "*": {"*": ["*"]},
        }

see the following for full explanation:

https://solidity.readthedocs.io/en/v0.5.8/using-the-compiler.html#setting-the-evm-version-to-target

uliantsev-a commented 5 years ago

@Fierozen thank it helped me. I work with test contract and use web3py and run solc in console through docker. For use this solution i ran next comand:

cat contracts/input.json | docker run -i -v ~/Projects/simple_transaction/contracts:/source ethereum/solc:stable -o /source --bin --allow-paths /source --standard-json > contract.json

with next input.json:

{
  "language": "Solidity",
  "sources": {
      "source/greeting.sol": {
        "urls": ["/source/greeting.sol"]
      }
  },
  "settings": {"evmVersion": "byzantium", "outputSelection": {"*": { "*": [ "*" ] }}}
}

and use bytecode with abi from result

rar8000 commented 4 years ago

@Fierozen @uliantsev-a In python (from original post), you can simply add the evm flag to the compilesource() call. Since the function uses a **kwargs passthrough to solc, and autoreplaces underscores with dashes - in the argument name.

compile_source(source=contract_source_code, evm_version='byzantium')

Additionally I used py-solc-x (import solcx). Also added a byzantium block to my private network's genesis config ("byzantiumBlock": 0).