IndexError: list index out of range

JhonnyJason commented 3 years ago

Version Information

vyper Version (output of vyper --version): 0.2.8+commit.069936f
OS: linux
Python Version (output of python --version): Python 3.9.0

Environment (output of pip freeze):

alabaster==0.7.12
appdirs==1.4.4
asn1crypto==1.4.0
asttokens==2.0.4
Babel==2.8.1
backcall==0.2.0
Brlapi==0.8.0
btrfsutil==5.9
CacheControl==0.12.6
certifi==2020.6.20
cffi==1.14.4
chardet==3.0.4
colorama==0.4.4
contextlib2==0.6.0.post1
cryptography==3.2.1
cvxopt==1.2.5
cycler==0.10.0
cypari2==2.1.2
cysignals==1.10.2
decorator==4.4.2
distlib==0.3.1
distro==1.5.0
docutils==0.16
filelock==3.0.12
fpylll==0.5.2.dev0
future==0.18.2
gmpy2==2.1.0b5
html5lib==1.1
idna==2.10
imagesize==1.2.0
ipython==7.19.0
ipython-genutils==0.1.0
jedi==0.17.2
Jinja2==2.11.2
kiwisolver==1.3.1
lensfun==0.3.95
louis==3.15.0
MarkupSafe==1.1.1
matplotlib==3.3.3
mpmath==1.1.0
msgpack==1.0.0
networkx==2.5
numpy==1.19.4
ordered-set==4.0.2
packaging==20.7
parso==0.7.1
pep517==0.9.1
pexpect==4.8.0
pickleshare==0.7.5
Pillow==8.0.1
pipenv==2020.11.15
ply==3.11
pplpy==0.8.4
progress==1.5
prompt-toolkit==3.0.8
psutil==5.7.3
ptyprocess==0.6.0
pycparser==2.20
pycryptodome==3.9.9
Pygments==2.7.2
pyOpenSSL==20.0.0
pyparsing==2.4.7
python-dateutil==2.8.1
pytz==2020.4
requests==2.25.0
resolvelib==0.5.2
retrying==1.3.3
rpy2==3.3.6
sage==9.2
scipy==1.5.4
semantic-version==2.8.5
six==1.15.0
snowballstemmer==2.0.0
Sphinx==3.3.1
sphinxcontrib-applehelp==1.0.2
sphinxcontrib-devhelp==1.0.2
sphinxcontrib-htmlhelp==1.0.3
sphinxcontrib-jsmath==1.0.1
sphinxcontrib-qthelp==1.0.3
sphinxcontrib-serializinghtml==1.1.4
sympy==1.7
toml==0.10.2
traitlets==4.3.3
tzlocal==2.1
urllib3==1.25.10
virtualenv==20.2.0
virtualenv-clone==0.5.4
vyper==0.2.8
wcwidth==0.2.5
webencodings==0.5.1

What's your issue about?

Following code produces this Compile Error and it fails to compile: IndexError: list index out of range

# @version ^0.2.8

INITIAL_MINTABLE_PRO_ETH: constant(uint256) = 666
INITIAL_MINT_CEILING: constant(uint256) = 100000000000000000000000 #100k

############################################################
struct Parameters:
    mintableProETH: Bytes[2]
    mintCeiling: Bytes[15]

############################################################
params: Parameters

############################################################
@external
def __init__():
    self.params.mintableProETH = slice(convert(INITIAL_MINTABLE_PRO_ETH, bytes32), 30, 2)
    self.params.mintCeiling = slice(convert(INITIAL_MINT_CEILING, bytes32), 17, 15)
   ## This also fails in the same way
   #  self.params.mintableProETH = slice(convert(666, bytes32), 30, 2)
   #  self.params.mintCeiling = slice(convert(100000000000000000000000, bytes32), 17, 15)

How can it be fixed?

I don't know what really is the cause of this. I do have a workaround though.

INITIAL_MINTABLE_PRO_ETH: constant(Bytes[2]) = 0x029a # 666 -> hex
INITIAL_MINT_CEILING: constant(Bytes[15]) = 0x152d02c7e14af6800000 # 100k -> hex

############################################################
struct Parameters:
    mintableProETH: Bytes[2]
    mintCeiling: Bytes[15]

############################################################
params: Parameters

############################################################
@external
def __init__(_name: String[26], _symbol: String[6]):
    self.params.mintableProETH = INITIAL_MINTABLE_PRO_ETH
    self.params.mintCeiling = INITIAL_MINT_CEILING

Cheers!

agroce commented 3 years ago

This may be related to something I'm seeing in fuzzing. Using the compiler without a file to compile strings, I get:

>>> vyper.compiler.compile_code('\r[[]]\ndef _(e:[],l:[]):\n    """"""""""""""""""""""""""""""""""""""""""""""""""""""\n    f.n()')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python3.8/dist-packages/vyper-0.2.8-py3.8.egg/vyper/compiler/__init__.py", line 151, in compile_code
    return compile_codes(
  File "/usr/local/lib/python3.8/dist-packages/vyper-0.2.8-py3.8.egg/vyper/opcodes.py", line 222, in _wrapper
    return fn(*args, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/vyper-0.2.8-py3.8.egg/vyper/compiler/__init__.py", line 110, in compile_codes
    raise exc
  File "/usr/local/lib/python3.8/dist-packages/vyper-0.2.8-py3.8.egg/vyper/compiler/__init__.py", line 105, in compile_codes
    out[contract_name][output_format] = OUTPUT_FORMATS[output_format](compiler_data)
  File "/usr/local/lib/python3.8/dist-packages/vyper-0.2.8-py3.8.egg/vyper/compiler/output.py", line 138, in build_bytecode_output
    return f"0x{compiler_data.bytecode.hex()}"
  File "/usr/local/lib/python3.8/dist-packages/vyper-0.2.8-py3.8.egg/vyper/compiler/phases.py", line 126, in bytecode
    self._bytecode = generate_bytecode(self.assembly)
  File "/usr/local/lib/python3.8/dist-packages/vyper-0.2.8-py3.8.egg/vyper/compiler/phases.py", line 114, in assembly
    self._assembly = generate_assembly(self.lll_nodes)
  File "/usr/local/lib/python3.8/dist-packages/vyper-0.2.8-py3.8.egg/vyper/compiler/phases.py", line 102, in lll_nodes
    self._gen_lll()
  File "/usr/local/lib/python3.8/dist-packages/vyper-0.2.8-py3.8.egg/vyper/compiler/phases.py", line 97, in _gen_lll
    self._lll_nodes, self._lll_runtime = generate_lll_nodes(self.global_ctx)
  File "/usr/local/lib/python3.8/dist-packages/vyper-0.2.8-py3.8.egg/vyper/compiler/phases.py", line 90, in global_ctx
    self.vyper_module_folded, self.interface_codes
  File "/usr/local/lib/python3.8/dist-packages/vyper-0.2.8-py3.8.egg/vyper/compiler/phases.py", line 81, in vyper_module_folded
    self._vyper_module_folded = generate_folded_ast(self.vyper_module)
  File "/usr/local/lib/python3.8/dist-packages/vyper-0.2.8-py3.8.egg/vyper/compiler/phases.py", line 74, in vyper_module
    self._vyper_module = generate_ast(self.source_code, self.source_id, self.contract_name)
  File "/usr/local/lib/python3.8/dist-packages/vyper-0.2.8-py3.8.egg/vyper/compiler/phases.py", line 154, in generate_ast
    return vy_ast.parse_to_ast(source_code, source_id, contract_name)
  File "/usr/local/lib/python3.8/dist-packages/vyper-0.2.8-py3.8.egg/vyper/ast/utils.py", line 36, in parse_to_ast
    annotate_python_ast(py_ast, source_code, class_types, source_id, contract_name)
  File "/usr/local/lib/python3.8/dist-packages/vyper-0.2.8-py3.8.egg/vyper/ast/annotation.py", line 281, in annotate_python_ast
    tokens = asttokens.ASTTokens(source_code, tree=parsed_ast)
  File "/usr/local/lib/python3.8/dist-packages/asttokens-2.0.4-py3.8.egg/asttokens/asttokens.py", line 65, in __init__
    self.mark_tokens(self._tree)
  File "/usr/local/lib/python3.8/dist-packages/asttokens-2.0.4-py3.8.egg/asttokens/asttokens.py", line 76, in mark_tokens
    MarkTokens(self).visit_tree(root_node)
  File "/usr/local/lib/python3.8/dist-packages/asttokens-2.0.4-py3.8.egg/asttokens/mark_tokens.py", line 49, in visit_tree
    util.visit_tree(node, self._visit_before_children, self._visit_after_children)
  File "/usr/local/lib/python3.8/dist-packages/asttokens-2.0.4-py3.8.egg/asttokens/util.py", line 199, in visit_tree
    ret = postvisit(current, par_value, value)
  File "/usr/local/lib/python3.8/dist-packages/asttokens-2.0.4-py3.8.egg/asttokens/mark_tokens.py", line 92, in _visit_after_children
    nfirst, nlast = self._methods.get(self, node.__class__)(node, first, last)
  File "/usr/local/lib/python3.8/dist-packages/asttokens-2.0.4-py3.8.egg/asttokens/mark_tokens.py", line 189, in handle_attr
    name = self._code.next_token(dot)
  File "/usr/local/lib/python3.8/dist-packages/asttokens-2.0.4-py3.8.egg/asttokens/asttokens.py", line 141, in next_token
    while is_non_coding_token(self._tokens[i].type):
IndexError: list index out of range

agroce commented 3 years ago

(I did not get a chance to see if same exception chain, since the inputs are obviously different.

By the way, is it fine to submit bugs through just the string sent to 'vyper.compiler.compile_code'? Not going through vyper CLI is a huge win for persistent AFL fuzzing, but due to some file reading (with odd character) issues, they don't always reproduce. But I assume CompilerPanics or non-vyper exceptions from compile_code are fine, since under the hood CI is calling that, anyway?

agroce commented 3 years ago

(Attn @iamdefinitelyahuman on the fuzzing q)

agroce commented 3 years ago

I'll report mine separately, since for me the code in this bug report fails with:

root@c2bc5e4cc2af:~/vyfuzz# vyper v.vy 
ValueError: start (21,89) precedes previous end (22,0)

different symptom, while I can produce IndexError.

JhonnyJason commented 3 years ago

@agroce Very interesting the different Error Message :thinking:

Well I think the biggest problem for IndexError is that it is too general

agroce commented 3 years ago

Yeah, I think we probably just have different bugs, see https://github.com/vyperlang/vyper/issues/2258

charles-cooper commented 2 years ago

This was fixed in #2500

vyperlang / vyper