search

awslabs / aws-shell

An integrated shell for working with the AWS CLI.
Apache License 2.0
7.15k stars 770 forks source link

docutils error #195

Open altaurog opened 6 years ago

altaurog commented 6 years ago

I am getting the following traceback, apparently from the background document index creation thread:

Exception in thread Thread-1:
Traceback (most recent call last):
  File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner
    self.run()
  File "/usr/lib/python3.6/threading.py", line 864, in run
    self._target(*self._args, **self._kwargs)lumn  [F5] Help: ON  [F9] Focus: cli  [F10] Exit                             
  File "/home/aryeh/.local/lib/python3.6/site-packages/awsshell/makeindex.py", line 88, in write_doc_index
    do_write_doc_index(db, help_command, close_db_on_finish=should_close)
  File "/home/aryeh/.local/lib/python3.6/site-packages/awsshell/makeindex.py", line 93, in do_write_doc_index
    _index_docs(db, help_command)
  File "/home/aryeh/.local/lib/python3.6/site-packages/awsshell/makeindex.py", line 111, in _index_docs
    _index_docs(db, sub_help_command)
  File "/home/aryeh/.local/lib/python3.6/site-packages/awsshell/makeindex.py", line 108, in _index_docs
    text_docs = render_docs_for_cmd(sub_help_command)
  File "/home/aryeh/.local/lib/python3.6/site-packages/awsshell/makeindex.py", line 121, in render_docs_for_cmd
    text_content = convert_rst_to_basic_text(original_cli_help)
  File "/home/aryeh/.local/lib/python3.6/site-packages/awsshell/makeindex.py", line 146, in convert_rst_to_basic_text
    settings_overrides={'report_level': 5})
  File "/home/aryeh/.local/lib/python3.6/site-packages/docutils/core.py", line 416, in publish_string
    enable_exit_status=enable_exit_status)
  File "/home/aryeh/.local/lib/python3.6/site-packages/docutils/core.py", line 664, in publish_programmatically
    output = pub.publish(enable_exit_status=enable_exit_status)
  File "/home/aryeh/.local/lib/python3.6/site-packages/docutils/core.py", line 217, in publish
    self.settings)
  File "/home/aryeh/.local/lib/python3.6/site-packages/docutils/readers/__init__.py", line 72, in read
    self.parse()
  File "/home/aryeh/.local/lib/python3.6/site-packages/docutils/readers/__init__.py", line 78, in parse
    self.parser.parse(self.input, document)
  File "/home/aryeh/.local/lib/python3.6/site-packages/docutils/parsers/rst/__init__.py", line 191, in parse
    self.statemachine.run(inputlines, document, inliner=self.inliner)
  File "/home/aryeh/.local/lib/python3.6/site-packages/docutils/parsers/rst/states.py", line 171, in run
    input_source=document['source'])
  File "/home/aryeh/.local/lib/python3.6/site-packages/docutils/statemachine.py", line 239, in run
    context, state, transitions)
  File "/home/aryeh/.local/lib/python3.6/site-packages/docutils/statemachine.py", line 460, in check_line
    return method(match, context, next_state)
  File "/home/aryeh/.local/lib/python3.6/site-packages/docutils/parsers/rst/states.py", line 2988, in text
    self.section(title.lstrip(), source, style, lineno + 1, messages)
  File "/home/aryeh/.local/lib/python3.6/site-packages/docutils/parsers/rst/states.py", line 327, in section
    self.new_subsection(title, lineno, messages)
  File "/home/aryeh/.local/lib/python3.6/site-packages/docutils/parsers/rst/states.py", line 395, in new_subsection
    node=section_node, match_titles=True)
  File "/home/aryeh/.local/lib/python3.6/site-packages/docutils/parsers/rst/states.py", line 282, in nested_parse
    node=node, match_titles=match_titles)
  File "/home/aryeh/.local/lib/python3.6/site-packages/docutils/parsers/rst/states.py", line 196, in run
    results = StateMachineWS.run(self, input_lines, input_offset)
  File "/home/aryeh/.local/lib/python3.6/site-packages/docutils/statemachine.py", line 239, in run
    context, state, transitions)
  File "/home/aryeh/.local/lib/python3.6/site-packages/docutils/statemachine.py", line 460, in check_line
    return method(match, context, next_state)
  File "/home/aryeh/.local/lib/python3.6/site-packages/docutils/parsers/rst/states.py", line 2988, in text
    self.section(title.lstrip(), source, style, lineno + 1, messages)
  File "/home/aryeh/.local/lib/python3.6/site-packages/docutils/parsers/rst/states.py", line 327, in section
    self.new_subsection(title, lineno, messages)
  File "/home/aryeh/.local/lib/python3.6/site-packages/docutils/parsers/rst/states.py", line 395, in new_subsection
    node=section_node, match_titles=True)
  File "/home/aryeh/.local/lib/python3.6/site-packages/docutils/parsers/rst/states.py", line 282, in nested_parse
    node=node, match_titles=match_titles)
  File "/home/aryeh/.local/lib/python3.6/site-packages/docutils/parsers/rst/states.py", line 196, in run
    results = StateMachineWS.run(self, input_lines, input_offset)
  File "/home/aryeh/.local/lib/python3.6/site-packages/docutils/statemachine.py", line 239, in run
    context, state, transitions)
  File "/home/aryeh/.local/lib/python3.6/site-packages/docutils/statemachine.py", line 460, in check_line
    return method(match, context, next_state)
  File "/home/aryeh/.local/lib/python3.6/site-packages/docutils/parsers/rst/states.py", line 1150, in indent
    elements = self.block_quote(indented, line_offset)
  File "/home/aryeh/.local/lib/python3.6/site-packages/docutils/parsers/rst/states.py", line 1165, in block_quote
    self.nested_parse(blockquote_lines, line_offset, blockquote)
  File "/home/aryeh/.local/lib/python3.6/site-packages/docutils/parsers/rst/states.py", line 282, in nested_parse
    node=node, match_titles=match_titles)
  File "/home/aryeh/.local/lib/python3.6/site-packages/docutils/parsers/rst/states.py", line 196, in run
    results = StateMachineWS.run(self, input_lines, input_offset)
  File "/home/aryeh/.local/lib/python3.6/site-packages/docutils/statemachine.py", line 239, in run
    context, state, transitions)
  File "/home/aryeh/.local/lib/python3.6/site-packages/docutils/statemachine.py", line 460, in check_line
    return method(match, context, next_state)
  File "/home/aryeh/.local/lib/python3.6/site-packages/docutils/parsers/rst/states.py", line 1150, in indent
    elements = self.block_quote(indented, line_offset)
  File "/home/aryeh/.local/lib/python3.6/site-packages/docutils/parsers/rst/states.py", line 1165, in block_quote
    self.nested_parse(blockquote_lines, line_offset, blockquote)
  File "/home/aryeh/.local/lib/python3.6/site-packages/docutils/parsers/rst/states.py", line 282, in nested_parse
    node=node, match_titles=match_titles)
  File "/home/aryeh/.local/lib/python3.6/site-packages/docutils/parsers/rst/states.py", line 196, in run
    results = StateMachineWS.run(self, input_lines, input_offset)
  File "/home/aryeh/.local/lib/python3.6/site-packages/docutils/statemachine.py", line 239, in run
    context, state, transitions)
  File "/home/aryeh/.local/lib/python3.6/site-packages/docutils/statemachine.py", line 460, in check_line
    return method(match, context, next_state)
  File "/home/aryeh/.local/lib/python3.6/site-packages/docutils/parsers/rst/states.py", line 2398, in line
    line=self.state_machine.abs_line_number())
  File "/home/aryeh/.local/lib/python3.6/site-packages/docutils/utils/__init__.py", line 236, in severe
    return self.system_message(self.SEVERE_LEVEL, *args, **kwargs)
  File "/home/aryeh/.local/lib/python3.6/site-packages/docutils/utils/__init__.py", line 194, in system_message
    raise SystemMessage(msg, level)
docutils.utils.SystemMessage: <string>:148: (SEVERE/4) Unexpected section title or transition.

Here’s the output of pip freeze

asn1crypto==0.22.0
aws-shell==0.2.0
awscli==1.14.34
blinker==1.3
boto3==1.5.24
botocore==1.8.38
certifi==2018.1.18
chardet==3.0.4
cloud-init==17.1
colorama==0.3.7
command-not-found==0.3
configobj==5.0.6
cryptography==1.9
distro-info==0.17
docutils==0.14
flake8==3.5.0
httplib2==0.9.2
idna==2.6
Jinja2==2.9.6
jmespath==0.9.3
jsonpatch==1.16
jsonpointer==1.10
jsonschema==2.5.1
keyring==10.4.0
keyrings.alt==2.2
language-selector==0.1
MarkupSafe==1.0
mccabe==0.6.1
oauthlib==2.0.1
olefile==0.44
pew==1.1.2
Pillow==4.1.1
pipenv==9.0.3
prompt-toolkit==1.0.15
pyasn1==0.4.2
pycodestyle==2.3.1
pycrypto==2.6.1
pyflakes==1.6.0
Pygments==2.2.0
pygobject==3.24.1
PyJWT==1.4.2
pyserial==3.4
python-apt==1.4.0b3
python-dateutil==2.6.1
python-debian==0.1.30
pyxdg==0.25
PyYAML==3.12
requests==2.18.4
roman==2.0.0
rsa==3.4.2
s3transfer==0.1.12
SecretStorage==2.3.1
six==1.10.0
ssh-import-id==5.7
systemd-python==234
ufw==0.35
unattended-upgrades==0.1
urllib3==1.22
virtualenv==15.1.0
virtualenv-clone==0.2.6
wcwidth==0.1.7
danoyoung commented 6 years ago

We're seeing the same thing.

... ... ... File "/usr/local/lib/python2.7/site-packages/docutils/statemachine.py", line 239, in run context, state, transitions) File "/usr/local/lib/python2.7/site-packages/docutils/statemachine.py", line 460, in check_line return method(match, context, next_state) File "/usr/local/lib/python2.7/site-packages/docutils/parsers/rst/states.py", line 2398, in line line=self.state_machine.abs_line_number()) File "/usr/local/lib/python2.7/site-packages/docutils/utils/init.py", line 236, in severe return self.system_message(self.SEVERE_LEVEL, *args, **kwargs) File "/usr/local/lib/python2.7/site-packages/docutils/utils/init.py", line 194, in system_message raise SystemMessage(msg, level) SystemMessage: :148: (SEVERE/4) Unexpected section title or transition.

wnkz commented 6 years ago

Same here.

macOS 10.13.3 Python 3.6.4

aws-shell==0.2.0
awscli==1.14.39
boto3==1.5.29
botocore==1.8.43
danoyoung commented 6 years ago

Ditto....

aws-shell==0.2.0 awscli==1.14.39 boto==2.48.0 boto3==1.5.29 botocore==1.8.43

wnkz commented 6 years ago

The problem seems to come from aws-cli's help command:

> aws servicediscovery register-instance help
<string>:148: (SEVERE/4) Unexpected section title or transition.

****

<string>:148: (SEVERE/4) Unexpected section title or transition.

****
wnkz commented 6 years ago

In the meantime, as a workaround, you can do this so aws-shell stop crashing at every launch:

> sqlite3 ~/.aws/shell/1.14.41-completions.json.docs
SQLite version 3.19.3 2017-06-27 16:48:08
Enter ".help" for usage hints.
sqlite> insert or replace into docindex (key, value) values ('aws.servicediscovery.register-instance', '') ;
altaurog commented 6 years ago

@wnkz Could be there is more than one case of invalid rst? I am still getting an exception.

danoyoung commented 6 years ago

Me too.... ... ... SystemMessage: :148: (SEVERE/4) Unexpected section title or transition.

wnkz commented 6 years ago

@altaurog well this is the only one I found yet ; if this isn't working you can still wrap this call https://github.com/awslabs/aws-shell/blob/master/awsshell/makeindex.py#L144 in a try/except and return an empty string in case of failure so it finishes creating the index.

Also, from what I see in your pip freeze, you seem to use awscli==1.14.34 so the sqlite database you need to edit would be 1.14.34-completions.json.docs

danoyoung commented 6 years ago

tried this with awscli==1.14.41/1.14.41-completions.json.docs and still getting the:

SystemMessage: :148: (SEVERE/4) Unexpected section title or transition.

wnkz commented 6 years ago

Maybe it's working for me because I let the process skip the invalid rst. If you want to try it I edited https://github.com/awslabs/aws-shell/blob/master/awsshell/makeindex.py like this 

    try:
        converted = publish_string(
            contents, writer=BasicTextWriter(),
            settings_overrides={'report_level': 5})
        return converted.decode('utf-8')
    except:
        return ''

On the first run, It'll still show the error on stderr but it won't try to create the doc index on the next start.

danoyoung commented 6 years ago

@wnkz this work around appears to work, thank you.

wnkz commented 6 years ago

A fix has been merged https://github.com/boto/botocore/pull/1390 ; hope they make a release soon.