search

coala / cEPs

coala Enhancement Proposals
MIT License
33 stars 53 forks source link

cEP-0018.md: Add information about fixes #170

Closed virresh closed 6 years ago

virresh commented 6 years ago

Closes https://github.com/coala/cEPs/issues/169

Here is a github gist of the code that I currently have: https://gist.github.com/virresh/643cd1a2d9149668980814d970f98fbc#file-roundtripper-py-L13

Specifically speaking, the round-tripping of bears failed terribly, Antlr has a concept of HiddenTokenStreams, under which some tokens are discarded while being parsed by the lexer itself, and never make it to the parser, so if we want to reconstruct the source, we need to get the original source from the original token stream, which I tried above, but even using the code as suggested here, I am not able to retrieve the tokens on the hidden channels.

This simply means that any whitespace that was not associated with a comment/docstring is lost (because it was not sent on to the hidden channels) and is not being retrieved (although we have the line numbers and column information intact with us, since that was saved alongside with the parse-tree) The alternative we have is we add whitespaces on our own according to the column number and line number (but this obviously is wrong, since we simply won't know what the actual whitespace used was for, was it a tab or a space) PS: The python runtime has no documentation whatsoever apart from the short comments left in the source here and there, so it is highly possible I got stuff wrong, pls correct

The above is what makes me believe ANTLR can not be used to make complex fixes

jayvdb commented 6 years ago

@gitmate-bot rebase

gitmate-bot commented 6 years ago

Hey! I'm GitMate.io! This pull request is being rebased automatically. Please DO NOT push while rebase is in progress or your changes would be lost permanently :warning:

gitmate-bot commented 6 years ago

Automated rebase failed! Please rebase your pull request manually via the command line.

Reason:

Command: git push --force origin HEAD:antlrfixcEP

Exit Code: 128

Cause:

fatal: unable to access '<hidden_oauth_token>/': Could not resolve host: github.com
jayvdb commented 6 years ago

@gitmate-bot rebase

gitmate-bot commented 6 years ago

Hey! I'm GitMate.io! This pull request is being rebased automatically. Please DO NOT push while rebase is in progress or your changes would be lost permanently :warning:

gitmate-bot commented 6 years ago

Automated rebase with GitMate.io was successful! :tada:

jayvdb commented 6 years ago

ack https://github.com/coala/cEPs/pull/170/commits/b8a172996dd0bbf67eeed73358ae2d4d481332b4

jayvdb commented 6 years ago

@gitmate-bot ff

gitmate-bot commented 6 years ago

Hey! I'm GitMate.io! This pull request is being fastforwarded automatically. Please DO NOT push while fastforward is in progress or your changes would be lost permanently :warning:

gitmate-bot commented 6 years ago

Automated fastforward with GitMate.io was successful! :tada: