Symbol search on `joxeankoret/maltindex` doesn't seem to work

[beyang]

• Go to https://sourcegraph.com/github.com/joxeankoret/maltindex@6bbbb67eaca104aea7782f2e9ba1a1479c316074/-/blob/diaphora_index_batch.py#L67:19
• Issue a symbol search for "#cbatchrunner". No results, even though it exists here: https://sourcegraph.com/github.com/joxeankoret/maltindex@6bbbb67eaca104aea7782f2e9ba1a1479c316074/-/blob/diaphora_index_batch.py#L27-28

[keegancsmith]

My guess is py2 syntax is throwing off the parser we use for symbols. If the implementation hasn't changed, I used the builtin parse which is very fast but will expect py3 syntax.

[akhleung]

Yeah, we're still using the builtin parser for workspace symbols (global references too)

[beyang]

Can work around this somehow (like try the python 3 parser and then fall back to the python 2 parser)? How much of a pain would that be? I'm guessing there's a fair amount of Python 2 out there still both in OSS and private code.

[akhleung]

If we fall back to a Python 2 parser, it probably can't be the stdlib parser since our langserver is in Python 3. We could fall back on a different parser though (which would be slower).

[sqs]

I've seen a technique where you shell out to a small python2 program and have it output a JSON serialization of the AST that it parsed using its stdlib parser.

[keegancsmith]

That should work, as long as we make the program we shell out to take a list of things to parse. Otherwise the overhead of starting up python will dominate. We could go back to using a more relaxed parser like jedi does, but it is much slower. So I think using the subprocess approach would be worth it.