search

atom / first-mate

TextMate helpers
http://atom.github.io/first-mate
MIT License
91 stars 57 forks source link

decodeTokens is missing its 'scopes' arguments in the doc #89

Open arcanis opened 7 years ago

arcanis commented 7 years ago

It seems the last release added an extra parameter to pass to decodeTokens - without it, the decode process might fail with an error similar to this one:

Error: Expected popped scope to be meta.function.method.js, but it was undefined
    at GrammarRegistry.module.exports.GrammarRegistry.decodeTokens (/home/arcanis/cid/node_modules/first-mate/lib/grammar-registry.js:232:19)
    at Object.<anonymous> (/home/arcanis/cid/x.js:15:14)
    at Module._compile (module.js:571:32)
    at Object.Module._extensions..js (module.js:580:10)
    at Module.load (module.js:488:32)
    at tryModuleLoad (module.js:447:12)
    at Function.Module._load (module.js:439:3)
    at Module.runMain (module.js:605:10)
    at run (bootstrap_node.js:420:7)
    at startup (bootstrap_node.js:139:9)

Btw, why is that? Is it to avoid the need to store the scopes' enter tags at the beginning of each tokenized line? Because unless I'm mistaken, it also means that one has to iterate over every previous line in order to know what are the scopes for any random line.

50Wliu commented 7 years ago

Looking at the latest changes, it doesn't look like anything changed with regard to decodeTokens. Can you give the first version where the exception occurs?

arcanis commented 7 years ago

Yep I was looking at the blame and it seems to have been that way for quite a long time. Weird, I thought it wasn't required before .. :|

That being said, my point about the perf hit still stands, I think. Have I missed something? Is there a builtin way to decode tokens that doesn't require to decode every previous line? I looked at the Atom source code, and apparently the decode process is done manually, which might be why you've never experienced this issue before.

arcanis commented 7 years ago

Oooooh I think I got it. There's an extra undocumented field called openScopeTags inside the structure returned by first-mate.