Fontification doesn't work when buffer name starts with a space

[aaron-em]

Weird as it sounds, the issue title is an exact description of the problem. I've no idea why this should be, but I've observed it under emacs -q in both the 20150909 and 20150713 versions of js2-mode from MELPA stable.

Steps to reproduce:

• Open any Javascript file, doing M-x js2-mode RET if necessary
• Note that the buffer is fontified
• Rename the buffer to start with a space: M-: (rename-buffer " name with a leading space") RET
• Reinvoke js2-mode: M-x js2-mode RET
• Note that the pretty colors go away
• Rename the buffer again, removing the leading space
• Reinvoke js2-mode again
• Note that the pretty colors come back

Sorry not to be able to provide a more detailed report. I'm happy to assist to the best of my ability in investigating the issue, but I'm finding js2-mode's code to be rather above my level, so not sure how much help I can be.

[dgutov]

That's because of how font-lock-mode works. We apply highlighting using font-lock-face.

And buffers whose names start with a space are considered hidden, and font-lock-mode doesn't get enabled in them. Why would you need to use this kind of name?

[aaron-em]

Fair question. I'd really rather not, but the decision isn't mine: Org-mode's native fontification of source blocks happens, behind the scenes, in a buffer whose name starts with a space. These buffers have to persist beyond the function call that creates them, so with-temp-buffer isn't an option, and Org creates one of them per major mode used, so hiding them is preferable to having the user's buffer list cluttered with possibly quite a few of them.

Org gets around the behavior you mention by explicitly calling font-lock-fontify-buffer after it sets the buffer's major mode. In most modes, I guess because lack of fontification in hidden buffers is only by default, this works; unfortunately js2-mode is not such a mode.

On the other hand, having in the interim hacked org-src-font-lock-fontify-block not to use leading spaces in buffer names and tried again, I still didn't get my source blocks properly fontified; it looks to me as though this is a result of js2-mode's async reparse, which org doesn't know to wait around for. Unfortunately, I don't know of a way to have js2-mode do its parsing synchronously, so I don't suppose there is any benefit to a fix for the issue in any case - even afterward, js2-mode still wouldn't be usable with Org-mode source blocks.

[dgutov]

Hmm, so the problem is that js2-mode doesn't perform fontification even if font-lock-fontify-buffer is called, whereas other modes do.

There might be something we can do here, but is js2-mode really better than js-mode for small snippets of code, like the ones you see in Org source blocks? I'd think it would rather be more annoying, with highlighting too many "undeclared" variables, etc.

[aaron-em]

Not all Org-mode source blocks are small. For example, I discovered this issue in the process of using Org-mode to write a literate single-page web application in a single Org file. Some of my Javascript source blocks are quite sizable, and it's not just a question of display; js2-mode is IMO a lot more comfortable for editing than js-mode, and switching major modes in an Org Src buffer (what you get from C-c ' in a source block) hoses everything up. (As far as undeclared externs go, that's basically what js2-additional-externs is for, right?)

I freely concede that what I'm doing here is something of a stunt. But it'd still be nice to be able to do it with the benefit of js2-mode. :smile:

[dgutov]

As far as undeclared externs go, that's basically what js2-additional-externs is for, right?

Yeah, ok, that should work. But hopefully in the latest Emacs releases (e.g. the upcoming 25.1) js-mode works almost as well, aside from the lack of syntax checking.

Regarding asynchronous parsing, if you're not typing, it doesn't seem like it should enter the picture. js2-mode at the end of its major mode function calls js2-reparse; so js2-mode-idle-reparse, with its rescheduling, shouldn't be used either. Mayber org-src just needs to explicitly enable font-lock-mode in the src buffers, before calling font-lock-fontify-buffer?

[aaron-em]

So something I hadn't really noticed before is that org-src-font-lock-fontify-block doesn't actually call major mode functions in fontification buffers unless it notices (by comparing with major-mode) that a given buffer isn't already in the intended mode. I suppose this is meant to reduce fontification time when dealing with major modes which are expensive to invoke.

Anyway, by removing the leading space from the buffer name, and also replacing the conditional major mode function call with an unconditional one, I've been able to end up with a fontification buffer that actually gets fontified. So that's not actually a problem, it's just that I was misreading the org source, and I can work around the conditional call on my own.

All that leaves is the problem where even an explicit call to font-lock-fontify-buffer doesn't fontify a js2-mode buffer whose name starts with a space. From the sound of it, that's not really standard behavior for Emacs major modes and also not the intended behavior of js2-mode, so I think it's reasonable to call it a bug.

[dgutov]

I think it's reasonable to call it a bug.

Sure. It just might be non-trivial to fix it. But patches welcome.