Emacs long lines problem

[oatmealm]

I'm wondering if Emacs can handle the long line stored in ov-highlight-data by this mode in each file. I'm seeing serious performance problems when scrolling to the end of the buffer of past a certain point. I have `global-so-long-mode' enabled, but the problem persists...

[jkitchin]

I haven't seen any performance issues. I don't use this library alot though, so I probably haven't tested it for really long strings. if there is a problem, I would guess it is related to font-lock. if you open your file in fundamental mode, do you still see a problem? if so, it is probably related to the long line. If not, it would point to font lock I think.

[oatmealm]

I would guess it is related to font-lock. if you open your file in fundamental mode, do you still see a problem?

Assuming it is the scanning of the buffer and font-locking, is there a simple workaround for the problem? I'd still need to work in a font-locked, variable pitched buffers, often with bidi support enabled.

[jkitchin]

there might be a way to make it not use fontlock on that region. the suggestion is just to see if the performance problem goes away, not a permanent solution for working in.

looking at the code, it looks like that long line is in a comment, and it should be hidden in invisibility.

[jkitchin]

Do you have a file that has a lot of comments in it or something? It would be helpful to have some kind of example file to check performance on.

[oatmealm]

Sure. I'll upload as soon as I can. I use it freely to highlight, rather than add comments. Yet the 'ov-highlight-data' line can become quite long, and in my case in org-mode (in doom Emacs) it is fontified with the the comment font-face. There's no problem otherwise I believe.

[oatmealm]

Btw I did just notice that in org-mode the ov-highlight-data is visible. Does it suppose to respect 'display "..." I.e become invisible in the buffer?

[jkitchin]

it is pretty odd that you would see the whole line to me. For me I see

Do you see this in the messages anywhere?

File local-variables error: (void-function ov-highlight-load)

if so that means ov-highlight is not loaded. in that case you would see a long line. I would be surprised though if you also see the highlights in that case.

[oatmealm]

It does get loaded since I'm seeing all the highlights as you'd expect when re-opening a buffer that was marked up for highlights.

When org-mode is active the data is fully visible... the font-face data for this line is:

There are text properties here:
  face                 font-lock-comment-face
  font-lock-fontified  t
  fontified            t
  line-prefix          [Show]
  wrap-prefix          [Show]

vs. text mode enabled in same buffer

[jkitchin]

That doesn't seem right to me. That line should be hidden by an invisibility spec. I don't know why it would be visible in org-mode, and hidden in text mode. do you have any org-mode hooks that affect invisibility?

[oatmealm]

Hi, sorry for the delay. I'm not sure. It's quite a busy hook it seems, but the ones that seem to tweak the display, which I see, don't seem to apply to plain text, only src blocks, line numbers etc.

(er/add-org-mode-expansions
 org-appear-mode
 org-transclusion-mode
 nroam-setup-maybe
 writeroom-mode
 org-ref-org-menu
 evil-collection-mu4e-org-set-header-to-normal-mode
 +lookup--init-org-mode-handlers-h
 (closure
 (t)
 (&rest _)
 (add-hook 'before-save-hook 'org-encrypt-entries nil t))
 #[0 "\301\211\207"
    [imenu-create-index-function org-imenu-get-tree]
    2]
 #[0 "\300\301\302\303\304$\207"
    [add-hook change-major-mode-hook org-show-all append local]
    5]
 #[0 "\300\301\302\303\304$\207"
    [add-hook change-major-mode-hook org-babel-show-result-all append local]
    5]
 org-babel-result-hide-spec
 org-babel-hide-all-hashes
 mixed-pitch-mode
 doom--setq-display-line-numbers-for-org-mode-h
 doom--setq-scalable-fonts-allowed-for-org-mode-h
 doom--setq-line-spacing-for-org-mode-h
 doom--setq-bidi-paragraph-separate-re-for-org-mode-h
 doom--setq-bidi-paragraph-start-re-for-org-mode-h
 doom--setq-bidi-paragraph-direction-for-org-mode-h
 doom-disable-show-paren-mode-h
 doom-disable-show-trailing-whitespace-h
 +org-enable-auto-reformat-tables-h
 +org-enable-auto-update-cookies-h
 +org-make-last-point-visible-h
 evil-org-mode
 toc-org-enable
 embrace-org-mode-hook
 org-eldoc-load
 +literate-enable-recompile-h
 org-ref-setup-label-finders)

[oatmealm]

I've tried the following, with no success. I'm assuming that if (like it is in the case of Doom) buffer-invisibility-spec is not empty, than I need to add ov-highlight-data explictly (i.e. it's not enough that the text property was set to invisible but ov-highlight itself) . But it doesn't seem to work. What I'm not sure is what does add-to-invisibility-spec expects to get as a reference to the object to set invisible... is it `ov-highlight-data'?

(use-package! ov-highlight
  :init
  (map! :map org-mode-map
        :localleader
        :prefix ("_" . "highlight")
        "_" #'ov-highlight/body
        "k" #'ov-highlight-clear
        "b" #'ov-highlight-bold
        "f" #'ov-highlight/ov-highlight-foreground-and-exit)
  :config
  (defun ov-highlight-data-hide-spec ()
    "Hide overlight .
Add `ov-highlight-data' as an invisibility spec for hiding
portions of the encoded data."
    (add-to-invisibility-spec '(ov-highlight-data . t)))
  (add-hook! 'org-mode-hook 'ov-highlight-data-hide-spec))

I can verify it's working in the buffer by looking at buffer-invisibility-spec

((org-babel-hide-result . t)
 (ov-highlight-data . t)
 (org-hide-block . t)
 (org-link)
 (outline . t)
 t)

[jkitchin]

I don't know what the issue could be. ov-highlight should take care of making it invisible, and if it works in text mode but not in org-mode that is pretty suspicious that something funny is happening from some kind of package probably. I would consider checking with a minimal init file and if it works right there, try bisecting your init file.

there aren't too many good alternatives. You could look into splitting the long line into shorter lines. I don't know how tricky that would be to read in, it is probably doable. Alternatively, you could save them to an external file. both of these would require some changes to the code.

[oatmealm]

Well, just realized you're actually setting (add-to-invisibility-spec 'ov-highlight-data) in one of the hooks... thing is the variable did not include it until I forced it with my code...

[oatmealm]

I've switched to an optimized fork of org-mode, which removes fontification of folded content and other aspects the author thought were inefficient, and it seem to work for me so far, even though ov-highlight data still visible, though that's probably a Doom problem. Didn't get a chance to explore further. Closing to get this out of the way.

Thanks