Performance issue for JSON

Yevgnen commented 2 years ago

Hi. When I open a 3.5M JSON file using json-mode (not the built-in javascript-mode) with global-tree-sitter-mode and tree-sitter-hl-mode are on, emacs takes about 20s to finish. With both modes are off, the file is opened instantly.

My emacs version: GNU Emacs 27.2 (build 1, x86_64-apple-darwin21.2.0, Carbon Version 165 AppKit 2113.2) of 2022-01-25 and below is the profiling result

- command-execute                                               12055  99%
 - call-interactively                                           12055  99%
  - funcall-interactively                                       11811  97%
   - dired-find-file                                            11787  97%
    - find-file                                                 11786  97%
     - find-file-noselect                                       11786  97%
      - find-file-noselect-1                                    11784  97%
       - after-find-file                                        11777  97%
        - normal-mode                                           11771  97%
         - set-auto-mode                                        11766  97%
          - set-auto-mode-0                                     11766  97%
           - json-mode                                          11766  97%
            - run-mode-hooks                                    11766  97%
             - run-hooks                                        11765  97%
              - global-tree-sitter-mode-enable-in-buffers              11760  97%
               - turn-on-tree-sitter-mode                       11760  97%
                - tree-sitter-mode                              11760  97%
                 - tree-sitter--do-parse                        11760  97%
                  - tsc-parse-chunks                               45   0%
                     tsc--buffer-input                             45   0%
              - magit-auto-revert-mode-enable-in-buffers                  4   0%
               - magit-turn-on-auto-revert-mode-if-desired                  4   0%
                - magit-toplevel                                    2   0%
                 - magit-rev-parse-safe                             2   0%
                  - apply                                           2   0%
                   - magit-git-str                                  2   0%
                    - magit-process-git                             2   0%
                     - apply                                        2   0%
                      - magit-process-file                          2   0%
                       - apply                                      2   0%
                        - process-file                              2   0%
                           apply                                    2   0%
                - magit-file-tracked-p                              1   0%
                 - magit-git-success                                1   0%
                  - magit-git-exit-code                             1   0%
                   - magit-process-git                              1   0%
                    - apply                                         1   0%
                     - magit-process-file                           1   0%
                      - apply                                       1   0%
                       - process-file                               1   0%
                          apply                                     1   0%
                - auto-revert-mode                                  1   0%
                 - auto-revert-buffers                              1   0%
                  - apply                                           1   0%
                   - auto-revert-buffers--buffer-list-filter                  1   0%
                    - #<compiled 0x1ff842445e3d>                    1   0%
                     - auto-revert-notify-add-watch                  1   0%
                        file-notify-add-watch                       1   0%
              - envrc-global-mode-enable-in-buffers                  1   0%
               - envrc-mode                                         1   0%
                - envrc--update                                     1   0%
                 - envrc--find-env-dir                              1   0%
                    locate-dominating-file                          1   0%
             - hack-local-variables                                 1   0%
              - hack-dir-local-variables                            1   0%
               - dir-locals-find-file                               1   0%
                - locate-dominating-file                            1   0%
                   dir-locals--all-files                            1   0%
         - run-hooks                                                5   0%
          - magit-auto-revert-mode-enable-in-buffers                  4   0%
           - magit-turn-on-auto-revert-mode-if-desired                  4   0%
            - magit-toplevel                                        3   0%
             - magit-rev-parse-safe                                 3   0%
              - apply                                               3   0%
               - magit-git-str                                      3   0%
                - magit-process-git                                 3   0%
                 - apply                                            3   0%
                  - magit-process-file                              3   0%
                   - apply                                          3   0%
                    - process-file                                  3   0%
                       apply                                        3   0%
            - auto-revert-mode                                      1   0%
             - auto-revert-buffers                                  1   0%
              - apply                                               1   0%
               - auto-revert-buffers--buffer-list-filter                  1   0%
                  #<compiled 0x1ff842445e3d>                        1   0%
          - envrc-global-mode-enable-in-buffers                     1   0%
           - envrc-mode                                             1   0%
            - envrc--update                                         1   0%
             - envrc--find-env-dir                                  1   0%
                locate-dominating-file                              1   0%
        - run-hooks                                                 5   0%
         - projectile-find-file-hook-function                       5   0%
          - projectile-track-known-projects-find-file-hook                  2   0%
           - projectile-project-p                                   1   0%
            - projectile-project-root                               1   0%
             - cl-some                                              1   0%
              - #<compiled 0x1ff826696541>                          1   0%
               - file-truename                                      1   0%
                - file-truename                                     1   0%
                   file-truename                                    1   0%
           - projectile-add-known-project                           1   0%
            - projectile-merge-known-projects                       1   0%
             - projectile-save-known-projects                       1   0%
              - projectile-serialize                                1   0%
               - write-region                                       1   0%
                - select-safe-coding-system                         1   0%
                 - find-auto-coding                                 1   0%
                    auto-coding-alist-lookup                        1   0%
          - projectile-update-mode-line                             1   0%
           - projectile-default-mode-line                           1   0%
            - projectile-project-name                               1   0%
             - projectile-project-root                              1   0%
              - cl-some                                             1   0%
               - #<compiled 0x1ff82669648d>                         1   0%
                - file-truename                                     1   0%
                 - file-truename                                    1   0%
                  - file-truename                                   1   0%
                     file-truename                                  1   0%
          - projectile-cache-files-find-file-hook                   1   0%
           - projectile-cache-current-file                          1   0%
            - projectile-project-root                               1   0%
             - cl-some                                              1   0%
              - #<compiled 0x1ff8266964f9>                          1   0%
                 file-truename                                      1   0%
          - projectile-visit-project-tags-table                     1   0%
           - projectile-expand-root                                 1   0%
            - projectile-project-root                               1   0%
             - cl-some                                              1   0%
              - #<compiled 0x1ff82669672d>                          1   0%
               - file-truename                                      1   0%
                - file-truename                                     1   0%
                 - file-truename                                    1   0%
                  - file-truename                                   1   0%
                   - file-truename                                  1   0%
                      file-truename                                 1   0%
       - insert-file-contents                                       2   0%
        - set-auto-coding                                           2   0%
         - find-auto-coding                                         2   0%
            sgml-html-meta-auto-coding-function                     1   0%
      - file-truename                                               1   0%
       - file-truename                                              1   0%
        - file-truename                                             1   0%
           file-truename                                            1   0%
      - find-buffer-visiting                                        1   0%
       - file-truename                                              1   0%
        - file-truename                                             1   0%
           file-truename                                            1   0%
      dired-get-file-for-visit                                      1   0%
   - execute-extended-command                                      24   0%
    - sit-for                                                      23   0%
       redisplay                                                    5   0%
    - command-execute                                               1   0%
     - call-interactively                                           1   0%
      - funcall-interactively                                       1   0%
         profiler-report                                            1   0%
  - byte-code                                                     244   2%
   - read-extended-command                                        244   2%
    - completing-read                                             244   2%
     - selectrum-completing-read                                  244   2%
      - selectrum--read                                           244   2%
       - read-from-minibuffer                                     127   1%
        - selectrum--update                                        75   0%
         - selectrum--update-input-changed                         66   0%
          - selectrum--update-refined-candidates                   38   0%
           - prescient-filter                                      36   0%
            - prescient-filter-regexps                              1   0%
             - mapcar                                               1   0%
              - #<compiled 0x1ff8268ff8a5>                          1   0%
               - mapcar                                             1   0%
                - #<compiled 0x1ff8268ff911>                        1   0%
                   prescient-initials-regexp                        1   0%
          - selectrum--update-dynamic-candidates                   28   0%
           - selectrum--preprocess                                 19   0%
            - selectrum-prescient--preprocess                      19   0%
             - prescient-sort                                      19   0%
              - sort                                               16   0%
                 #<compiled 0x1ff84a593735>                        12   0%
           - selectrum--normalize-collection                        9   0%
            - all-completions                                       9   0%
             - #<compiled 0x449bc49b>                               9   0%
              - complete-with-action                                9   0%
               - all-completions                                    4   0%
                  #<compiled 0x1ff84a59370d>                        4   0%
         - selectrum--insert-candidates                             9   0%
          - selectrum--vertical-display-style                       9   0%
           - selectrum--affixate                                    7   0%
            - #<compiled 0x1ff84a593855>                            7   0%
             - apply                                                7   0%
              - marginalia--affixate                                7   0%
               - marginalia--cached                                 7   0%
                - marginalia-annotate-command                       7   0%
                   marginalia-annotate-binding                      4   0%
                   marginalia--function-doc                         3   0%
             apply                                                  1   0%
           - completion-metadata-get                                1   0%
            - apply                                                 1   0%
             - marginalia--completion-metadata-get                  1   0%
              - completion-metadata-get                             1   0%
               - apply                                              1   0%
                - marginalia--completion-metadata-get                  1   0%
                 - run-hook-with-args-until-success                  1   0%
                    marginalia-classify-by-prompt                   1   0%
+ redisplay_internal (C function)                                  22   0%
+ timer-event-handler                                               3   0%
+ ...                                                               0   0%

ubolonton commented 2 years ago

Please provide more details for debugging. For example:

The JSON file with the issue.
Major and minor modes used in the buffer.
Minimal config to reproduce the issue, if possible.

Yevgnen commented 2 years ago

Hi, the issue seems happens for file with JSON lines, re-dumping the file to a list of dict seems kinda fixed the problem: Opening the file is fast, but scrolling is much slower than with tree-sitter-hl-mode off.

The JSON file is https://github.com/CLUEbenchmark/SimCLUE/blob/main/datasets/test_public.json
Major mode: json-mode
MInor modes: Enabled minor modes: Auto-Composition Auto-Compression Auto-Encryption Blink-Cursor Electric-Indent File-Name-Shadow Font-Lock Global-Eldoc Global-Font-Lock Global-Tree-Sitter Line-Number Mac-Mouse-Wheel Menu-Bar Tool-Bar Tooltip Transient-Mark Tree-Sitter Tree-Sitter-Hl

The minimal config is


(setq user-init-file (or load-file-name (buffer-file-name)))
(setq user-emacs-directory (file-name-directory user-init-file))

(require 'package) (add-to-list 'package-archives '("tromey" . "http://tromey.com/elpa/")) (add-to-list 'package-archives '("melpa" . "http://melpa.org/packages/") t) (setq package-user-dir (expand-file-name "elpa/" user-emacs-directory)) (package-initialize)

;; Install use-package that we require for managing all other dependencies (unless (package-installed-p 'use-package) (package-refresh-contents) (package-install 'use-package))

(use-package json-mode :ensure t)

(use-package tree-sitter :ensure t :init (require 'tree-sitter) (global-tree-sitter-mode 1) (add-hook 'tree-sitter-after-on-hook #'tree-sitter-hl-mode))

;; Language bundle. (use-package tree-sitter-langs :ensure t :init (require 'tree-sitter-langs))

ag91 commented 2 years ago

@Yevgnen do you get slowness also while updating the file? I am using https://github.com/milisims/tree-sitter-org in some large org files and I observe a significant slow down in Org Mode. Maybe these issues are related.

emacs-tree-sitter / elisp-tree-sitter

Performance issue for JSON #204