search

kaushalmodi / ox-hugo

A carefully crafted Org exporter back-end for Hugo
https://ox-hugo.scripter.co
GNU General Public License v3.0
882 stars 132 forks source link

Post exports hang or take a very long time when single org file gets large #487

Closed kishvanchee closed 2 years ago

kishvanchee commented 2 years ago

Actual Behavior

The export is slow at times when I'm exporting the posts recently. This also takes up a lot of CPU (a few times my entire system hung up cos emacs took a lot of CPU). I have even had emacs hang after a point. Is this an expected behaviour? For context, I have around 100 or so org-subtrees (posts) and many of them are drafts (TODO / DRAFT). The no of lines of the org file comes close to 3k. I am using one post per org-subtree since that's the recommended method.

Screenshot_20211230_182715

Is this a bug with ox-hugo / org-exporter? Anything I'm missing? Would splitting my posts into separate files be the solution here? If you have examples (sites) you know of who have large number of separate org files, that'd also be helpful.

I see this has been discussed here but no concrete answer on the scalability as such has been provided https://github.com/kaushalmodi/ox-hugo/issues/428 Just linking it here for reference

Expected Behavior

I would expect a smooth export for every export I do.

How to Reproduce the Issue

Not exactly sure how to reproduce this, it's just a big org file. I've noticed this only recently. I didn't have this issue previously.

Ox-Hugo Debug Information

Debug Info # Debug information for `ox-hugo` ## Emacs Version GNU Emacs 27.2 (build 1, x86_64-pc-linux-gnu, GTK+ Version 3.24.27, cairo version 1.17.4) of 2021-03-26 ## Org Version Org mode version 9.6 (9.6-??-e0b1a05 @ /home/kishore/.emacs.d/.local/straight/build-27.2/org/) ## Hugo Version hugo v0.91.2+extended linux/amd64 BuildDate=unknown ### Org `load-path` shadows **Warning**: Possible mixed installation of Org /home/kishore/.emacs.d/.local/straight/build-27.2/org/ob-C hides /usr/share/emacs/27.2/lisp/org/ob-C /home/kishore/.emacs.d/.local/straight/build-27.2/org/ob-forth hides /usr/share/emacs/27.2/lisp/org/ob-forth /home/kishore/.emacs.d/.local/straight/build-27.2/org/ob-core hides /usr/share/emacs/27.2/lisp/org/ob-core /home/kishore/.emacs.d/.local/straight/build-27.2/org/ob-dot hides /usr/share/emacs/27.2/lisp/org/ob-dot /home/kishore/.emacs.d/.local/straight/build-27.2/org/ob-fortran hides /usr/share/emacs/27.2/lisp/org/ob-fortran /home/kishore/.emacs.d/.local/straight/build-27.2/org/ob-octave hides /usr/share/emacs/27.2/lisp/org/ob-octave /home/kishore/.emacs.d/.local/straight/build-27.2/org/ob-lua hides /usr/share/emacs/27.2/lisp/org/ob-lua /home/kishore/.emacs.d/.local/straight/build-27.2/org/ob-groovy hides /usr/share/emacs/27.2/lisp/org/ob-groovy /home/kishore/.emacs.d/.local/straight/build-27.2/org/org-colview hides /usr/share/emacs/27.2/lisp/org/org-colview /home/kishore/.emacs.d/.local/straight/build-27.2/org/ob-comint hides /usr/share/emacs/27.2/lisp/org/ob-comint /home/kishore/.emacs.d/.local/straight/build-27.2/org/ob-ocaml hides /usr/share/emacs/27.2/lisp/org/ob-ocaml /home/kishore/.emacs.d/.local/straight/build-27.2/org/ol-bbdb hides /usr/share/emacs/27.2/lisp/org/ol-bbdb /home/kishore/.emacs.d/.local/straight/build-27.2/org/ob-org hides /usr/share/emacs/27.2/lisp/org/ob-org /home/kishore/.emacs.d/.local/straight/build-27.2/org/ob-exp hides /usr/share/emacs/27.2/lisp/org/ob-exp /home/kishore/.emacs.d/.local/straight/build-27.2/org/org-indent hides /usr/share/emacs/27.2/lisp/org/org-indent /home/kishore/.emacs.d/.local/straight/build-27.2/org/org-duration hides /usr/share/emacs/27.2/lisp/org/org-duration /home/kishore/.emacs.d/.local/straight/build-27.2/org/org hides /usr/share/emacs/27.2/lisp/org/org /home/kishore/.emacs.d/.local/straight/build-27.2/org/org-attach hides /usr/share/emacs/27.2/lisp/org/org-attach /home/kishore/.emacs.d/.local/straight/build-27.2/org/ob-emacs-lisp hides /usr/share/emacs/27.2/lisp/org/ob-emacs-lisp /home/kishore/.emacs.d/.local/straight/build-27.2/org/ox-icalendar hides /usr/share/emacs/27.2/lisp/org/ox-icalendar /home/kishore/.emacs.d/.local/straight/build-27.2/org/ob-screen hides /usr/share/emacs/27.2/lisp/org/ob-screen /home/kishore/.emacs.d/.local/straight/build-27.2/org/org-compat hides /usr/share/emacs/27.2/lisp/org/org-compat /home/kishore/.emacs.d/.local/straight/build-27.2/org/ob-sql hides /usr/share/emacs/27.2/lisp/org/ob-sql /home/kishore/.emacs.d/.local/straight/build-27.2/org/ob-scheme hides /usr/share/emacs/27.2/lisp/org/ob-scheme /home/kishore/.emacs.d/.local/straight/build-27.2/org/org-faces hides /usr/share/emacs/27.2/lisp/org/org-faces /home/kishore/.emacs.d/.local/straight/build-27.2/org/ob-ruby hides /usr/share/emacs/27.2/lisp/org/ob-ruby /home/kishore/.emacs.d/.local/straight/build-27.2/org/org-table hides /usr/share/emacs/27.2/lisp/org/org-table /home/kishore/.emacs.d/.local/straight/build-27.2/org/org-timer hides /usr/share/emacs/27.2/lisp/org/org-timer /home/kishore/.emacs.d/.local/straight/build-27.2/org/org-list hides /usr/share/emacs/27.2/lisp/org/org-list /home/kishore/.emacs.d/.local/straight/build-27.2/org/ob hides /usr/share/emacs/27.2/lisp/org/ob /home/kishore/.emacs.d/.local/straight/build-27.2/org/ox-md hides /usr/share/emacs/27.2/lisp/org/ox-md /home/kishore/.emacs.d/.local/straight/build-27.2/org/org-src hides /usr/share/emacs/27.2/lisp/org/org-src /home/kishore/.emacs.d/.local/straight/build-27.2/org/ol-docview hides /usr/share/emacs/27.2/lisp/org/ol-docview /home/kishore/.emacs.d/.local/straight/build-27.2/org/org-clock hides /usr/share/emacs/27.2/lisp/org/org-clock /home/kishore/.emacs.d/.local/straight/build-27.2/org/ox-org hides /usr/share/emacs/27.2/lisp/org/ox-org /home/kishore/.emacs.d/.local/straight/build-27.2/org/ox-beamer hides /usr/share/emacs/27.2/lisp/org/ox-beamer /home/kishore/.emacs.d/.local/straight/build-27.2/org/ox-html hides /usr/share/emacs/27.2/lisp/org/ox-html /home/kishore/.emacs.d/.local/straight/build-27.2/org/ox-ascii hides /usr/share/emacs/27.2/lisp/org/ox-ascii /home/kishore/.emacs.d/.local/straight/build-27.2/org/ob-latex hides /usr/share/emacs/27.2/lisp/org/ob-latex /home/kishore/.emacs.d/.local/straight/build-27.2/org/ob-sed hides /usr/share/emacs/27.2/lisp/org/ob-sed /home/kishore/.emacs.d/.local/straight/build-27.2/org/org-loaddefs hides /usr/share/emacs/27.2/lisp/org/org-loaddefs /home/kishore/.emacs.d/.local/straight/build-27.2/org/ox-odt hides /usr/share/emacs/27.2/lisp/org/ox-odt /home/kishore/.emacs.d/.local/straight/build-27.2/org/org-protocol hides /usr/share/emacs/27.2/lisp/org/org-protocol /home/kishore/.emacs.d/.local/straight/build-27.2/org/org-crypt hides /usr/share/emacs/27.2/lisp/org/org-crypt /home/kishore/.emacs.d/.local/straight/build-27.2/org/ob-tangle hides /usr/share/emacs/27.2/lisp/org/ob-tangle /home/kishore/.emacs.d/.local/straight/build-27.2/org/org-footnote hides /usr/share/emacs/27.2/lisp/org/org-footnote /home/kishore/.emacs.d/.local/straight/build-27.2/org/ol-rmail hides /usr/share/emacs/27.2/lisp/org/ol-rmail /home/kishore/.emacs.d/.local/straight/build-27.2/org/org-feed hides /usr/share/emacs/27.2/lisp/org/org-feed /home/kishore/.emacs.d/.local/straight/build-27.2/org/org-capture hides /usr/share/emacs/27.2/lisp/org/org-capture /home/kishore/.emacs.d/.local/straight/build-27.2/org/org-macs hides /usr/share/emacs/27.2/lisp/org/org-macs /home/kishore/.emacs.d/.local/straight/build-27.2/org/ob-eval hides /usr/share/emacs/27.2/lisp/org/ob-eval /home/kishore/.emacs.d/.local/straight/build-27.2/org/ob-processing hides /usr/share/emacs/27.2/lisp/org/ob-processing /home/kishore/.emacs.d/.local/straight/build-27.2/org/ol hides /usr/share/emacs/27.2/lisp/org/ol /home/kishore/.emacs.d/.local/straight/build-27.2/org/ox-texinfo hides /usr/share/emacs/27.2/lisp/org/ox-texinfo /home/kishore/.emacs.d/.local/straight/build-27.2/org/ob-clojure hides /usr/share/emacs/27.2/lisp/org/ob-clojure /home/kishore/.emacs.d/.local/straight/build-27.2/org/org-entities hides /usr/share/emacs/27.2/lisp/org/org-entities /home/kishore/.emacs.d/.local/straight/build-27.2/org/ol-mhe hides /usr/share/emacs/27.2/lisp/org/ol-mhe /home/kishore/.emacs.d/.local/straight/build-27.2/org/ox-man hides /usr/share/emacs/27.2/lisp/org/ox-man /home/kishore/.emacs.d/.local/straight/build-27.2/org/ob-haskell hides /usr/share/emacs/27.2/lisp/org/ob-haskell /home/kishore/.emacs.d/.local/straight/build-27.2/org/ol-eww hides /usr/share/emacs/27.2/lisp/org/ol-eww /home/kishore/.emacs.d/.local/straight/build-27.2/org/org-macro hides /usr/share/emacs/27.2/lisp/org/org-macro /home/kishore/.emacs.d/.local/straight/build-27.2/org/org-datetree hides /usr/share/emacs/27.2/lisp/org/org-datetree /home/kishore/.emacs.d/.local/straight/build-27.2/org/ob-js hides /usr/share/emacs/27.2/lisp/org/ob-js /home/kishore/.emacs.d/.local/straight/build-27.2/org/ob-sqlite hides /usr/share/emacs/27.2/lisp/org/ob-sqlite /home/kishore/.emacs.d/.local/straight/build-27.2/org/ol-irc hides /usr/share/emacs/27.2/lisp/org/ol-irc /home/kishore/.emacs.d/.local/straight/build-27.2/org/ob-css hides /usr/share/emacs/27.2/lisp/org/ob-css /home/kishore/.emacs.d/.local/straight/build-27.2/org/ob-maxima hides /usr/share/emacs/27.2/lisp/org/ob-maxima /home/kishore/.emacs.d/.local/straight/build-27.2/org/org-mobile hides /usr/share/emacs/27.2/lisp/org/org-mobile /home/kishore/.emacs.d/.local/straight/build-27.2/org/org-mouse hides /usr/share/emacs/27.2/lisp/org/org-mouse /home/kishore/.emacs.d/.local/straight/build-27.2/org/org-goto hides /usr/share/emacs/27.2/lisp/org/org-goto /home/kishore/.emacs.d/.local/straight/build-27.2/org/ob-python hides /usr/share/emacs/27.2/lisp/org/ob-python /home/kishore/.emacs.d/.local/straight/build-27.2/org/ob-calc hides /usr/share/emacs/27.2/lisp/org/ob-calc /home/kishore/.emacs.d/.local/straight/build-27.2/org/org-agenda hides /usr/share/emacs/27.2/lisp/org/org-agenda /home/kishore/.emacs.d/.local/straight/build-27.2/org/org-habit hides /usr/share/emacs/27.2/lisp/org/org-habit /home/kishore/.emacs.d/.local/straight/build-27.2/org/ob-sass hides /usr/share/emacs/27.2/lisp/org/ob-sass /home/kishore/.emacs.d/.local/straight/build-27.2/org/ob-lilypond hides /usr/share/emacs/27.2/lisp/org/ob-lilypond /home/kishore/.emacs.d/.local/straight/build-27.2/org/ob-lob hides /usr/share/emacs/27.2/lisp/org/ob-lob /home/kishore/.emacs.d/.local/straight/build-27.2/org/ol-w3m hides /usr/share/emacs/27.2/lisp/org/ol-w3m /home/kishore/.emacs.d/.local/straight/build-27.2/org/ol-eshell hides /usr/share/emacs/27.2/lisp/org/ol-eshell /home/kishore/.emacs.d/.local/straight/build-27.2/org/ob-eshell hides /usr/share/emacs/27.2/lisp/org/ob-eshell /home/kishore/.emacs.d/.local/straight/build-27.2/org/ob-plantuml hides /usr/share/emacs/27.2/lisp/org/ob-plantuml /home/kishore/.emacs.d/.local/straight/build-27.2/org/ob-R hides /usr/share/emacs/27.2/lisp/org/ob-R /home/kishore/.emacs.d/.local/straight/build-27.2/org/org-lint hides /usr/share/emacs/27.2/lisp/org/org-lint /home/kishore/.emacs.d/.local/straight/build-27.2/org/org-element hides /usr/share/emacs/27.2/lisp/org/org-element /home/kishore/.emacs.d/.local/straight/build-27.2/org/ob-shell hides /usr/share/emacs/27.2/lisp/org/ob-shell /home/kishore/.emacs.d/.local/straight/build-27.2/org/ob-lisp hides /usr/share/emacs/27.2/lisp/org/ob-lisp /home/kishore/.emacs.d/.local/straight/build-27.2/org/ol-bibtex hides /usr/share/emacs/27.2/lisp/org/ol-bibtex /home/kishore/.emacs.d/.local/straight/build-27.2/org/ox hides /usr/share/emacs/27.2/lisp/org/ox /home/kishore/.emacs.d/.local/straight/build-27.2/org/ob-ditaa hides /usr/share/emacs/27.2/lisp/org/ob-ditaa /home/kishore/.emacs.d/.local/straight/build-27.2/org/ol-gnus hides /usr/share/emacs/27.2/lisp/org/ol-gnus /home/kishore/.emacs.d/.local/straight/build-27.2/org/ob-makefile hides /usr/share/emacs/27.2/lisp/org/ob-makefile /home/kishore/.emacs.d/.local/straight/build-27.2/org/org-plot hides /usr/share/emacs/27.2/lisp/org/org-plot /home/kishore/.emacs.d/.local/straight/build-27.2/org/ol-info hides /usr/share/emacs/27.2/lisp/org/ol-info /home/kishore/.emacs.d/.local/straight/build-27.2/org/ob-perl hides /usr/share/emacs/27.2/lisp/org/ob-perl /home/kishore/.emacs.d/.local/straight/build-27.2/org/org-pcomplete hides /usr/share/emacs/27.2/lisp/org/org-pcomplete /home/kishore/.emacs.d/.local/straight/build-27.2/org/org-keys hides /usr/share/emacs/27.2/lisp/org/org-keys /home/kishore/.emacs.d/.local/straight/build-27.2/org/org-archive hides /usr/share/emacs/27.2/lisp/org/org-archive /home/kishore/.emacs.d/.local/straight/build-27.2/org/ob-awk hides /usr/share/emacs/27.2/lisp/org/ob-awk /home/kishore/.emacs.d/.local/straight/build-27.2/org/org-inlinetask hides /usr/share/emacs/27.2/lisp/org/org-inlinetask /home/kishore/.emacs.d/.local/straight/build-27.2/org/ox-publish hides /usr/share/emacs/27.2/lisp/org/ox-publish /home/kishore/.emacs.d/.local/straight/build-27.2/org/ob-java hides /usr/share/emacs/27.2/lisp/org/ob-java /home/kishore/.emacs.d/.local/straight/build-27.2/org/org-id hides /usr/share/emacs/27.2/lisp/org/org-id /home/kishore/.emacs.d/.local/straight/build-27.2/org/org-num hides /usr/share/emacs/27.2/lisp/org/org-num /home/kishore/.emacs.d/.local/straight/build-27.2/org/org-ctags hides /usr/share/emacs/27.2/lisp/org/org-ctags /home/kishore/.emacs.d/.local/straight/build-27.2/org/ob-table hides /usr/share/emacs/27.2/lisp/org/ob-table /home/kishore/.emacs.d/.local/straight/build-27.2/org/org-refile hides /usr/share/emacs/27.2/lisp/org/org-refile /home/kishore/.emacs.d/.local/straight/build-27.2/org/ox-latex hides /usr/share/emacs/27.2/lisp/org/ox-latex /home/kishore/.emacs.d/.local/straight/build-27.2/org/ob-matlab hides /usr/share/emacs/27.2/lisp/org/ob-matlab /home/kishore/.emacs.d/.local/straight/build-27.2/org/org-attach-git hides /usr/share/emacs/27.2/lisp/org/org-attach-git /home/kishore/.emacs.d/.local/straight/build-27.2/org/org-tempo hides /usr/share/emacs/27.2/lisp/org/org-tempo /home/kishore/.emacs.d/.local/straight/build-27.2/org/ob-ref hides /usr/share/emacs/27.2/lisp/org/ob-ref /home/kishore/.emacs.d/.local/straight/build-27.2/org/org-install hides /usr/share/emacs/27.2/lisp/org/org-install /home/kishore/.emacs.d/.local/straight/build-27.2/org/ob-gnuplot hides /usr/share/emacs/27.2/lisp/org/ob-gnuplot /home/kishore/.emacs.d/.local/straight/build-27.2/org/org-version hides /usr/share/emacs/27.2/lisp/org/org-version Study the output of `M-x list-load-path-shadows`. ## `ox-hugo` defcustoms
org-hugo-section “posts”
org-hugo-use-code-for-kbd nil
org-hugo-preserve-filling t
org-hugo-delete-trailing-ws t
org-hugo-prefer-hyphen-in-tags t
org-hugo-allow-spaces-in-tags t
org-hugo-tag-processing-functions (org-hugo–tag-processing-fn-replace-with-spaces-maybe org-hugo–tag-processing-fn-replace-with-hyphens-maybe)
org-hugo-auto-set-lastmod nil
org-hugo-export-with-toc nil
org-hugo-export-with-section-numbers nil
org-hugo-front-matter-format “toml”
org-hugo-default-static-subdirectory-for-externals “ox-hugo”
org-hugo-external-file-extensions-allowed-for-copying (“jpg” “jpeg” “tiff” “png” “svg” “gif” “mp4” “pdf” “odt” “doc” “ppt” “xls” “docx” “pptx” “xlsx”)
org-hugo-date-format “%Y-%m-%dT%T%z”
org-hugo-paired-shortcodes “”
org-hugo-langs-no-descr-in-code-fences nil
org-hugo-suppress-lastmod-period 0.0
org-hugo-front-matter-format “toml”
kaushalmodi commented 2 years ago

Does it hang when:

I see your Org version as 9.6. So that's the Org main branch version. Does this issue happen on the Org stable or bugfix branch?

Was this issue happening on emacs 26.x? Or did you start seeing it only on emacs 27.x (did you upgrade to emacs 27.x recently?)

I am just trying to understand what change caused this issue. If you think of the time when the export used to work fine, what were the emacs and Org versions and how much time back was that (so that we can estimate the version/commit of ox-hugo then). Was it the same OS or a different one?

Here's how you can get more debug info:

If you can share your Org file, I can attempt to reproduce the issue.

kaushalmodi commented 2 years ago

Also, does emacs lock up permanently? (That should never happens and possibly points to some bug in emacs)

Or does it hang only for few seconds (15 seconds)? If you are exporting dozens of posts, it could take that long, but not if you exporting only 1 post (current subtree), regardless of the total size of the Org file.

kishvanchee commented 2 years ago

I should have clarified. I always do C-c C-e H H to export only the subtree. I have exported all posts before but that was like around 10-15 or so posts which took less than a minute. I don't do that now, I just tried that out initially because I was curious.

I do not have org-hugo-auto-export-mode enabled. I wasn't aware of the feature. I don't think I'd be enabling it now either.

I see your Org version as 9.6. So that's the Org main branch version. Does this issue happen on the Org stable or bugfix branch?

I'm not sure about this. I would have to try it out and let you know. I currently use doom-emacs for my config so I'd need to configure this accordingly.

Was this issue happening on emacs 26.x? Or did you start seeing it only on emacs 27.x (did you upgrade to emacs 27.x recently?)

I don't think it happened in emacs 26.x. I updated to emacs 27.x as soon as it was available in the arch linux repositories. So my observation is restricted to 27.x. I've been using this version for close to a year now.

I am just trying to understand what change caused this issue. If you think of the time when the export used to work fine, what were the emacs and Org versions and how much time back was that (so that we can estimate the version/commit of ox-hugo then). Was it the same OS or a different one?

Looking through my git log for my blog, I think the last working version is dated Oct 25 2021. Let me link some issues I read up on when I upgraded my doom emacs and the configuration. You had also commented on one of them. OS config has not changed.

This https://github.com/hlissner/doom-emacs/issues/5721 is the issue I came across when I hit the error org-element--cache-sync: Symbol’s function definition is void: backtrace-get-frames. I ended up doing unpin! org which solved that issue. Here's https://github.com/nobiot/org-transclusion/issues/109 another related issue linked from there. At the same time I faced this issue https://github.com/hlissner/doom-emacs/issues/5913 as well which was also solved by unpin! org.

Also, does emacs lock up permanently? (That should never happens and possibly points to some bug in emacs)

This happens when I try to save the file. Not when I export. Export happens successfully for a subtree but takes a very very long time (> 1-2 mins)

I passed pkill -USR2 emacs and it entered the debug trace. Here's the paste for that. This is for when emacs hangs on save.

Debugger entered--entering a function:
* #f(compiled-function () #<bytecode 0x155dc6088329>)()
  org-element--parse-to(4743)
  org-element--cache-verify-element((paragraph (:begin 4743 :end 4774 :contents-begin 4743 :contents-end 4774 :post-blank 0 :post-affiliated 4743 :mode nil :granularity element :cached t :parent (section (:begin 3984 :end 4774 :contents-begin 3984 :contents-end 4774 :robust-begin 3984 :robust-end 4772 :post-blank 0 :post-affiliated 3984 :mode section :granularity element :cached t :parent (headline (:raw-value "Course Review: UCB CS61A" :begin 3916 :end 4774 :pre-blank 0 :contents-begin 3984 :contents-end 4774 :robust-begin 4119 :robust-end 4772 :level 2 :priority nil :tags (... ...) :todo-keyword #("DRAFT" 0 5 ...) :todo-type todo :post-blank 0 :footnote-section-p nil :archivedp nil :commentedp nil :post-affiliated 3916 :EXPORT_FILE_NAME "course-review-ucb-cs61a" :EXPORT_HUGO_CUSTOM_FRONT_MATTER ":series \"100 Days to Offload challenge\"" :title "Course Review: UCB CS61A" :mode nil :granularity element :org-element--cache-sync-key (35 . 3504) :cached t ...)) :org-element--cache-sync-key (35 . 3983))) :org-element--cache-sync-key (35 . 4743))))
  org-element-at-point(4774)
  org-element-cache-map(ignore :granularity headline+inlinetask)
  org-element-cache-map(#f(compiled-function (el) #<bytecode 0x155dbd1a1939>) :next-re "^\\*+ " :fail-re "^\\*+ " :narrow t)
  org-scan-tags(org-encrypt-entry (lambda (todo tags-list level) (progn (setq org-cached-props nil) (or (and (member "crypt" tags-list))))) nil)
  org-encrypt-entries()
  funcall(org-encrypt-entries)
  (condition-case e (funcall hook) ((debug error) (signal 'doom-hook-error (list hook e))))
  doom-run-hook(org-encrypt-entries)
  run-hook-wrapped(doom-run-hook org-encrypt-entries)
  (condition-case e (run-hook-wrapped hook #'doom-run-hook) ((debug doom-hook-error) (if debug-on-error nil (lwarn hook :error "Error running hook %S because: %s" (if (symbolp (car (cdr e))) (symbol-name (car (cdr e))) (car (cdr e))) (car (cdr (cdr e))))) (signal 'doom-hook-error (cons hook (cdr e)))))
  (let ((hook (car --dolist-tail--))) (condition-case e (run-hook-wrapped hook #'doom-run-hook) ((debug doom-hook-error) (if debug-on-error nil (lwarn hook :error "Error running hook %S because: %s" (if (symbolp (car ...)) (symbol-name (car ...)) (car (cdr e))) (car (cdr (cdr e))))) (signal 'doom-hook-error (cons hook (cdr e))))) (setq --dolist-tail-- (cdr --dolist-tail--)))
  (while --dolist-tail-- (let ((hook (car --dolist-tail--))) (condition-case e (run-hook-wrapped hook #'doom-run-hook) ((debug doom-hook-error) (if debug-on-error nil (lwarn hook :error "Error running hook %S because: %s" (if (symbolp ...) (symbol-name ...) (car ...)) (car (cdr ...)))) (signal 'doom-hook-error (cons hook (cdr e))))) (setq --dolist-tail-- (cdr --dolist-tail--))))
  (let ((--dolist-tail-- hooks)) (while --dolist-tail-- (let ((hook (car --dolist-tail--))) (condition-case e (run-hook-wrapped hook #'doom-run-hook) ((debug doom-hook-error) (if debug-on-error nil (lwarn hook :error "Error running hook %S because: %s" (if ... ... ...) (car ...))) (signal 'doom-hook-error (cons hook (cdr e))))) (setq --dolist-tail-- (cdr --dolist-tail--)))))
  doom-run-hooks(before-save-hook)
  apply(doom-run-hooks before-save-hook)
  run-hooks(before-save-hook)
  basic-save-buffer(t)
  save-buffer(1)
  funcall-interactively(save-buffer 1)
  call-interactively(save-buffer nil nil)
  command-execute(save-buffer)

I passed pkill -USR2 emacs and it entered the debug trace. Here's the paste for that. This is for when emacs hangs on export.

Debugger entered--Lisp error: (quit)
  format("%S" #s(avl-tree- :dummyroot [[[[[[... ... ... 0] [... ... ... 0] (headline ...) 0] [[... ... ... 0] [... ... ... 0] (headline ...) 0] (headline (:raw-value "Tools I use" :begin 12683 :end 13000 :pre-blank 0 :contents-begin 12703 :contents-end 13000 :robust-begin 12752 :robust-end 12998 :level 2 :priority nil :tags nil :todo-keyword ... :todo-type todo :post-blank 0 :footnote-section-p nil :archivedp nil :commentedp nil :post-affiliated 12683 :EXPORT_FILE_NAME "tools-i-use" :title "Tools I use" :mode nil :granularity element :cached t :parent ... :org-element--cache-sync-key nil)) 0] [[[... ... ... 0] [... ... ... 0] (headline ...) 0] [[... ... ... 0] [... ... ... 1] (headline ...) -1] (headline (:raw-value "Learning The Art of Computer Programming" :begin 39962 :end 40122 :pre-blank 0 :contents-begin 40043 :contents-end 40122 :robust-begin nil :robust-end nil :level 2 :priority nil :tags ... :todo-keyword ... :todo-type todo :post-blank 0 :footnote-section-p nil :archivedp nil :commentedp nil :post-affiliated 39962 :EXPORT_FILE_NAME "learning-the-art-of-computer-programming" :title "Learning The Art of Computer Programming" :mode nil :granularity element :cached t :parent ... :org-element--cache-sync-key nil)) 0] (headline (:raw-value "Comparing CSAPP and COA (Hennessey Patterson)" :begin 23190 :end 24083 :pre-blank 0 :contents-begin 23260 :contents-end 24083 :robust-begin 23342 :robust-end 24081 :level 2 :priority nil :tags (...) :todo-keyword #("TODO" 0 4 ...) :todo-type todo :post-blank 0 :footnote-section-p nil :archivedp nil :commentedp nil :post-affiliated 23190 :EXPORT_FILE_NAME "comparing-csapp-and-coa--hennessey-patterson" :title "Comparing CSAPP and COA (Hennessey Patterson)" :mode nil :granularity element :cached t :parent (headline ...) :org-element--cache-sync-key nil)) 0] [[[[... ... ... 0] [... ... ... -1] (headline ...) 0] [[... ... ... 0] [... ... ... 0] (headline ...) 0] (headline (:raw-value "Resume in orgmode" :begin 70673 :end 76445 :pre-blank 0 :contents-begin 70743 :contents-end 76445 :robust-begin 70914 :robust-end 76443 :level 2 :priority nil :tags ... :todo-keyword ... :todo-type done :post-blank 0 :footnote-section-p nil :archivedp nil :commentedp nil :post-affiliated 70673 :closed ... :EXPORT_FILE_NAME "resume-in-orgmode" :EXPORT_HUGO_CUSTOM_FRONT_MATTER ":series \"100 Days to Offload challenge\" :code true" :title "Resume in orgmode" :mode nil :granularity element :cached t ...)) -1] [[[... ... ... 0] [... ... ... 0] (headline ...) 0] [[... ... ... 0] [... ... ... 0] (headline ...) 1] (headline (:raw-value "Resume in yaml and jinja templates" :begin 98237 :end 103692 :pre-blank 0 :contents-begin 98322 :contents-end 103692 :robust-begin 98510 :robust-end 103690 :level 2 :priority nil :tags ... :todo-keyword ... :todo-type done :post-blank 0 :footnote-section-p nil :archivedp nil :commentedp nil :post-affiliated 98237 :closed ... :EXPORT_FILE_NAME "resume-in-yaml-and-jinja-templates" :EXPORT_HUGO_CUSTOM_FRONT_MATTER ":series \"100 Days to Offload challenge\" :code true" :title "Resume in yaml and jinja templates" :mode nil :granularity element :cached t ...)) 0] (headline (:raw-value "Disc Math" :begin 80588 :end 80637 :pre-blank 0 :contents-begin 80602 :contents-end 80637 :robust-begin 80604 :robust-end 80635 :level 3 :priority nil :tags nil :todo-keyword nil :todo-type nil :post-blank 0 :footnote-section-p nil :archivedp nil :commentedp nil :post-affiliated 80588 :title "Disc Math" :mode nil :granularity element :cached t :parent (headline ...) :org-element--cache-sync-key (3 . 80588))) -1] (headline (:raw-value "How to procrastinate" :begin 55270 :end 55358 :pre-blank 0 :contents-begin 55299 :contents-end 55358 :robust-begin nil :robust-end nil :level 2 :priority nil :tags nil :todo-keyword #("TODO" 0 4 (face ... wrap-prefix ... line-prefix ... ws-butler-chg chg fontified t)) :todo-type todo :post-blank 0 :footnote-section-p nil :archivedp nil :commentedp nil :post-affiliated 55270 :EXPORT_FILE_NAME "how-to-procrastinate" :title "How to procrastinate" :mode nil :granularity element :cached t :parent (headline (:raw-value "Posts" :begin 4041 :end 112268 :pre-blank 0 :contents-begin 4049 :contents-end 112268 :robust-begin 4094 :robust-end 112266 :level 1 :priority nil :tags nil :todo-keyword nil :todo-type nil :post-blank 0 :footnote-section-p nil :archivedp nil :commentedp nil :post-affiliated 4041 :EXPORT_HUGO_SECTION "post" :title ... :mode nil :granularity nil :parent ... :cached t :org-element--cache-sync-key nil)) :org-element--cache-sync-key (3 . 55270))) 0] nil nil 0] :cmpfun org-element--cache-compare))
  org-hugo--get-pre-processed-buffer()
  org-hugo--export-subtree-to-md(nil nil)
  org-hugo-export-wim-to-md(nil nil nil)
  (lambda (a _s v _b) (org-hugo-export-wim-to-md nil a v))(nil nil nil nil)
  org-export-dispatch(nil)
  funcall-interactively(org-export-dispatch nil)
  call-interactively(org-export-dispatch nil nil)
  command-execute(org-export-dispatch)

Please let me know if that was sufficient. If not I can execute your recommendation below.

Here's how you can get more debug info: Set debug-on-quit to t. This will show the backtrace each time you hit C-g. Try to reproduce the hang and while emacs is hung, hit C-g. Please share the backtrace you see next.

kaushalmodi commented 2 years ago

This is for when emacs hangs on save.

This looks related to the doom config and the new Org cache feature in Org. This is unrelated to exports. Can you report this to the Org mailing list?

Here's the paste for that. This is for when emacs hangs on export.

org-hugo--get-pre-processed-buffer()

I am aware of that performance bottleneck in that function. That function is needed if you have Org internal links from one subtree to another. If you don't have links from one post subtree to another, I can come up with a variable that skips this pre-processing.

kaushalmodi commented 2 years ago

If you have examples (sites) you know of who have large number of separate org files, that'd also be helpful.

You might find more than one example on how to do per-file exports on https://ox-hugo.scripter.co/doc/examples/ .

But if you are exporting only one subtree at a time, it should not take minutes to export! I would like to fix this in ox-hugo if possible or at least identify the root cause and have it fixed elsewhere (in Org or emacs). Your help in debugging this will be appreciated.

kishvanchee commented 2 years ago

I am aware of that performance bottleneck in that function. That function is needed if you have Org internal links from one subtree to another. If you don't have links from one post subtree to another, I can come up with a variable that skips this pre-processing.

Unfortunately, I do have internal links from one subtree to another.

But if you are exporting only one subtree at a time, it should not take minutes to export! I would like to fix this in ox-hugo if possible or at least identify the root cause and have it fixed elsewhere (in Org or emacs). Your help in debugging this will be appreciated.

I did this subtree export once again and the CPU usage was so high my system hung. I had to restart now.

I have invited you on gitlab for my website https://gitlab.com/kishvanchee/personal-website/ to your username https://gitlab.com/kaushalmodi . Please feel free to clone and use it to debug ox-hugo.

Please let me know if you require anything from my end to help you debug this.

kishvanchee commented 2 years ago

You might find more than one example on how to do per-file exports on https://ox-hugo.scripter.co/doc/examples/ .

Yes thank you I found it helpful. https://github.com/xianmin/xianmin.org/tree/master/orgpost is one that stands out that does both subtree and per-file exports (per year). If this doesn't get resolved in ox-hugo, most probably I'd have to adopt both approaches at the same time.

This looks related to the doom config and the new Org cache feature in Org. This is unrelated to exports. Can you report this to the Org mailing list?

I feel this is mostly related to only Org and not doom configs. I'll try to report this on the Org mailing list (I'm new to that :sweat_smile: ). Maybe I'll have to downgrade Org for now to 9.4 to see how it goes.

kishvanchee commented 2 years ago

Since my system restarted, I'm not facing the delay/hang during the export of the subtree right now. Maybe I'll be able to give you a better traceback at a later point of time. Here's one traceback I could collect by pressing C-g on time.

Debugger entered--Lisp error: (quit)
  write-region(nil nil "/home/kishore/.cache/org-persist//27/a897ab-5f2b-4..." nil 0)
  org-persist-write((org-element--headline-cache org-element--cache) #<buffer content.org>)
  org-persist-write-all(#<buffer *Ox-hugo Pre-processed content.org *<2><2>>)
  org-persist-write-all-buffer()
  kill-buffer(#<buffer *Ox-hugo Pre-processed content.org *<2><2>>)
  #f(compiled-function () #<bytecode 0x1580fb51cc4d>)()
  org-babel-exp-process-buffer()
  #f(compiled-function (backend &optional subtreep visible-only body-only ext-plist) "Transcode current Org buffer into BACKEND code.\n\nBACKEND is either an export back-end, as returned by, e.g.,\n`org-export-create-backend', or a symbol referring to\na registered back-end.\n\nIf narrowing is active in the current buffer, only transcode its\nnarrowed part.\n\nIf a region is active, transcode that region.\n\nWhen optional argument SUBTREEP is non-nil, transcode the\nsub-tree at point, extracting information from the headline\nproperties first.\n\nWhen optional argument VISIBLE-ONLY is non-nil, don't export\ncontents of hidden elements.\n\nWhen optional argument BODY-ONLY is non-nil, only return body\ncode, without surrounding template.\n\nOptional argument EXT-PLIST, when provided, is a property list\nwith external parameters overriding Org default settings, but\nstill inferior to file-local settings.\n\nReturn code as a string." #<bytecode 0x1580fc4f42d9>)(hugo :subtreep nil nil (:output-file "/home/kishore/kishore_repositories/personal-websit..."))
  apply(#f(compiled-function (backend &optional subtreep visible-only body-only ext-plist) "Transcode current Org buffer into BACKEND code.\n\nBACKEND is either an export back-end, as returned by, e.g.,\n`org-export-create-backend', or a symbol referring to\na registered back-end.\n\nIf narrowing is active in the current buffer, only transcode its\nnarrowed part.\n\nIf a region is active, transcode that region.\n\nWhen optional argument SUBTREEP is non-nil, transcode the\nsub-tree at point, extracting information from the headline\nproperties first.\n\nWhen optional argument VISIBLE-ONLY is non-nil, don't export\ncontents of hidden elements.\n\nWhen optional argument BODY-ONLY is non-nil, only return body\ncode, without surrounding template.\n\nOptional argument EXT-PLIST, when provided, is a property list\nwith external parameters overriding Org default settings, but\nstill inferior to file-local settings.\n\nReturn code as a string." #<bytecode 0x1580fc4f42d9>) (hugo :subtreep nil nil (:output-file "/home/kishore/kishore_repositories/personal-websit...")))
  (let ((old-async-init-file org-export-async-init-file) (org-export-async-init-file (make-temp-file "doom-org-async-export"))) (let ((temp-file org-export-async-init-file) (temp-buffer (generate-new-buffer " *temp file*"))) (unwind-protect (prog1 (save-current-buffer (set-buffer temp-buffer) (prin1 (list (quote progn) (list ... ... ... ... ...) (cons ... ...)) (current-buffer))) (save-current-buffer (set-buffer temp-buffer) (write-region nil nil temp-file nil 0))) (and (buffer-name temp-buffer) (kill-buffer temp-buffer)))) (apply fn args))
  +org--fix-async-export-a(#f(compiled-function (backend &optional subtreep visible-only body-only ext-plist) "Transcode current Org buffer into BACKEND code.\n\nBACKEND is either an export back-end, as returned by, e.g.,\n`org-export-create-backend', or a symbol referring to\na registered back-end.\n\nIf narrowing is active in the current buffer, only transcode its\nnarrowed part.\n\nIf a region is active, transcode that region.\n\nWhen optional argument SUBTREEP is non-nil, transcode the\nsub-tree at point, extracting information from the headline\nproperties first.\n\nWhen optional argument VISIBLE-ONLY is non-nil, don't export\ncontents of hidden elements.\n\nWhen optional argument BODY-ONLY is non-nil, only return body\ncode, without surrounding template.\n\nOptional argument EXT-PLIST, when provided, is a property list\nwith external parameters overriding Org default settings, but\nstill inferior to file-local settings.\n\nReturn code as a string." #<bytecode 0x1580fc4f42d9>) hugo :subtreep nil nil (:output-file "/home/kishore/kishore_repositories/personal-websit..."))
  apply(+org--fix-async-export-a #f(compiled-function (backend &optional subtreep visible-only body-only ext-plist) "Transcode current Org buffer into BACKEND code.\n\nBACKEND is either an export back-end, as returned by, e.g.,\n`org-export-create-backend', or a symbol referring to\na registered back-end.\n\nIf narrowing is active in the current buffer, only transcode its\nnarrowed part.\n\nIf a region is active, transcode that region.\n\nWhen optional argument SUBTREEP is non-nil, transcode the\nsub-tree at point, extracting information from the headline\nproperties first.\n\nWhen optional argument VISIBLE-ONLY is non-nil, don't export\ncontents of hidden elements.\n\nWhen optional argument BODY-ONLY is non-nil, only return body\ncode, without surrounding template.\n\nOptional argument EXT-PLIST, when provided, is a property list\nwith external parameters overriding Org default settings, but\nstill inferior to file-local settings.\n\nReturn code as a string." #<bytecode 0x1580fc4f42d9>) (hugo :subtreep nil nil (:output-file "/home/kishore/kishore_repositories/personal-websit...")))
  org-export-as(hugo :subtreep nil nil (:output-file "/home/kishore/kishore_repositories/personal-websit..."))
  #f(compiled-function (backend file &optional async subtreep visible-only body-only ext-plist post-process) "Call `org-export-as' with output to a specified file.\n\nBACKEND is either an export back-end, as returned by, e.g.,\n`org-export-create-backend', or a symbol referring to\na registered back-end.  FILE is the name of the output file, as\na string.\n\nA non-nil optional argument ASYNC means the process should happen\nasynchronously.  The resulting buffer will then be accessible\nthrough the `org-export-stack' interface.\n\nOptional arguments SUBTREEP, VISIBLE-ONLY, BODY-ONLY and\nEXT-PLIST are similar to those used in `org-export-as', which\nsee.\n\nOptional argument POST-PROCESS is called with FILE as its\nargument and happens asynchronously when ASYNC is non-nil.  It\nhas to return a file name, or nil.  Export back-ends can use this\nto send the output file through additional processing, e.g,\n\n  (defun org-latex-export-to-latex\n    (&optional async subtreep visible-only body-only ext-plist)\n    (interactive)\n    (let ((outfile (org-export-output-file-name \".tex\" subtreep)))\n      (org-export-to-file \\='latex outfile\n        async subtreep visible-only body-only ext-plist\n        #'org-latex-compile)))\n\nWhen expressed as an anonymous function, using `lambda',\nPOST-PROCESS needs to be quoted.\n\nThe function returns either a file name returned by POST-PROCESS,\nor FILE." #<bytecode 0x1580fc60cb09>)(hugo "/home/kishore/kishore_repositories/personal-websit..." nil :subtreep nil)
  apply(#f(compiled-function (backend file &optional async subtreep visible-only body-only ext-plist post-process) "Call `org-export-as' with output to a specified file.\n\nBACKEND is either an export back-end, as returned by, e.g.,\n`org-export-create-backend', or a symbol referring to\na registered back-end.  FILE is the name of the output file, as\na string.\n\nA non-nil optional argument ASYNC means the process should happen\nasynchronously.  The resulting buffer will then be accessible\nthrough the `org-export-stack' interface.\n\nOptional arguments SUBTREEP, VISIBLE-ONLY, BODY-ONLY and\nEXT-PLIST are similar to those used in `org-export-as', which\nsee.\n\nOptional argument POST-PROCESS is called with FILE as its\nargument and happens asynchronously when ASYNC is non-nil.  It\nhas to return a file name, or nil.  Export back-ends can use this\nto send the output file through additional processing, e.g,\n\n  (defun org-latex-export-to-latex\n    (&optional async subtreep visible-only body-only ext-plist)\n    (interactive)\n    (let ((outfile (org-export-output-file-name \".tex\" subtreep)))\n      (org-export-to-file \\='latex outfile\n        async subtreep visible-only body-only ext-plist\n        #'org-latex-compile)))\n\nWhen expressed as an anonymous function, using `lambda',\nPOST-PROCESS needs to be quoted.\n\nThe function returns either a file name returned by POST-PROCESS,\nor FILE." #<bytecode 0x1580fc60cb09>) (hugo "/home/kishore/kishore_repositories/personal-websit..." nil :subtreep nil))
  (let (before-save-hook after-save-hook) (apply fn args))
  +org--dont-trigger-save-hooks-a(#f(compiled-function (backend file &optional async subtreep visible-only body-only ext-plist post-process) "Call `org-export-as' with output to a specified file.\n\nBACKEND is either an export back-end, as returned by, e.g.,\n`org-export-create-backend', or a symbol referring to\na registered back-end.  FILE is the name of the output file, as\na string.\n\nA non-nil optional argument ASYNC means the process should happen\nasynchronously.  The resulting buffer will then be accessible\nthrough the `org-export-stack' interface.\n\nOptional arguments SUBTREEP, VISIBLE-ONLY, BODY-ONLY and\nEXT-PLIST are similar to those used in `org-export-as', which\nsee.\n\nOptional argument POST-PROCESS is called with FILE as its\nargument and happens asynchronously when ASYNC is non-nil.  It\nhas to return a file name, or nil.  Export back-ends can use this\nto send the output file through additional processing, e.g,\n\n  (defun org-latex-export-to-latex\n    (&optional async subtreep visible-only body-only ext-plist)\n    (interactive)\n    (let ((outfile (org-export-output-file-name \".tex\" subtreep)))\n      (org-export-to-file \\='latex outfile\n        async subtreep visible-only body-only ext-plist\n        #'org-latex-compile)))\n\nWhen expressed as an anonymous function, using `lambda',\nPOST-PROCESS needs to be quoted.\n\nThe function returns either a file name returned by POST-PROCESS,\nor FILE." #<bytecode 0x1580fc60cb09>) hugo "/home/kishore/kishore_repositories/personal-websit..." nil :subtreep nil)
  apply(+org--dont-trigger-save-hooks-a #f(compiled-function (backend file &optional async subtreep visible-only body-only ext-plist post-process) "Call `org-export-as' with output to a specified file.\n\nBACKEND is either an export back-end, as returned by, e.g.,\n`org-export-create-backend', or a symbol referring to\na registered back-end.  FILE is the name of the output file, as\na string.\n\nA non-nil optional argument ASYNC means the process should happen\nasynchronously.  The resulting buffer will then be accessible\nthrough the `org-export-stack' interface.\n\nOptional arguments SUBTREEP, VISIBLE-ONLY, BODY-ONLY and\nEXT-PLIST are similar to those used in `org-export-as', which\nsee.\n\nOptional argument POST-PROCESS is called with FILE as its\nargument and happens asynchronously when ASYNC is non-nil.  It\nhas to return a file name, or nil.  Export back-ends can use this\nto send the output file through additional processing, e.g,\n\n  (defun org-latex-export-to-latex\n    (&optional async subtreep visible-only body-only ext-plist)\n    (interactive)\n    (let ((outfile (org-export-output-file-name \".tex\" subtreep)))\n      (org-export-to-file \\='latex outfile\n        async subtreep visible-only body-only ext-plist\n        #'org-latex-compile)))\n\nWhen expressed as an anonymous function, using `lambda',\nPOST-PROCESS needs to be quoted.\n\nThe function returns either a file name returned by POST-PROCESS,\nor FILE." #<bytecode 0x1580fc60cb09>) (hugo "/home/kishore/kishore_repositories/personal-websit..." nil :subtreep nil))
  #f(advice-wrapper :around #f(compiled-function (backend file &optional async subtreep visible-only body-only ext-plist post-process) "Call `org-export-as' with output to a specified file.\n\nBACKEND is either an export back-end, as returned by, e.g.,\n`org-export-create-backend', or a symbol referring to\na registered back-end.  FILE is the name of the output file, as\na string.\n\nA non-nil optional argument ASYNC means the process should happen\nasynchronously.  The resulting buffer will then be accessible\nthrough the `org-export-stack' interface.\n\nOptional arguments SUBTREEP, VISIBLE-ONLY, BODY-ONLY and\nEXT-PLIST are similar to those used in `org-export-as', which\nsee.\n\nOptional argument POST-PROCESS is called with FILE as its\nargument and happens asynchronously when ASYNC is non-nil.  It\nhas to return a file name, or nil.  Export back-ends can use this\nto send the output file through additional processing, e.g,\n\n  (defun org-latex-export-to-latex\n    (&optional async subtreep visible-only body-only ext-plist)\n    (interactive)\n    (let ((outfile (org-export-output-file-name \".tex\" subtreep)))\n      (org-export-to-file \\='latex outfile\n        async subtreep visible-only body-only ext-plist\n        #'org-latex-compile)))\n\nWhen expressed as an anonymous function, using `lambda',\nPOST-PROCESS needs to be quoted.\n\nThe function returns either a file name returned by POST-PROCESS,\nor FILE." #<bytecode 0x1580fc60cb09>) +org--dont-trigger-save-hooks-a)(hugo "/home/kishore/kishore_repositories/personal-websit..." nil :subtreep nil)
  apply(#f(advice-wrapper :around #f(compiled-function (backend file &optional async subtreep visible-only body-only ext-plist post-process) "Call `org-export-as' with output to a specified file.\n\nBACKEND is either an export back-end, as returned by, e.g.,\n`org-export-create-backend', or a symbol referring to\na registered back-end.  FILE is the name of the output file, as\na string.\n\nA non-nil optional argument ASYNC means the process should happen\nasynchronously.  The resulting buffer will then be accessible\nthrough the `org-export-stack' interface.\n\nOptional arguments SUBTREEP, VISIBLE-ONLY, BODY-ONLY and\nEXT-PLIST are similar to those used in `org-export-as', which\nsee.\n\nOptional argument POST-PROCESS is called with FILE as its\nargument and happens asynchronously when ASYNC is non-nil.  It\nhas to return a file name, or nil.  Export back-ends can use this\nto send the output file through additional processing, e.g,\n\n  (defun org-latex-export-to-latex\n    (&optional async subtreep visible-only body-only ext-plist)\n    (interactive)\n    (let ((outfile (org-export-output-file-name \".tex\" subtreep)))\n      (org-export-to-file \\='latex outfile\n        async subtreep visible-only body-only ext-plist\n        #'org-latex-compile)))\n\nWhen expressed as an anonymous function, using `lambda',\nPOST-PROCESS needs to be quoted.\n\nThe function returns either a file name returned by POST-PROCESS,\nor FILE." #<bytecode 0x1580fc60cb09>) +org--dont-trigger-save-hooks-a) (hugo "/home/kishore/kishore_repositories/personal-websit..." nil :subtreep nil))
  (let ((old-async-init-file org-export-async-init-file) (org-export-async-init-file (make-temp-file "doom-org-async-export"))) (let ((temp-file org-export-async-init-file) (temp-buffer (generate-new-buffer " *temp file*"))) (unwind-protect (prog1 (save-current-buffer (set-buffer temp-buffer) (prin1 (list (quote progn) (list ... ... ... ... ...) (cons ... ...)) (current-buffer))) (save-current-buffer (set-buffer temp-buffer) (write-region nil nil temp-file nil 0))) (and (buffer-name temp-buffer) (kill-buffer temp-buffer)))) (apply fn args))
  +org--fix-async-export-a(#f(advice-wrapper :around #f(compiled-function (backend file &optional async subtreep visible-only body-only ext-plist post-process) "Call `org-export-as' with output to a specified file.\n\nBACKEND is either an export back-end, as returned by, e.g.,\n`org-export-create-backend', or a symbol referring to\na registered back-end.  FILE is the name of the output file, as\na string.\n\nA non-nil optional argument ASYNC means the process should happen\nasynchronously.  The resulting buffer will then be accessible\nthrough the `org-export-stack' interface.\n\nOptional arguments SUBTREEP, VISIBLE-ONLY, BODY-ONLY and\nEXT-PLIST are similar to those used in `org-export-as', which\nsee.\n\nOptional argument POST-PROCESS is called with FILE as its\nargument and happens asynchronously when ASYNC is non-nil.  It\nhas to return a file name, or nil.  Export back-ends can use this\nto send the output file through additional processing, e.g,\n\n  (defun org-latex-export-to-latex\n    (&optional async subtreep visible-only body-only ext-plist)\n    (interactive)\n    (let ((outfile (org-export-output-file-name \".tex\" subtreep)))\n      (org-export-to-file \\='latex outfile\n        async subtreep visible-only body-only ext-plist\n        #'org-latex-compile)))\n\nWhen expressed as an anonymous function, using `lambda',\nPOST-PROCESS needs to be quoted.\n\nThe function returns either a file name returned by POST-PROCESS,\nor FILE." #<bytecode 0x1580fc60cb09>) +org--dont-trigger-save-hooks-a) hugo "/home/kishore/kishore_repositories/personal-websit..." nil :subtreep nil)
  apply(+org--fix-async-export-a #f(advice-wrapper :around #f(compiled-function (backend file &optional async subtreep visible-only body-only ext-plist post-process) "Call `org-export-as' with output to a specified file.\n\nBACKEND is either an export back-end, as returned by, e.g.,\n`org-export-create-backend', or a symbol referring to\na registered back-end.  FILE is the name of the output file, as\na string.\n\nA non-nil optional argument ASYNC means the process should happen\nasynchronously.  The resulting buffer will then be accessible\nthrough the `org-export-stack' interface.\n\nOptional arguments SUBTREEP, VISIBLE-ONLY, BODY-ONLY and\nEXT-PLIST are similar to those used in `org-export-as', which\nsee.\n\nOptional argument POST-PROCESS is called with FILE as its\nargument and happens asynchronously when ASYNC is non-nil.  It\nhas to return a file name, or nil.  Export back-ends can use this\nto send the output file through additional processing, e.g,\n\n  (defun org-latex-export-to-latex\n    (&optional async subtreep visible-only body-only ext-plist)\n    (interactive)\n    (let ((outfile (org-export-output-file-name \".tex\" subtreep)))\n      (org-export-to-file \\='latex outfile\n        async subtreep visible-only body-only ext-plist\n        #'org-latex-compile)))\n\nWhen expressed as an anonymous function, using `lambda',\nPOST-PROCESS needs to be quoted.\n\nThe function returns either a file name returned by POST-PROCESS,\nor FILE." #<bytecode 0x1580fc60cb09>) +org--dont-trigger-save-hooks-a) (hugo "/home/kishore/kishore_repositories/personal-websit..." nil :subtreep nil))
  org-export-to-file(hugo "/home/kishore/kishore_repositories/personal-websit..." nil :subtreep nil)
  org-hugo-export-to-md(nil :subtreep nil)
  org-hugo--export-subtree-to-md(nil nil)
  org-hugo-export-wim-to-md(nil nil nil)
  (lambda (a _s v _b) (org-hugo-export-wim-to-md nil a v))(nil nil nil nil)
  org-export-dispatch(nil)
  funcall-interactively(org-export-dispatch nil)
  call-interactively(org-export-dispatch nil nil)
  command-execute(org-export-dispatch)

I hope that is helpful, if not please let me know.

kishvanchee commented 2 years ago

Temporarily I have downgraded Org to 9.4.6 from here - https://git.savannah.gnu.org/cgit/emacs/org-mode.git/tag/?h=release_9.4.6. Hoping this would solve some of the issues :crossed_fingers:

reyman commented 2 years ago

You might find more than one example on how to do per-file exports on https://ox-hugo.scripter.co/doc/examples/ .

Yes thank you I found it helpful. https://github.com/xianmin/xianmin.org/tree/master/orgpost is one that stands out that does both subtree and per-file exports (per year). If this doesn't get resolved in ox-hugo, most probably I'd have to adopt both approaches at the same time.

This looks related to the doom config and the new Org cache feature in Org. This is unrelated to exports. Can you report this to the Org mailing list?

I feel this is mostly related to only Org and not doom configs. I'll try to report this on the Org mailing list (I'm new to that 😅 ). Maybe I'll have to downgrade Org for now to 9.4 to see how it goes.

I'm also present (and probably Nobiot) on emacs org mailling list. If you need some help to debug this problem with :w . Like you probably already see i also open an issue to doom, waiting for a reply : https://github.com/hlissner/doom-emacs/issues/5924

finex commented 2 years ago

Hi, I'm experiencing the same issue with Doom Emacs (28-git from archlinux repo) and latest org 9.5.2. I'm using the auto export on save, but it does freeze even with the manual export. I've to press C-g, otherwise it stops endlessly.

If you can tell me how to downgrade org version at least we can understand if the problem is org 9.5.

Thank you.

kishvanchee commented 2 years ago

@finex If you're using doom-emacs you can pin packages to a specific commit like this in packages.el and do doom sync -u. The following pins it to 9.4.6. You can pin it according to the commit from the releases tags here

(package! org :recipe
  (:host github
   :repo "emacs-straight/org-mode")
  :pin "652430128896e690dc6ef2a83891a1209094b3da")
finex commented 2 years ago

@kishvanchee Hi, I confirm that pinning the version to 9.4.6 resolve the problem. Do you think it's a problem on org-mode and it will be fixed in the future or should a fix be implemented on the ox-hugo project to maintain the compatibility with the latest org-mode version? Thank you very much!

kaushalmodi commented 2 years ago

@finex Given that rolling back the version of Org mode fixes this problem, it's caused due to a change in Org mode. Now, the change could be good and could be pointing to an inefficient piece of code in ox-hugo.

But in order to fix it, we will need to share a reproducible example with Org mode list so that the folks there can take a look at this.

I have seen the same issue, but it's intermittent. So I am unable to come up with a recipe. If you have an Org file that you can share, that always shows this problem, I can share that with the list. Otherwise, I'll keep on looking for a way to reproduce this problem consistently.

kishvanchee commented 2 years ago

@kaushalmodi I'm not sure if this would help, do you think an org file with say 200 entries help you debug? I'm thinking of a recipe like

* Posts
** Post 001
:PROPERTY:
...
:END:
Lorem ipsum ... <paragraphs>
** Post 002
...
** Post 200
...

If yes I can whip up a quick script to produce the file

kaushalmodi commented 2 years ago

Today, I saw Emacs freezing due to the simple Org tagging command (C-c C-q). Tagging was snappy once again once I disabled caching on Org elements:

(setq org-element-use-cache nil)

@kishvanchee @finex If you have time, please update to the latest Org version from its main branch, disable cache and see if that slowness or hanging persists.

finex commented 2 years ago

@kaushalmodi Hi, I've updated to the latest Org version and disabled encryption like said on https://github.com/hlissner/doom-emacs/issues/5924#issuecomment-1003198513: (remove-hook 'before-save-hook 'org-encrypt-entries t) ... and it worked.

Now I will try to follow your suggestion. I will tell you the results later.

Thank you.

finex commented 2 years ago

Hi, I've just updated to the latest doom-emacs version + latest ox-hugo, removed both the temporary fixes (the cache one and the before-safe-hook one) and now both the org export and org tagging are working flawlessly.

I've tried to add the:

(setq org-element-use-cache nil)

and it still works fine.

The org version is still the same but the ox-hugo is the one from yesterday commits.

Let's see what it happens on the next days. If it is an intermittent problem or if it is really fixed with the latest version.

finex commented 2 years ago

Ok, it has freezed again after 30 minutes. Now I will add (setq org-element-use-cache nil). Let's see.

finex commented 2 years ago

Ok, using setq org-element-use-cache nil also fix the problem.

rodelrod commented 2 years ago

I've been having this problem occasionally.

Just had it in a 26-line file. It was freezing saving but also just moving around the buffer. C-g wasn't working most of the time, had to pkill -USR2 emacs.

(setq org-element-use-cache nil) was the only way that allowed me to finally save the file.

kaushalmodi commented 2 years ago

(setq org-element-use-cache nil) was the only way that allowed me to finally save the file.

@rodelrod Please report this on the Org mailing list so that it can be fixed in the devel or main branch of Org mode. The Org element caching is a bleeding edge feature under development.