search

iandol / scrivomatic

A writing workflow using Scrivener's style system + Pandoc for output…
https://iandol.github.io/scrivomatic/
GNU General Public License v3.0
296 stars 31 forks source link

error trying to compile simple workflow: `require': cannot load such file -- paru/filter (LoadError) #37

Closed Sigfried closed 4 years ago

Sigfried commented 4 years ago

I'm trying to follow the instructions in https://raw.githubusercontent.com/iandol/scrivomatic/master/Workflow.scriv.zip (downloaded from https://github.com/iandol/scrivomatic#tldr-simple-summary) and am having a problem that seems similar to #13 except I just installed all the software and everything should be up to date. Here's my scrivomatic log:

=== ------------------------------------------------------ ===
=== Scrivomatic V1.0.29 Report @ 2020-08-17 06:42:42 -0400 ===
=== ------------------------------------------------------ ===
 Working directory: /Volumes/workspace/sigfried/workflow-testing/workflow-mmd
 Initiating with Ruby 2.6.3
===------ Input Options: ------===
#<struct Scrivomatic::OPT input="Workflow.md", output=nil, to=nil, yaml="pandocomatic.yaml", command="pandocomatic", envpath="/Users/sigfried/bin:/Users/sigfried/.rbenv/shims:/usr/local/opt/ruby/bin:/usr/local/lib/ruby/gems/2.7.0/bin:/Library/TeX/texbin:/usr/local/bin:/Users/sigfried/.local/bin", build=false, cleanup=false, verbose=true, dry_run=false, open_log=true, data_dir=nil>
===------ Final ENV PATH: ------===
/Users/sigfried/bin:/Users/sigfried/.rbenv/shims:/usr/local/opt/ruby/bin:/usr/local/lib/ruby/gems/2.7.0/bin:/Library/TeX/texbin:/usr/local/bin:/Users/sigfried/.local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin:/Applications/Scrivener.app/Contents/Resources/MultiMarkdown/bin
===------ TOOL PATHS: ------===
---pandoc: /usr/local/bin/pandoc | V: 2.9.2.1
---pandocomatic: /usr/local/bin/pandocomatic | V: 0.2.7.2
---ruby: /Users/sigfried/.rbenv/shims/ruby | V: ruby 2.7.1p83 (2020-03-31 revision a0c7c23c9c) [x86_64-darwin19]
---rbenv versions:
---gem: /Users/sigfried/.rbenv/shims/gem
---python: /usr/local/bin/python
---xelatex: /Library/TeX/texbin/xelatex
---latexmk: /Library/TeX/texbin/latexmk

 … running pandocomatic, please wait …
===------ COMMAND OUTPUT: ------===
:: Running: /usr/local/bin/pandocomatic -c pandocomatic.yaml --debug Workflow.md 
::: The pandoc option 'base_header_level' (with value '1') is not recognized by paru. This option is skipped.
::: (5) + converting /Volumes/workspace/sigfried/workflow-testing/workflow-mmd/Workflow.md 4 times:
::: (4)   - convert Workflow.md -> Workflow.tex
::: pandoc  --from=markdown \
:::     --to=latex \
:::     --template=custom.latex \
:::     --toc \
:::     --toc-depth=4 \
:::     --filter=assimilateMetadata.rb \
:::     --standalone \
:::     --bibliography=workflow.bib \
:::     --csl=apa-workflow.csl
::: /usr/local/Cellar/ruby/2.7.1_2/lib/ruby/2.7.0/rubygems/core_ext/kernel_require.rb:92:in `require': cannot load such file -- paru/filter (LoadError)
:::     from /usr/local/Cellar/ruby/2.7.1_2/lib/ruby/2.7.0/rubygems/core_ext/kernel_require.rb:92:in `require'
:::     from ./assimilateMetadata.rb:22:in `<main>'
::: Error running filter assimilateMetadata.rb:
::: Filter returned error status 1
::: Error running pandoc => error while running:
::: 
::: pandoc  --from=markdown \
:::     --to=latex \
:::     --template=custom.latex \
:::     --toc \
:::     --toc-depth=4 \
:::     --filter=assimilateMetadata.rb \
:::     --standalone \
:::     --bibliography=workflow.bib \
:::     --csl=apa-workflow.csl
::: 
::: Pandoc responded with:
::: 
::: /usr/local/Cellar/ruby/2.7.1_2/lib/ruby/2.7.0/rubygems/core_ext/kernel_require.rb:92:in `require': cannot load such file -- paru/filter (LoadError)
:::     from /usr/local/Cellar/ruby/2.7.1_2/lib/ruby/2.7.0/rubygems/core_ext/kernel_require.rb:92:in `require'
:::     from ./assimilateMetadata.rb:22:in `<main>'
::: Error running filter assimilateMetadata.rb:
::: Filter returned error status 1
::: 
:: exit status: pid 67553 exit 243

!!!---scrivomatic::runCommand() RETURN non-zero value: /usr/local/bin/pandocomatic -c pandocomatic.yaml --debug Workflow.md !!!

Any ideas what I'm doing wrong? Thanks!

Sigfried commented 4 years ago

I tried it again, following the instructions more precisely. Same problem. Then I tried to run the command that was failing from the command line, and got the following error. I'm including it in case it's at all informative.

➜ pandoc        --from=markdown --to=latex --template=custom.latex --toc --toc-depth=4 --filter=assimilateMetadata.rb --standalone --bibliography=workflow.bib --csl=apa-workflow.csl workflow.md

Traceback (most recent call last):
  14: from ./assimilateMetadata.rb:74:in `<main>'
  13: from /Library/Ruby/Gems/2.6.0/gems/paru-0.4.1.2/lib/paru/filter.rb:251:in `run'
  12: from /Library/Ruby/Gems/2.6.0/gems/paru-0.4.1.2/lib/paru/filter.rb:277:in `filter'
  11: from /Library/Ruby/Gems/2.6.0/gems/paru-0.4.1.2/lib/paru/filter.rb:336:in `read_document'
  10: from /Library/Ruby/Gems/2.6.0/gems/paru-0.4.1.2/lib/paru/filter/document.rb:85:in `from_JSON'
  9: from /Library/Ruby/Gems/2.6.0/gems/paru-0.4.1.2/lib/paru/filter/document.rb:85:in `new'
  8: from /Library/Ruby/Gems/2.6.0/gems/paru-0.4.1.2/lib/paru/filter/document.rb:118:in `initialize'
  7: from /Library/Ruby/Gems/2.6.0/gems/paru-0.4.1.2/lib/paru/filter/node.rb:117:in `initialize'
  6: from /Library/Ruby/Gems/2.6.0/gems/paru-0.4.1.2/lib/paru/filter/node.rb:117:in `each'
  5: from /Library/Ruby/Gems/2.6.0/gems/paru-0.4.1.2/lib/paru/filter/node.rb:119:in `block in initialize'
  4: from /Library/Ruby/Gems/2.6.0/gems/paru-0.4.1.2/lib/paru/filter/node.rb:119:in `new'
  3: from /Library/Ruby/Gems/2.6.0/gems/paru-0.4.1.2/lib/paru/filter/table.rb:56:in `initialize'
  2: from /Library/Ruby/Gems/2.6.0/gems/paru-0.4.1.2/lib/paru/filter/table.rb:56:in `new'
  1: from /Library/Ruby/Gems/2.6.0/gems/paru-0.4.1.2/lib/paru/filter/attr.rb:48:in `initialize'
/Library/Ruby/Gems/2.6.0/gems/paru-0.4.1.2/lib/paru/filter/attr.rb:48:in `[]': odd number of arguments for Hash (ArgumentError)
Error running filter assimilateMetadata.rb:
Filter returned error status 1
iandol commented 4 years ago

@Sigfried -- Hi sorry you seem to have run into problems. I've just tested using the Workflow.scriv sample project and I can't reproduce your error.

Looking at your log, it seems your ruby and gems appear inconsistent, compare to my log:

===------ TOOL PATHS: ------===
---pandoc: /usr/local/bin/pandoc | V: 2.10.1
---pandocomatic: /Users/ian/.rbenv/shims/pandocomatic | V: 0.2.7.2
---ruby: /Users/ian/.rbenv/shims/ruby | V: ruby 2.7.1p83 (2020-03-31 revision a0c7c23c9c) [x86_64-darwin19]
---rbenv versions:
  system
* 2.7.1 (set by /Users/ian/.rbenv/version)
---rbenv: /usr/local/bin/rbenv
---gem: /Users/ian/.rbenv/shims/gem

Note my pandocmatic AND ruby are both managed by rbenv. Your pandocomatic is installed as a system gem, but your ruby is an rbenv one. You can try to ensure rbenv to use its Ruby: rbenv global 2.7.1; rbenv shell 2.7.1 then run rbenv versions and you should see something like this:

➜ rbenv versions
  system
* 2.7.1 (set by RBENV_VERSION environment variable)

Then install pandocmatic again into your rbenv ruby (which gem tells you which gem you're using and gem list shows you what gems are installed); then try again.

I assume if you remove the filter option everything elseworks? The filter assimilateMetadata.rb is only useful if you have academic author/affiliation info you want to convert to "cool" formatted output, for a simple compile you can remove it from your pandocomatic.yaml configuration and try again.

I know this can all seem quite confusing, as there are several moving parts at play: Scrivener + Pandoc + Pandocomatic — each has a knowledge requirement and at first everything looks opaque. But each tool is understandable, and once you get a cognitive map of how each interacts, it should become easier. Note you can completely remove pandocomatic and Scrivener + Pandoc is still a great combination, it just requires more manual command-line editing. I like pandocomatic as it lets me set up a saved configuration for a pandoc compile, one that I can control from a Scrivener frontmatter document so no need to edit commands and so forth...

Sigfried commented 4 years ago

Thanks so much for figuring this out. It turned out I didn't even have rbenv installed --- well, obviously it was installed somewhere, but not on my path. I installed it, installed 2.7.1, and... I haven't tested it all, because as I mentioned on the forum, it's sounding like Scrivener isn't going to work for what I want. It may be that what I want is impossible -- that's usually the way it goes -- but I'm still working on other options for now.

Thanks!

iandol commented 4 years ago

@Sigfried — actually most of the details about pandoc & pandocomatic equally apply to any tool, not just Scrivener; the post-processing workflow works just as well if I write in Visual Studio Code or other writing IDE.. Pandoc remains the best convertor, and pandocomatic the most flexible way to manage lots of Pandoc settings, whatever generates the text...

zavidos commented 4 years ago

@iandol Hi Iandol, I used scrivener+pandoc+pandocomatic+scrivomatic for two years with lot of satisfaction. Now I change my mac and wanted to reinstall everything fresh but unfortunately my setup broke up and I think the problem is the same reported in this thread. I have lot of errors with filters I used for author and affiliation, now I tried with the simple workflow attached in your guide (with only paper-with-ref-docx as I suspect a problem in latex installation due to the fact that tlmgr is not in $PATH on Catalina when basictex is installed). The result is the following:

=== ------------------------------------------------------ ===
=== Scrivomatic V1.0.29 Report @ 2020-09-01 12:00:49 +0200 ===
=== ------------------------------------------------------ ===
 Working directory: /Users/zavidos/Desktop/workflow-mmd
 Initiating with Ruby 2.6.3
===------ Input Options: ------===
#<struct Scrivomatic::OPT input="Workflow.md", output=nil, to=nil, yaml="pandocomatic.yaml", command="pandocomatic", envpath="/Users/zavidos/bin:/usr/local/opt/ruby/bin:/usr/local/lib/ruby/gems/2.7.0/bin:/Library/TeX/texbin:/usr/local/bin", build=false, cleanup=false, verbose=true, dry_run=false, open_log=true, data_dir=nil>
===------ Final ENV PATH: ------===
/Users/zavidos/bin:/usr/local/opt/ruby/bin:/usr/local/lib/ruby/gems/2.7.0/bin:/Library/TeX/texbin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin:/Applications/Scrivener.app/Contents/Resources/MultiMarkdown/bin
===------ TOOL PATHS: ------===
---pandoc: /usr/local/bin/pandoc | V: 2.10.1
---pandocomatic: /usr/local/bin/pandocomatic | V: 0.2.7.2
---ruby: /usr/local/opt/ruby/bin/ruby | V: ruby 2.7.1p83 (2020-03-31 revision a0c7c23c9c) [x86_64-darwin19]
---rbenv versions:
* system (set by /Users/zavidos/.rbenv/version)
---rbenv: /usr/local/bin/rbenv
---gem: /usr/local/opt/ruby/bin/gem
---python: /usr/bin/python
---xelatex: /Library/TeX/texbin/xelatex
---latexmk: /Library/TeX/texbin/latexmk

 … running pandocomatic, please wait …
===------ COMMAND OUTPUT: ------===
:: Running: /usr/local/bin/pandocomatic -c pandocomatic.yaml --debug Workflow.md 
::: pandoc  --from=markdown \
:::     --to=docx \
:::     --reference-doc=custom.docx \
:::     --filter=simplifyMetadata.rb \
:::     --filter=prependAll.rb \
:::     --standalone \
:::     --bibliography=workflow.bib \
:::     --csl=apa-workflow.csl \
:::     --output=/Users/zavidos/Desktop/workflow-mmd/Workflow.docx
::: /usr/local/Cellar/ruby/2.7.1_2/lib/ruby/2.7.0/rubygems/core_ext/kernel_require.rb:92:in `require': cannot load such file -- paru/filter (LoadError)
:::     from /usr/local/Cellar/ruby/2.7.1_2/lib/ruby/2.7.0/rubygems/core_ext/kernel_require.rb:92:in `require'
:::     from ./simplifyMetadata.rb:10:in `<main>'
::: Error running filter simplifyMetadata.rb:
::: Filter returned error status 1
::: Error running pandoc => error while running:
::: 
::: pandoc  --from=markdown \
:::     --to=docx \
:::     --reference-doc=custom.docx \
:::     --filter=simplifyMetadata.rb \
:::     --filter=prependAll.rb \
:::     --standalone \
:::     --bibliography=workflow.bib \
:::     --csl=apa-workflow.csl \
:::     --output=/Users/zavidos/Desktop/workflow-mmd/Workflow.docx
::: 
::: Pandoc responded with:
::: 
::: /usr/local/Cellar/ruby/2.7.1_2/lib/ruby/2.7.0/rubygems/core_ext/kernel_require.rb:92:in `require': cannot load such file -- paru/filter (LoadError)
:::     from /usr/local/Cellar/ruby/2.7.1_2/lib/ruby/2.7.0/rubygems/core_ext/kernel_require.rb:92:in `require'
:::     from ./simplifyMetadata.rb:10:in `<main>'
::: Error running filter simplifyMetadata.rb:
::: Filter returned error status 1
::: 
:: exit status: pid 25698 exit 243

!!!---scrivomatic::runCommand() RETURN non-zero value: /usr/local/bin/pandocomatic -c pandocomatic.yaml --debug Workflow.md !!!

It seems pandocomatic, ruby and gem are in /usr/local instead of my home directory as in your logs but I installed everything with homebrew as per guide. Rbenv version, gem list and brew list are the following:

Thank you for any assistance

% rbenv version
system (set by /Users/zavidos/.rbenv/version)
% gem list

*** LOCAL GEMS ***

bigdecimal (default: 1.4.1)
bundler (default: 1.17.2)
CFPropertyList (2.3.6)
cmath (default: 1.0.0)
csv (default: 3.0.9)
date (default: 2.0.0)
dbm (default: 1.0.0)
did_you_mean (1.3.0)
e2mmap (default: 0.1.0)
etc (default: 1.0.1)
fcntl (default: 1.0.0)
fiddle (default: 1.0.0)
fileutils (default: 1.1.0)
forwardable (default: 1.2.0)
io-console (default: 0.4.7)
ipaddr (default: 1.2.2)
irb (default: 1.0.0)
json (default: 2.1.0)
libxml-ruby (3.1.0)
logger (default: 1.3.0)
matrix (default: 0.1.0)
mini_portile2 (2.4.0)
minitest (5.11.3)
mutex_m (default: 0.1.0)
net-telnet (0.2.0)
nokogiri (1.10.1)
openssl (default: 2.1.2)
optimist (3.0.1)
ostruct (default: 0.1.0)
pandocomatic (0.2.7.2)
paru (0.4.1.2)
power_assert (1.1.3)
prime (default: 0.1.0)
psych (default: 3.1.0)
rake (12.3.2)
rdoc (default: 6.1.0)
rexml (default: 3.1.9)
rss (default: 0.2.7)
scanf (default: 1.0.0)
sdbm (default: 1.0.0)
shell (default: 0.7)
sqlite3 (1.3.13)
stringio (default: 0.0.2)
strscan (default: 1.0.0)
sync (default: 0.5.0)
test-unit (3.2.9)
thwait (default: 0.1.0)
tracer (default: 0.1.0)
webrick (default: 1.4.2)
xmlrpc (0.3.0)
zlib (default: 1.0.0)
% brew list
autoconf    openssl@1.1 pandoc-crossref rbenv       ruby-build
gptfdisk    pandoc      pkg-config  readline    smartmontools
libyaml     pandoc-citeproc popt        ruby
% which ruby    
/usr/bin/ruby
% ruby --version
ruby 2.6.3p62 (2019-04-16 revision 67580) [universal.x86_64-darwin19]
iandol commented 4 years ago

@zavidos — my preference is to use rbenv to install ruby, whereas although you have installed rbenv, you still use brew to install ruby (i.e. rbenv is supposed to manage ruby but it cannot as it didn't install anything, homebrew did). So you can try to uninstall homebrew ruby, then use rbenv itself to install a ruby and switch to that, then install pandocomatic and try again.

https://github.com/iandol/scrivomatic/blob/master/Installing-Ruby.md#using-rbenv

Nevertheless, I am surprised given your log it is failing as at least all your paths are consistently pointing to the homebrew-ruby unlike the previous log where everything was mixed up. A simpler solution if you are on Catalina is to uninstall the homebrew-ruby AND rbenv, and just use the system Ruby (remembering to install pandocomatic there):

https://github.com/iandol/scrivomatic/blob/master/Installing-Ruby.md#users-using-macos-catalina

Good luck and let me know how it goes!

zavidos commented 4 years ago

@iandol two words: "it works"

This is the log:

=== ------------------------------------------------------ ===
=== Scrivomatic V1.0.29 Report @ 2020-09-03 08:18:14 +0200 ===
=== ------------------------------------------------------ ===
 Working directory: /Users/zavidos/Desktop/workflow-mmd
 Initiating with Ruby 2.6.3
===------ Input Options: ------===
#<struct Scrivomatic::OPT input="workflow.md", output=nil, to=nil, yaml="pandocomatic.yaml", command="pandocomatic", envpath="/Users/zavidos/bin:/Users/zavidos/.rbenv/shims:/usr/local/lib/ruby/gems/2.7.0/bin:/Library/TeX/texbin:/usr/local/bin", build=false, cleanup=false, verbose=true, dry_run=false, open_log=true, data_dir=nil>
===------ Final ENV PATH: ------===
/Users/zavidos/bin:/Users/zavidos/.rbenv/shims:/usr/local/lib/ruby/gems/2.7.0/bin:/Library/TeX/texbin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin:/Applications/Scrivener.app/Contents/Resources/MultiMarkdown/bin
===------ TOOL PATHS: ------===
---pandoc: /usr/local/bin/pandoc | V: 2.10.1
---pandocomatic: /Users/zavidos/.rbenv/shims/pandocomatic | V: 0.2.7.2
---ruby: /Users/zavidos/.rbenv/shims/ruby | V: ruby 2.7.1p83 (2020-03-31 revision a0c7c23c9c) [x86_64-darwin19]
---rbenv versions:
  system
* 2.7.1 (set by /Users/zavidos/.rbenv/version)
---rbenv: /usr/local/bin/rbenv
---gem: /Users/zavidos/.rbenv/shims/gem
---python: /usr/bin/python
---xelatex: /Library/TeX/texbin/xelatex
---latexmk: /Library/TeX/texbin/latexmk

 … running pandocomatic, please wait …
===------ COMMAND OUTPUT: ------===
:: Running: /Users/zavidos/.rbenv/shims/pandocomatic -c pandocomatic.yaml --debug workflow.md 
::: pandoc  --from=markdown \
:::     --to=docx \
:::     --reference-doc=custom.docx \
:::     --filter=simplifyMetadata.rb \
:::     --filter=prependAll.rb \
:::     --standalone \
:::     --bibliography=workflow.bib \
:::     --csl=apa-workflow.csl \
:::     --output=/Users/zavidos/Desktop/workflow-mmd/workflow.docx
::: Pandocomatic needed 1.0 seconds to convert '/Users/zavidos/Desktop/workflow-mmd/workflow.md' to 'workflow.docx'.
:: exit status: pid 69011 exit 0

I removed ruby from homebrew and pandocomatic (with sudo as catalina require sudo gem..). I installed rbenv through homebrew and ruby 2.7.1 through rbenv. But then if I try to install pandocomatic it uses system's ruby instead of rbenv one. So I had to set rbenv global 2.7.1 and also add rbenv dir to $PATH.

Maybe you want to add these two point to your beautiful scrivomatic guide:

Thanks a lot for all your work

iandol commented 4 years ago

@zavidos -- great! I did reword the Ruby install instructions to make clear that a user should choose either option but not more than one. Catalina does have Ruby V2.6.3 which should work without needing rbenv etc. for now, but this will be removed once macOS 11 is released later this year AFAIK, which is why my preference is still to use rbenv.

Regarding texbin, I do search for it and add it with scrivomatic: https://github.com/iandol/scrivomatic/blob/master/scrivomatic#L152 — the user path is not used by Scrivener so I have to recreate it (i.e.LaTeX compiles should work from Scrivener even if it isn't in path). But yes, as you still need to manage the packages in your terminal you need to access either tlmgr, or install the GUI manager: brew cask install tex-live-utility. Just FYI, brew also has the minimal LaTeX setup: brew cask install basictex.