Pandoc-Citeproc Cannot decode byte '\x9f'

[charlieharrysmith]

Hi, I hope someone might be able to help me.

I installed Scrivomatic this morning and so far I've had a consistent issue getting past the creation of .md files during the compile sequence – it appears pandoc thinks I'm using the wrong encoding somewhere. This is the despite the fact I've confirmed (to the best of my ability) that my .bib file is using UTF-8.

Googling seems to imply this is a common issue with pandoc conversions, but I can't figure out how any of the solutions apply to the Scrivomatic implementation of pandoc.

Is this something that's come up before as an issue? And is there a fix? Many thanks in advance.

Here's the debug log:

=== ------------------------------------------------------ === === Scrivomatic V1.0.29 Report @ 2020-07-06 12:48:46 +0100 === === ------------------------------------------------------ === Working directory: /Users/charliesmith/Desktop/CHS Draft Transfer.md Initiating with Ruby 2.6.3 ===------ Input Options: ------===

<struct Scrivomatic::OPT input="CHS\ Draft\ Transfer.md", output=nil, to=nil, yaml=nil, command="pandocomatic", envpath="/Users/charliesmith/bin:/Library/TeX/texbin:/usr/local/bin", build=false, cleanup=false, verbose=true, dry_run=false, open_log=true, data_dir=nil>

===------ Final ENV PATH: ------=== /Users/charliesmith/bin:/Library/TeX/texbin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin:/Applications/Scrivener.app/Contents/Resources/MultiMarkdown/bin ===------ TOOL PATHS: ------=== ---pandoc: /usr/local/bin/pandoc | V: 2.9.2.1 ---pandocomatic: /usr/local/bin/pandocomatic | V: 0.2.7.1 ---ruby: /usr/bin/ruby | V: ruby 2.6.3p62 (2019-04-16 revision 67580) [universal.x86_64-darwin19] ---rbenv versions: ---gem: /usr/bin/gem ---python: /usr/bin/python ---xelatex: /Library/TeX/texbin/xelatex

… running pandocomatic, please wait … ===------ COMMAND OUTPUT: ------=== :: Running: /usr/local/bin/pandocomatic --debug CHS\ Draft\ Transfer.md ::: (4) + converting /Users/charliesmith/Desktop/CHS Draft Transfer.md/CHS Draft Transfer.md 3 times: ::: (3) - convert CHS Draft Transfer.md -> CHS Draft Transfer.docx ::: pandoc --from=markdown \ ::: --to=docx \ ::: --reference-doc=/Users/charliesmith/.local/share/pandoc/templates/custom.docx \ ::: --standalone \ ::: --filter=/usr/local/bin/pandoc-citeproc \ ::: --bibliography=/Users/charliesmith/.local/share/pandoc/Zotero\ Bib\ Library.bib \ ::: --csl=/Users/charliesmith/.local/share/pandoc/csl/apa.csl \ ::: --output=/Users/charliesmith/Desktop/CHS\ Draft\ Transfer.md/CHS\ Draft\ Transfer.docx ::: pandoc-citeproc: Cannot decode byte '\x9f': Data.Text.Internal.Encoding.decodeUtf8: Invalid UTF-8 stream ::: Error running filter /usr/local/bin/pandoc-citeproc: ::: Filter returned error status 1 ::: Error running pandoc => error while running: ::: ::: pandoc --from=markdown \ ::: --to=docx \ ::: --reference-doc=/Users/charliesmith/.local/share/pandoc/templates/custom.docx \ ::: --standalone \ ::: --filter=/usr/local/bin/pandoc-citeproc \ ::: --bibliography=/Users/charliesmith/.local/share/pandoc/Zotero\ Bib\ Library.bib \ ::: --csl=/Users/charliesmith/.local/share/pandoc/csl/apa.csl \ ::: --output=/Users/charliesmith/Desktop/CHS\ Draft\ Transfer.md/CHS\ Draft\ Transfer.docx ::: ::: Pandoc responded with: ::: ::: pandoc-citeproc: Cannot decode byte '\x9f': Data.Text.Internal.Encoding.decodeUtf8: Invalid UTF-8 stream ::: Error running filter /usr/local/bin/pandoc-citeproc: ::: Filter returned error status 1 ::: :: exit status: pid 79231 exit 243

!!!---scrivomatic::runCommand() RETURN non-zero value: /usr/local/bin/pandocomatic --debug CHS\ Draft\ Transfer.md !!!

[charlieharrysmith]

Re-exporting my .bib file from Zotero in the BibLaTeX format solved this issue for me.