Added pandoc-zotxt.lua

[odkr]

pandoc-zotxt.lua looks up sources of citations in Zotero and adds them either to a document's references metadata field or to a bibliography file.

[odkr]

The tests require Zotero and zotxt. This is why they fail.

[jgm]

Is there a way to work around this? E.g. a fake zotxt that returns canned data? We can't check this in without passing tests.

[odkr]

Hmm, not sure. I'll have a look at it. I'm not quite sure where I'd get an HTTP server from. And it seems a bit absurd to add a C module/Python script.

[jgm]

I don't want to make this too painful. Why don't you just set it up so that the test doesn't run by default, so it won't register as a failure in our CI, and it can still be run locally?

Odin Kroeger notifications@github.com writes:

Hmm, not sure. I can't find an HTTP server in pure Lua. But maybe I can scrape something together from http://lua-users.org/lists/lua-l/2002-04/msg00180.html and bnet. I'll have a look at it.

-- You are receiving this because you commented. Reply to this email directly or view it on GitHub: https://github.com/pandoc/lua-filters/pull/58#issuecomment-491029718

[odkr]

I wasn’t aware that with Python spinning up an HTTP server is as simple as saying python3 -m http.server <portnumber>. So I thought making the necessary changes wouldn't take much of an effort. That turned out to be wrong, of course. And the tests still didn't work, no idea why. So, yes, I disabled most of them now.

[jgm]

It looks like you accidentally checked in a large number of files in lib/luarocks/rocks-5.3?

[odkr]

Oh, I wasn’t aware that LuaUnit ships with that much documentation. I removed those files and others that were accidents.

[jgm]

Nothing that can be installed using luarocks should be included in the repository. You can just have your test script install these, right?

[odkr]

Sure. Did I overlook some instructions to that effect? At any rate, I removed LuaUnit and the luarocks metadata for lunajson.

[odkr]

[ -e "share/lua/5.3/luaunit.lua" ] || \ luarocks install --tree . luaunit /bin/sh: 2: luarocks: not found

So, I can't just have my test script install LuaUnit. I'll revert and re-commit, but not today.

[tarleb]

LuaUnit is a nice library, which we probably should be using in more places. Maybe we can add it in a central location like .tools?

[jgm]

Instead, let's just have our travis script install luarocks, so it will be available to everyone.

[odkr]

So, that didn’t work:

[ -e "share/lua/5.3/luaunit.lua" ] || \
        luarocks install --tree=. luaunit
Installing http://luarocks.org/repositories/rocks/luaunit-3.3-1.src.rock...
Using http://luarocks.org/repositories/rocks/luaunit-3.3-1.src.rock... switching to 'build' mode
Archive:  /tmp/luarocks_luarocks-rock-luaunit-3.3-1-5920/luaunit-3.3-1.src.rock
  inflating: luaunit-3.3-1.rockspec  
  inflating: rock-luaunit-3.3.zip    
  […]
Updating manifest for /home/travis/build/pandoc/lua-filters/pandoc-zotxt.lua/./lib/luarocks/rocks
No existing manifest. Attempting to rebuild...
luaunit 3.3-1 is now built and installed in /home/travis/build/pandoc/lua-filters/pandoc-zotxt.lua/./ (license: BSD)
mkdir -p "test/tmp"
rm -f "test/tmp"/*
pandoc --lua-filter "test/unit/test.lua" -o /dev/null \
        -M query-base-url=http://localhost:23120 \
        -M tests=test_core /dev/null || \
        { EX=$?; sh test/scripts/httpdctl stop || :; exit "$EX"; }
Error running filter test/unit/test.lua:
[string "test/unit/test.lua"]:115: module 'luaunit' not found:
    no file 'luaunit.lua' in pandoc's datadir
    no field package.preload['luaunit']
    no file '/usr/local/share/lua/5.3/luaunit.lua'
    no file '/usr/local/share/lua/5.3/luaunit/init.lua'
    no file '/usr/local/lib/lua/5.3/luaunit.lua'
    no file '/usr/local/lib/lua/5.3/luaunit/init.lua'
    no file './luaunit.lua'
    no file './luaunit/init.lua'
    no file 'test/unit/../../share/lua/5.3/luaunit.lua'
    no file '/usr/local/lib/lua/5.3/luaunit.so'
    no file '/usr/local/lib/lua/5.3/loadall.so'
    no file './luaunit.so'
stack traceback:
    [C]: in function 'require'
    [string "test/unit/test.lua"]:115: in main chunk
make[1]: *** [test_core] Error 83
FAIL pandoc-zotxt.lua

The makefile works fine on my system though:

$ luarocks remove --tree=. luaunit
Checking stability of dependencies in the absence of
luaunit 3.3-1...

Removing luaunit 3.3-1...
Removal successful.
$ make test
[ -e "share/lua/5.3/luaunit.lua" ] || \
        luarocks install --tree=. luaunit
Installing https://luarocks.org/luaunit-3.3-1.src.rock

luaunit 3.3-1 is now installed in /Users/odin/repos/lua-filters/pandoc-zotxt.lua/. (license: BSD)

mkdir -p "test/tmp"
rm -f "test/tmp"/*
pandoc --lua-filter "test/unit/test.lua" -o /dev/null \
        -M query-base-url=http://localhost:23120 \
        -M tests=test_core /dev/null || \
        { EX=$?; sh test/scripts/httpdctl stop || :; exit "$EX"; }
........
Ran 8 tests in 0.004 seconds, 8 successes, 0 failures
OK
pandoc --lua-filter "test/unit/get_input_directory-pwd.lua" </dev/null

for TEST in `find "test/unit/warn" -type f -name '*.lua' -exec basename \{\} \; | sed 's/.lua$//' | sort`; do \
        pandoc --lua-filter "test/unit/warn/$TEST.lua" -o /dev/null \
            /dev/null 2>"test/tmp/$TEST.out"; \
        cmp "test/norms/warn/$TEST.out" "test/tmp/$TEST.out"; \
    done

[jgm]

Presumably we need to set the LUA_PATH so it finds these things?

[bpj]

But then you run the risk that other libraries installed elsewhere on the user's system are not found properly, unless you can just add (prepend) stuff to the variable in an OS neutral way. I vaguely remember you can, having fought with that variable once.

Den ons 15 maj 2019 00:15John MacFarlane notifications@github.com skrev:

Presumably we need to set the LUA_PATH so it finds these things?

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/pandoc/lua-filters/pull/58?email_source=notifications&email_token=AAI3OU3QJMOMLCYV5UINGY3PVM2XBA5CNFSM4HKWYBOKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODVM6IBY#issuecomment-492430343, or mute the thread https://github.com/notifications/unsubscribe-auth/AAI3OUYUFAVLSKICENDCAT3PVM2XBANCNFSM4HKWYBOA .

[odkr]

I don’t think LUA_PATH is the problem. luarocks install --tree=. luaunit should install LuaUnit relative to the current working directory, that is, in ./share/lua/5.3/luaunit.lua. Well, at least that's what it does on my system. And the test suite adds that directory to package.path:

--- The directory of the script.
local SCRIPT_DIR = split_path(PANDOC_SCRIPT_FILE)

--- The directory of the test suite.
local TEST_DIR = concat({SCRIPT_DIR, '..'}, PATH_SEP)

-- [...]

--- The repository directory.
local REPO_DIR = concat({TEST_DIR, '..'}, PATH_SEP)

package.path = package.path .. ';' ..
    concat({REPO_DIR, 'share', 'lua', '5.3', '?.lua'}, PATH_SEP)

Maybe something's wrong with this code then, I just don’t see what it could be. It does look for luaunit.lua at test/unit/../../share/lua/5.3/luaunit.lua, after all. It appears that luarocks installs LuaUnit elsewhere. But where?

[tarleb]

The filter is available at https://github.com/odkr/pandoc-zotxt.lua, which is much better than including it here. Thus closing.