Make versioned trees

[moteus]

AppVeyor supports build cache so you can build Lua/LuaRocks only once and put it to cache. But to put LuaRocks to cache you need install LuaRocks in separate dirs so each Lua version will have its own luarocks copy and rocks tree. And in install.bat you just need to set PATH/LUA_PATH/LUA_CPATH according to current Lua version.

[ignacio]

Yes. From what I've read in the docs, cached files last up to four hours. One should test if the cached files are there and build them if not.

[moteus]

I just tested it and i think cache is its own for each build matrix entry. So may be it should work as is. I just start new build but it very slow :(

[moteus]

https://github.com/moteus/lua-appveyor-example/tree/94d54759ffb9e0f031282804a447bfabc44e4dbb This code works. (https://ci.appveyor.com/project/moteus/lua-appveyor-example/build/0.0.1.10-test) Riht now I can not make PR. I can do it in few hours

[ignacio]

Great, thank you. Right now, I'm adding LuaJIT to the matrix so I'll have to add your changes by hand.

[moteus]

ok. note this is not last commit but previews one

[ignacio]

@moteus I see you are working on the cache thing and many improvements. Take into account that the scripts have changed heavily. Sorry for that.

[moteus]

Take into account that the scripts have changed heavily.

I saw this. Its too complicated now :) Why is needed? I think its sufficient just

::Install any library/utils

::install if not cached
if not exists SOME_DIR_OR_FILE (_INSTALL_HERE_)
::check installation
if not exists SOME_DIR_OR_FILE (exit /b 1)
::set environment variable
set ...

If you do not like appveyour DownloadFile you can just use wget or curl (add cinst curl to install section).

To extract tar.gz you can use eg 7z x -so lua-%LUA_VER%.tar.gz | 7z x -si -ttar or even curl http://www.lua.org/ftp/lua-%LUA_VER%.tar.gz | 7z x -si -so -tgzip | 7z x -si -ttar You really do not need temp files.

[ignacio]

Well, yes, it became a bit more convoluted now :smile:

The thing is I want to be able to run the script locally instead of pushing and waiting for appveyor to run (and drinking lots of coffee in te meantime). One of the things I needed to change was the way files were downloaded. Flags are way too different so I couldn't get away with set DOWNLOADER=something as I intended, and I didn't want to litter the code with if %APPVEYOR% use one thing else use another

I saw this method of faking functions in cmd but, overall, I'm not entirely happy with it. I think I'll give your suggestion of just using curl a go and see if things get any simpler.

Thanks for the tips about avoiding temp files.

[moteus]

One of the things I needed to change was the way files were downloaded.

Or you can can write basic tools on Lua :smile:

local cURL = require "cURL"
local path = require "path"
local  URL = require "net.url"

local cmd = assert(arg[1])

assert(cmd == "DownloadFile")

local url = assert(arg[2])

local info = assert(URL.parse(url), "Invalid URL: " .. url)

assert(info.path, "Invalid URL: " .. url)

local fileName = path.basename(info.path)

local f = assert(io.open(fileName, "wb"))

cURL.easy{
  url = url,
  writefunction = f
}:perform():close()

f:close()

[ignacio]

hehehe, well I rewrote LuaRocks install script from batch script to Lua because it was becoming a mess. btw, are you trying to sell me your modules? :smile:

[moteus]

No. I think that on windows there no common way on how to build/install/check libs/tools. Also there many developers who do not use Windows. And if there will be simple bat file based on which easy make appveyor config they may be make it and because appveyor provide artifacts there will be Windows binaris. Next step is upload binary rocks to luarocks server. Now problem is that there no way upload binary rock for several Lua version. (e.g. make rock with external deps)

[ignacio]

Sorry, that was a lame attempt at a joke.

You're right. I can see what is your vision. Push a tag on your git repo, run all tests, produce binary rocks and upload them to the main repo. Nice.

[ignacio]

@moteus what do you think of this last commit? You can see the build run here:

https://ci.appveyor.com/project/ignacio/lua-appveyor-example/build/0.0.1.64-test

[moteus]

I think that appveyor.yml contain too many env variables. I have no idea why some one may want change LuaRocks install dir. Why we define url to Lua in appveyor.yml but url for LuaJIT in install.bat? Why we define LR_EXTERNAL in appveyor.yml if we really can not change it? Why some one may want change LuaRocks install dir? I think all this variables should be private in install.bat and may be overwrited by appveyor.yml if it needs. Also may be LUA_VER should has short version and install script just select which one use. Also may be instead of provide %LUA% env variable install script just rename/copy luajit.exe to lua.exe?

environment:
  LUAROCKS_VER: 2.2.1
  matrix:
  - LUA_VER: 5.1
  - LUA_VER: 5.2
  - LUA_VER: 5.3
  - LJ_VER: 2.0
  - LJ_VER: 2.1

platform:
  - x86
  - x64

[ignacio]

I agree with your first points. Providing defaults for the urls, LR_EXTERNAL, LuaRocks path and so on, allowing them to be overriden in appveyor.yml.

"c:\external" is hardcoded in LuaRocks, so yes, for the time being it makes no sense to be defined in the yml file. Will remove it.

Also may be LUA_VER should has short version and install script just select which one use.

I'd like to keep LUA_VER as is, so you can test 5.1.5, 5.1.6, work versions, release candidates, etc, but I could infer the short version:

set LUA_VER=5.2.4
set LUA_SHORTV=%LUA_VER:~0,3%
:: prints 5.2
echo %LUA_SHORTV%

Also may be instead of provide %LUA% env variable install script just rename/copy luajit.exe to lua.exe?

Renaming luajit to lua seems like a hack to me. Doesn't busted try to autodetect the runtime trying to invoke it? Anyway, I can infer that if LUA_VER is defined, use Lua. If LJ_VER is defined, use LuaJIT.

[ignacio]

I did the aforementioned changes in 4114c4f395a956cfc7bb4a4025c35cba73c5aaf1 If the matrix builds ok I'll commit that.