Building lua-tiktoken on Windows

[pidgeon777]

This script may be useful, and also implementable on CopilotChat.nvim:

https://github.com/yetone/avante.nvim/commit/83d067695e1f473141279b63bee44f3c919ddcad

[gptlang]

I will add Windows as a target to the release task in https://github.com/gptlang/lua-tiktoken

[gptlang]

We should steal their whole makefile: https://github.com/yetone/avante.nvim/blob/main/Makefile

[gptlang]

@deathbeam did you know we can do

build = "make",

in Lazy.nvimto build stuff?

[gptlang]

https://github.com/gptlang/lua-tiktoken/releases (got windows down)

[gptlang]

I got it working with make https://github.com/CopilotC-Nvim/CopilotChat.nvim/commit/83bce52db3526fcb222921feccd790c3055d51c8

Not quite sure what the equivalent is in Windows

[pidgeon777]

This file was used by avante.nvim plugin to build the lua-tiktoken on Windows:

Build-LuaTiktoken.ps1:

param (
    [string]$Version = "luajit"
)

$BuildDir = "build"
$BuildFromSource = $true

function Build-FromSource($feature) {
    if (-not (Test-Path $BuildDir)) {
        New-Item -ItemType Directory -Path $BuildDir | Out-Null
    }

    $tempDir = Join-Path $BuildDir "lua-tiktoken-temp"
    git clone --branch v0.2.2 --depth 1 https://github.com/gptlang/lua-tiktoken.git $tempDir

    Push-Location $tempDir
    cargo build --features=$feature
    Pop-Location

    $targetFile = "tiktoken_core.dll"
    Copy-Item (Join-Path $tempDir "target\debug\tiktoken_core.dll") (Join-Path $BuildDir $targetFile)

    Remove-Item -Recurse -Force $tempDir
}

function Main {
    Write-Host "Building for $Version..."
    Build-FromSource $Version
}

# Run the main function
Main

[pidgeon777]

I tested the build on Windows 11 with this plugin config:

...
build = vim.fn.has("win32") == 1 and "powershell -ExecutionPolicy Bypass -File Build-LuaTiktoken.ps1" or "make"
...

and it worked.