Provide Teal types and change file extensions from .lua to .tl

[Calandiel]

Related to #139 #177

As the LSP got stricter typing support over time, current SotE doesn't fully take advantage of the possible notations, making it de facto dynamically typed for some parts of the codebase.

That's obviously less than ideal. We have an open issue meant to tackle it but we've also been discussing using Teal instead.

Teal is a typed dialect of Lua, similar in spirit to Typescript. It's advantage is that porting Lua code to it most of the time requires only adding type annotations - in our case, rewriting them from Lua's LSP annotation to Teal. I think this would be about the same amount of work as ironing out all current LSP warnings (including the ones that'd occur if we made type annotations more strict).

Opening this issue to keep track of the task and get further feedback on it.

cc @squealingpiggies @ineveraskedforthis @Calandiel @dsecrieru

[dsecrieru]

I noticed this trend lately, using dynamic typed programming languages and then going through a lot of effort to gain the safety of strongly typed languages. Why not use a strongly typed language from the start (especially for new projects)? Is it because said dynamic languages have some features that are wanted but not without compromising on type safety? Just trying to understand.

[Calandiel]

I noticed this trend lately, using dynamic typed programming languages and then going through a lot of effort to gain the safety of strongly typed languages. Why not use a strongly typed language from the start (especially for new projects)? Is it because said dynamic languages have some features that are wanted but not without compromising on type safety? Just trying to understand.

Because sote was meant to be made in Rust after we decided to drop Unity but Demian thought hed come back and he ready didn't want to learn a new language. As such I was asked to rewrite it again. I chose Lua due to its simplicity - I wanted to rewrite important parts of the software asap to maximize the odds of him returning to the project. I still have a repo (private) with "sote in rust"

[Calandiel]

Lua also gets you much easier modding, hot-loading, and control through the love2d framework without having to do low level work yourself but thats a cherry on top so to speak

[Calandiel]

If you guys think using rust codebase is more desirable, I can GPL it but I'd need help getting it up to the same amount of features as current sote in lua - I already have other projects outside work that take up my time 🙁

[squealingpiggies]

On the top of teal, I glossed over the documentation and it should be easy to piecemeal lua files into tl files as long as it works with love2d. If sticking with lua, I do think its worth considering converting over to teal for new code and converting only when touched. It definitely would solve the annotation issue and while being just about equal work.

If rust is to be considered, I would have to say it depends on how developed the gui is. As much as rust is my favorite language, it does not have good guis. Right now, the lua codebase has a decent set of tools and widgets that just need a little generalizing and to use their space as an anchor to scale within, and putting together pieces will become the trivial task of sectioning out subspaces within rectangles.

[Calandiel]

The UI library sote uses, Milky, was written by me when lua didnt have many good ui libraries available. I have reimplemented it in Rust for sote.

That being said, wasnt egui for Rust decent?

Anyway, I can give you guys access to the repo if youd like to take a look at what was there before I was tasked to port it out of Rust.

Main issues is that wed need to port all of netr's work, which was quite substantial.

Imho, using teal is simpler. It really is a "typescript of lua" . Only problem with it is that map modes just can't be both tile based and quick to update (rust and old sote used aggressive multithreading to make it palatably performant)

[dsecrieru]

Rust is interesting but I'd vote for easier transition for teal.

Just to throw one more wrench into the mix, did anyone look at jai? Should be the programming language for games.

[Calandiel]

I'm familiar with it but that'd be writing the game from scratch. I'd only consider Rust because I've ported a large chunk of it a year and a half ago before settling down on Lua.

In either case, yeah, seems like Teal is what we'll end up with

[dsecrieru]

Def not suggesting yet another re-write. Was just curious if anyone looked at jai.

Another observation would be that in g-ral, there shouldn't be a hard limit on which language to use. A game is usually a big system, so using multiple languages is not uncommon. Just use the right tool for the job.

[Calandiel]

Indeed! That's also why using the existing C++ as the first step is a p quick way to get world gen going :wink:

But yeah, I have looked at jai. I've been following it for a very long time, ever since Jonathan Blow announced it on stream. I concur with a lot of his ideas on how a "language for games" ought to work

[squealingpiggies]

I managed to implement tl loading on this branch (drag-n-drop a file and add two lines of code). I had to change the tl.lua file to assign its loader into the third slot otherwise it would fail when loading options.lua in main. While this is a quick and definitely improper fix, if the plan is to convert the entire codebase into tl, this change can be reverted in the future. I also created tl version of string.lua and table.lua and replaced all instances in the code with the new tl versions and the game still runs fine but loading is significantly slower.

[Calandiel]

Teal ought to preload everything ahead of time, otherwise require adds a compilation cost.

Presumably during asset loading or within main. Something along the lines of a large list of require calls to force lua to cache the packages before trying to use them. Other than that, it ought to be 0 or near 0 cost.

[Calandiel]

Oh, I noticed you use records for modules in teal. I believe that's not necessary?

This is a valid module in Teal:

local addsub = {}

function addsub.add(a: number, b: number): number
   return a + b
end

function addsub.sub(a: number, b: number): number
   return a - b
end

return addsub

As per: https://github.com/teal-language/tl/blob/master/docs/tutorial.md

[squealingpiggies]

I was struggling with the language server getting stuck in loops without the record when I was first converting it but that doesn't seem to be an issue anymore as I was able to strip the records and get the game to run. I also noticed I forgot select_one and tribe gen stopped (but loading worked saves worked and ran fine). Even adding it, the tribe gen still failed because of the data structure was hard to infer so I change it (and the associated call) to just be a table of strings mapped to a weight value so it was easier to access in the tl function using generics.