Scripting Layer

[pennomi]

For Flare-next, I'd like extremely fine-grained control over all aspects of the game. To do this, I suggest adding a full scripting layer to the game.

Obviously this is a really big deal and will require thorough updates to pretty much all the data-driven classes.

Ideas for why this might be awesome to do:

• Alternate AI implementations could be added as data files (no engine modification necessary)
• Quests could hold more than boolean data
• Powers could have much more complex effects
• Quests and NPC dialog could be easily branchable.

Of course there's a lot of design decisions to do here. Whatever we choose, it probably means adding another external dependency. I personally really love Python, so I'd recommend using that as an embedded scripting language, but I can see sense if anyone would disagree with that. Other good options seem to be chaiscript and lua.

[dorkster]

I had done some experimenting with Lua with Flare in the past (I deleted the branch since it was so out of date). Lua was pretty easy to integrate and performance was good, so I'd personally choose it as the language for such a feature.

[pennomi]

Would it be possible to internalize Lua? I wonder if we can avoid the extra dependency that way.

[dorkster]

@pennomi I doubt it. If it is possible, I also doubt that it would be worth the effort. Since we'd be controlling all of our data with Lua, I'd see it as a worthwhile dependency.

[clintbellanger]

I'll agree that Lua as a new dependency is well worth it, if we pull the game-specific code out into scripts. That is well beyond my expertise/experience though, I tend to stick to simple games.

[ghost]

Hello, I've been experimenting with Lua scripting for a while and I thought I'd share my progress. I managed to expose most of the functionality currently provided by Event and EventManager to Lua scripts.

To define events for a map, you need to add "eventfile" attribute to the map's header:

[header]
eventfile=scripts/some_script.lua

Events can then be defined by calling "event" function in the specified file. For example, to create a chest that remembers whether it has already been opened you would define the following event:

event {
    location = {x=20, y=30, w=1, h=1},
    hotspot = {x=20, y=30, w=1, h=1},
    tooltip = "Treasure Chest",

    on_mapload = function (ev)
        if check_status("chest_discovered") then
            mapmod("object", ev.location.x, ev.location.y, 50)
            return false -- remove event
        end
        return true -- keep event
    end,

    on_trigger = function (ev)
        msg("Treasure found")
        loot("currency", ev.location.x+1, ev.location.y, 100, 1000, 2000)
        mapmod("object", ev.location.x, ev.location.y, 50)
        set_status("chest_discovered")
    end
}

In general an event definition looks like this (of course you don't have to specify all the functions):

event {
    location = {x=10, y=20, w=1, h=1},
    hotspot = {x=10, y=20, w=1, h=1},
    tooltip = "Some event",
    cooldown = 60, -- specified in frames

    is_active = function (ev)
        -- return true if the event is active and false otherwise
        -- similar to what EventManager.isActive does
    end,

    is_reachable = function (ev)
        -- return true if the event is reachable and false otherwise
        -- similar to what Event.reachable_from is used for
    end,

    on_trigger = function (ev)
        -- called repeatedly while the player is inside "location"
    end,

    on_enter = function (ev)
        -- called once when the player enters "location"
    end,

    on_leave = function (ev)
        -- called once when the player leaves "location"
    end,

    on_mapload = function (ev)
        -- called when the map loads
    end,

    on_mapexit = function (ev)
        -- called when the player exits the map
    end,

    on_mapclear = function (ev)
        -- called when the map is cleared
    end
}

If on_trigger, on_enter, on_leave, on_mapload, on_mapexit or on_mapclear returns false (or nothing) then the event is deleted after being triggered. If it returns true then the event is kept and it can be triggered again. One of the neat things about these functions is that they receive event as their argument. This allows you to attach arbitrary data to an event and access/modify it in callbacks:

event {
    location = {x=10, y=20, w=1, h=1},
    cooldown = 60,
    my_variable = 1,

    on_trigger = function (ev)
        if ev.my_variable <= 3 then
            msg(string.format("my_variable is %d", ev.my_variable))
            ev.my_variable = ev.my_variable + 1
            return true
        end
        return false
    end
}

There are several issues with current implementation however:

1. Events must be defined separately from the map. This means you can't use Tiled to define events and specify their location.
2. Only one power can be activated per event (AFAIK this limitation is also present in current event system).
3. No localization support (xgettext has support for Lua, so I don't think implementing this will be difficult).

If you'd like to play with it, you can get the source here: https://github.com/mazayus/flare-engine-next/tree/scripting I also ported several maps (ancient_temple, frontier_outpost, frontier_plains, mineshaft_longsword and warp_zone). They are available here: https://github.com/mazayus/flare-game/tree/scripting

[pennomi]

Wow, very impressive. My thoughts:

1. Events must be defined separately from the map. This means you can't use Tiled to define events and specify their location. Could we potentially use tiled to create named regions that we could then bind the event to? That would be the best of both worlds IMO.
2. I've never used lua (though it looks easy). Is there an easy way to "subclass" (or something similar to that) for reducing code duplication? ie. Can all "treasure chest" events share the majority of their code?
3. I love the arbitrary data thing. That's just plain awesome.

[dorkster]

@mazayus Great work so far. I haven't looked at the code closely yet, but would something like this be possible instead of having a single script referenced in the map header?:

[event]
location=x,y,w,h
data=items/loot_tables/level_1-5.txt,true
script=scripts/chest.lua

The variables from location and data would be passed to the script before it was run. The data here isn't really important, it's more the idea that we could pass some information to a script instead of having everything in the script itself. This would let us keep event location (and possibly some relevant data) in Tiled. Also, this could allow for more reusable scripts, as @pennomi mentioned in his second point.

[ghost]

1) Yeah, I though about something similar: we could specify event ID when defining an event

event {
    id = "some_event",
    on_trigger = ...
}

and then reference that ID in the map file

[event]
location=10,20,1,1
hotspot=location
tooltip=Some event
id=some_event

2) "event" is just a function called with one argument - a table. So you can define a helper function for creating "treasure chests", for example:

function make_treasure_chest(pos_x, pos_y, status, min_gold, max_gold)
    event {
        location = {x=pos_x, y=pos_y, w=1, h=1},
        hotspot = {x=pos_x, y=pos_y, w=1, h=1},
        tooltip = "Treasure Chest",

        on_mapload = function (ev)
            if check_status(status) then
                mapmod("object", ev.location.x, ev.location.y, 50)
                return false -- remove event
            end
            return true -- keep event
        end,

        on_trigger = function (ev)
            msg("Treasure found")
            loot("currency", ev.location.x+1, ev.location.y, 100, min_gold, max_gold)
            mapmod("object", ev.location.x, ev.location.y, 50)
            set_status(status)
        end
    }
end

[ghost]

@dorkster I think this might work. It's similar to what I described in the first part of my previous comment. If any attribute specified in "[event]" section can be attached to an event then all callback functions (on_trigger, etc.) will be able to access it through its 'ev' parameter. I'll try to implement this tomorrow.

[pennomi]

@mazayus I think what you have suggested would be fine. I also think I like the programmatic control over the creation of the event as well. So for instance, I could take make_treasure_chest like you mentioned and randomly place several on the map if I wanted. This would be nearly impossible using Tiled. However, I could also use Tiled to visually place them on the map. Seems like a good amount of control to me. Of course, I may not have wrapped my head around the whole thing enough to come up with the "perfect" solution.

[ghost]

I pushed an update to my scripting branch. It is now possible to define events and attach arbitrary data to them directly in map files, e.g.

[event]
location=20,30,1,1
script=scripts/some_event.lua
my_variable=41
the_answer=forty-two

Event defined in some_event.lua has access to attributes defined earlier, e.g.

event {
    on_trigger = function (ev)
        local num = tonumber(ev.my_variable) + 1
        msg(string.format("%d is %s", num, ev.the_answer))
    end
}

Note that my_variable and the_answer are passed to the script as strings, so if you wish to treat them differently you have to cast them explicitly. In general only script, location, cooldown, hotspot and tooltip have special meaning, all other attributes are passed directly as strings: script is not passed at to the script at all, location and hotspot are converted to a table {x=integer, y=integer, w=integer, h=integer}, cooldown is converted to an integer, and tooltip is passed directly as a string.

It is still possible to specify eventfile attribute in the header, although I think it's not needed anymore - it should be equivalent to

[event]
script=scripts/events.lua

where events.lua defines several events.

I also updated ancient_temple to demonstrate this new way of defining events. Here's an example of defining and using a "treasure chest" event that we discussed yesterday: https://github.com/mazayus/flare-game/blob/scripting/mods/alpha_demo/scripts/ancient_temple/chest.lua https://github.com/mazayus/flare-game/blob/scripting/mods/alpha_demo/maps/ancient_temple.txt#L507 https://github.com/mazayus/flare-game/blob/scripting/mods/alpha_demo/maps/ancient_temple.txt#L516

[dorkster]

@mazayus That's even better than what I envisioned. Impressive.

On Tue, Aug 12, 2014 at 11:40 AM, Milan notifications@github.com wrote:

I pushed an update to my scripting branch. It is now possible to define events and attach arbitrary data to them directly in map files, e.g.

[event] location=20,30,1,1 script=scripts/some_event.lua my_variable=41 the_answer=forty-two

Event defined in some_event.lua has access to attributes defined earlier, e.g.

event { on_trigger = function (ev) local num = tonumber(ev.my_variable) + 1 msg(string.format("%d is %s", num, ev.the_answer)) end}

Note that my_variable and the_answer are passed to the script as strings, so if you wish to treat them differently you have to cast them explicitly. In general only script, location, cooldown, hotspot and tooltip have special meaning, all other attributes are passed directly as strings: script is not passed at to the script at all, location and hotspot are converted to a table {x=integer, y=integer, w=integer, h=integer}, cooldown is converted to an integer, and tooltip is passed directly as a string.

It is still possible to specify eventfile attribute in the header, although I think it's not needed anymore - it should be equivalent to

[event] script=scripts/events.lua

where events.lua defines several events.

I also updated ancient_temple to demonstrate this new way of defining events. Here's an example of defining and using a "treasure chest" event that we discussed yesterday:

https://github.com/mazayus/flare-game/blob/scripting/mods/alpha_demo/scripts/ancient_temple/chest.lua

https://github.com/mazayus/flare-game/blob/scripting/mods/alpha_demo/maps/ancient_temple.txt#L507

https://github.com/mazayus/flare-game/blob/scripting/mods/alpha_demo/maps/ancient_temple.txt#L516

— Reply to this email directly or view it on GitHub https://github.com/dorkster/flare-engine-next/issues/13#issuecomment-51932278 .

[pennomi]

I took a look at the code and it looks quite nice. This seems like an obvious win to me.

@mazayus Is there more on the event system that needs done? And do you have a tentative roadmap for binding other things with Lua? I for one think that having the enemy AI implementations written in Lua would be very useful.

[ghost]

I haven't tested the code thoroughly so there are probably some bugs lurking. The documentation is also rather poor. So that's what I'll be focusing on next - testing and documenting.

Also, as you can see from the code, I created a separate class LuaEventManager instead of modifying existing EventManager because EventManager is currently used for NPC dialog events (starting/completing a quest, giving quest rewards, etc.). So I guess I'll try to implement NPC scripting next.

Other than that I haven't planned much, so if you'd like to see scripted enemy behavior I'll give it a shot.

[clintbellanger]

This is really cool to see. I had trouble visualizing how a scripting language might work, and seeing the hooks to engine calls is eye opening.

[ghost]

Hello. I've been thinking about how scripted dialogs might work, and I've come up with two high-level ideas so far.

The first one is kind of 'code-driven' approach, where dialog text and logic are mixed together. A dialog definition might look something like this:

dialog {
    topic = "Money",

    is_active = function ()
        -- return true if this dialog should be offered
    end,

    on_activate = function ()
        he("Hi!")
        he("Need some money?")

        -- ask the player to make a choice
        local response = choose("Sure", "No way", "Uhmm...")
        if response == 1 then
            reward_currency(100)
            he("Here you are")
            he("Have a nice day")
            i("Thanks")
        elseif response == 2 then
            he("Your loss")
            i("Yeah, whatever")
        else
            he("Uhmm?")
            i("Uhmm...")
        end
    end
}

The second approach is more of a data-driven one. You still use Lua to define both dialog text and logic, but they are better separated from each other. Using this approach you define dialog nodes and transitions between them. A dialog definition might look something like this:

dialog { 
    id = 1,
    responses = {2, 3}, -- ids of the response dialogs
    text = "Hello. How are you?",

    is_active = function ()
        -- return true if this dialog should be offered
        return true
    end,

    on_enter = function ()
        -- executed before entering this dialog node
    end,

    on_leave = function ()
        -- executed after leaving this dialog node
    end
}

dialog {
    id = 2,
    text = "Shiny",
    is_active = function ()
        -- offer this dialog only if the player has more than 1000 gold
        return check_currency(1000)
    end
}

dialog {
    id = 3,
    text = "Mind your own business"
}

The first approach might be easier for simple dialogs, but it will probably lead to spaghetti code for dialogs with complex tree/graph structures. The second approach, on the other hand, is arguably harder to read and write, but it might be possible to write a node-based visualization/editing tool similar to ChatMapper (of course nothing that complex).

Any ideas on what will suit FLARE better?

[pennomi]

I've been thinking on this for a while now. I suggest the first option (not using dialog nodes) is better. The reason is because it doesn't force the programmer to write dialog in a specific way. Since Lua is a full programming language, it would be possible to have the dialog follow a node-based approach just by coding it to work that way. (In fact, we could provide pure-Lua helper functions to do that if it became a common task.) But once again, I don't think there's any reason to mandate the node paradigm; if nodes aren't needed for your game, you can just write it the simpler way.

[ghost]

Just a quick update. I've been working on the solution suggested by @pennomi, but I've ran into a problem that would be difficult (if at all possible) to solve directly. Specifically, at each step of the dialogue either "continue dialog" or "close dialog" button needs to be shown. The "close" button should be shown only when there are no more dialog messages to display. The problem is that a dialog is represented using Lua function (coroutine actually, but the modder doesn't need to know this - all he works with is a function). So figuring out when to show the "close" button entails figuring out whether there's still code to be executed in the function. For example:

dialog {
    on_continue = function (dlg)
        reward_xp(10)           -- 1
        i("hello")              -- 2
        reward_currency(10)     -- 3
        he("hello")             -- 4
    end
}

When on_continue is called it executes line 1, then it executes line 2 and pauses the coroutine. The only way to figure out whether the coroutine will end after the call to i returns is to actually try and resume the coroutine. But this will lead to line 3 being executed, and so on. There are a couple of workarounds: 1) Add a flag to i, he and she that indicates whether to show continue or close button, e.g.

    on_continue = function (dlg)
        i("hi")
        he("hello")
        i("bye", true)
    end

2) Add a field to dlg that indicates what button to show, e.g.

    on_continue = function (dlg)
        i("hi")
        he("hello")
        dlg.show_button = "close"
        i("bye")
    end

3) Always show the "continue" button and never show the "close" button. 4) Fiddle with Lua bytecode and try to figure out whether current instruction is the last instruction in the function. From what I researched, Lua doesn't give you access to the instruction pointer so this might be fruitless.

I don't like the first two options, because it's easy to forget to set the flag or change the field. The third option feels like a regression to me. The fourth one will probably be too complex and hacky.

[pennomi]

@mazayus Great work with the coroutines! I was interested in seeing how you planned on solving that issue. My personal opinion would be to do the first option (though the second is more explicit).

Other potential options would be to load everything in an array that's accessible by the engine. Think something like this: (And I don't do Lua, so I apologize in advance if this has improper syntax)

dialog {
    actions = [
        reward_xp(10),           -- 1
        i("hello"),              -- 2
        reward_currency(10),     -- 3
        he("hello"),             -- 4
    ]
    on_continue = function (dlg)
        -- display next message here
        show(actions[dlg.current_index])
        -- If it's the last one, call 
        dlg.show_button = "close"
    end
}

One advantage of this is that it actually would make node-based editing easier (but still not required). The on_continue could have a default implementation so it's not repeated over and over, but it could still be overridden if you needed to do something fancy.

[ghost]

@pennomi But how will this approach handle control structures like if and while? For example, what would the actions array look like in this case?

on_continue = function (dlg)
    if check_currency(100) then
        he("hi")
    else
        he("go away")
end

Regarding node-based editing, the implementation of the dialog is actually quite a bit more low-level than what I described. For example, dialog expects that every time on_continue function is called it returns the next message in the dialog. But I provided a helper function simpledialog that wraps on_continue in a coroutine and allows using it in the way I described earlier. So all the calls to dialog should have been calls to simpledialog instead. In a similar way a helper function can be provided for node-based dialogs.

[pennomi]

@mazayus Yes, I agree that it would make those cases much harder to code for.

With that in mind, I believe that having an optional "end" boolean arg would be the right way to go. Can Lua do keyword/named args? So the call would look something like this:

he("See you later.", end=true)

Ideally having a dialog piece with the end argument would also stop the rest of the coroutine from running.

on_continue = function (dlg)
    if !check_currency(100) then
        he("Get out of my face!", end=true)  -- this ends the conversation and stops the coroutine
    i("Good morning.")
    he("Top 'o the morning to ya.")
end

[ghost]

@pennomi Unfortunately Lua doesn't have keyword arguments. There is a special syntax for calling a function with one argument however, so the following two statements are equivalent:

myfunc({"abc", size = 3})
myfunc{"abc", size = 3}

[pennomi]

Well either way, as long as it's an optional argument I really don't care. Would that solution work from a technical standpoint?

[ghost]

@pennomi Yes, it's just a matter of yielding one more value from the coroutine. I'll implement this tomorrow.

[dorkster]

I closed the issues in the upstream flare repo pertaining to this. For future reference they were:

• https://github.com/flareteam/flare-engine/issues/1228
• https://github.com/flareteam/flare-engine/issues/1426

[AustinHaugerud]

Apologies, I forgot this issue was already open. Much of the work done by ghost seems good, is there any reason we couldn't simply keep expanding on that?

[igorko]

His user (mazayus) was removed and repo too. Ghost is just a github note that there is no such user anymore. No idea how we can recover his code or contact with him.

[AustinHaugerud]

Has anyone here ever worked with Papyrus before? It's the scripting language employed by Skyrim's engine. Personally I think a lot of the design decisions in that were great.

In Lua we could set up scripts to have quite clearly defined tasks. You create scripts with a certain kind of extension in mind and then do what you want from there. You could also access quests much like objects and scripts could use properties making changing them possible without even opening a text editor.

Obviously not every feature in Papyrus would be good for Flare, and there are some major differences like Papyrus being OOP, but I think we could get some good ideas from it.