Add `JArray.valueType` to the Lua API

[RealAntithesis]

I would like to test various expressions and print out the results to debug my lua scripts - but print() doesn't appear to print to the Skyrim console nor the JC log. Is this output going somewhere else? Assert() does work, but it prints a multi-line error message and stops execution.

[ryobg]

From my quick view I did not see some interference there, so I think, print works fine in a sense it uses the current process standard output. For example, if you run JC through native console application you will see the output. No clue, what Skyrim does with that though?

You may try to either return strings from your expression or look at the io Lua lib. Haven't tried it, but probably you can create a log file and write there - use it like print log. There are some tutorials for it. Maybe look here: https://www.tutorialspoint.com/lua/lua_file_io.htm

[RealAntithesis]

Yep, I'll try that and see how it goes.

[RealAntithesis]

Ok, that works now - thanks. Another question, still debugging: is there a way to determine the JC data type from within a lua script (similar to JArray.valueType)? I have an array that may contain JC objects or forms, but lua generates an error if I try to access a non-existent record - which the objects have (they are JMaps) but the forms obviously don't. Thanks.

[ryobg]

Hmm... can't verify it now. Have you tried to use JArray.valueType(obj, index) again? It should have the same parameters, usage and return type as from within the Papyrus.

[RealAntithesis]

Oh right, I was somehow of the impression that this wasn't available because it wasn't declared in the jc.lua file (some jc functions are there, but not others. JArray.valuetype was excluded). I'll try it when I get home.

[RealAntithesis]

The JArray.valuetype declaration was commented out in jc.lua (in the JArray function table). I had to uncomment this and create a new function JArray.valuetype to call from my init.lua script. It took some time (I used the other declared functions as a guide to the syntax and expected parameters), but it seems to work. I'll need to do more testing.

I'm not sure why these functions were excluded from jc.lua in the first place. Perhaps it was just that the default init.lua didn't need them?

While it does seem to work, I'm hesitant to modify the jc.lua file since this may break (either jc or AH Hotkeys or both) in future jcontainers versions and there may be some other reasons why these functions are commented out.

[ryobg]

Let me see your changes. I will evaluate and we will see.

[RealAntithesis]

See below. I've only added valueType for my immediate needs.

local JArrayNativeFuncs = retrieveNativeFunctions('JArray',
    {
      object = {'handle'},
      objectWithSize = {'handle', 'uint32_t'},
      --getInt = {'int32_t', 'handle, index, int32_t'},
      --getFlt = {'float', 'handle, index, float'},
      --setInt = {'void', 'handle, index, int32_t'},
      --setFlt = {'void', 'handle, index, float'},
      valueType = {'int32_t', 'handle, index'}, -- UNCOMMENTED
    }
  )
-- ADDED
  function JArray.valueType(optr, idx)
    return JArrayNativeFuncs.valueType(jc_context, optr.___id, convertIndex(idx))
  end

[SilverIce]

@RealAntithesis you can use Lua type function on the value, retrieved from the JC - https://www.lua.org/pil/2.html The functions like JArray.valueType aren't there since Lua value types != JC value types

[ryobg]

I saw that there is some conversion back and forth between JC & Lua, but haven't noticed that valueType will return something different? Kinda strange.

[RealAntithesis]

The lua type() function was returning the same thing for both forms and objects: cdata. Both arrayItem = JArray.__index(collection, i); type(arrayItem); and type(collection[i]) returns cdata whether the array item is a form or an object. However, JArray.valueType(collection,i) returns '4' for forms and '5' for objects, which is what the papyrus function returns to distinguish between the different JC types, and which (I believe) I need to determine whether I can test for records within the jmap or jarray without generating a lua error if it happens to be a form. I was assuming before that jmaps and arrays were being stored as lua tables, which would have distinguished them from forms, but apparently not (lua just sees them as cdata).

[RealAntithesis]

For reference, this is my custom find function. The purpose is to search for and return the index of the item in the jarray 'collection' that matches the expression in 'predicate' (this was derived from the pre-existing jc.find function in the default init.lua file). I also added 'startIndex' to begin the search at a particular index and, more recently, a JC value type parameter which focuses the search for particular item types (so it doesn't try to search for collection[i].someSubRecord if the item isn't a JC object, which will generate an error if it's a form). LogMessage() is my debugging print function that writes out to a file (to the Skyrim data folder it looks like if a path isn't specified).

function AH_Hotkeys.FindIndex(collection, predicate, startIndex, JCValueTypeFilter)
    AH_Hotkeys.LogMessage("AH_Hotkeys.FindIndex(): start. startIndex = "..startIndex.."; Type Filter = "..JCValueTypeFilter)
    local foundValueType
    local arrayItem
    if startIndex == nil then startIndex = 0 end
    -- AH_Hotkeys.LogMessage("Test = ")
    for i = startIndex + 1, #collection do
        if JCValueTypeFilter ~= nil then
            -- AH_Hotkeys.LogMessage("Getting JCType...")
            arrayItem = JArray.__index(collection, i)
            foundValueType = JArray.valueType(collection,i)
            AH_Hotkeys.LogMessage("AH_Hotkeys.FindIndex(): foundValueType = " .. foundValueType .. "; JCValueTypeFilter = " .. JCValueTypeFilter.."; lua type(collection[i]) = "..type(collection[i]).."; lua type(arrayItem i) = "..type(arrayItem))
            if foundValueType == JCValueTypeFilter then
                if predicate(collection[i]) then return i - 1 end
            end
        else
            if predicate(collection[i]) then return i - 1 end
        end
    end
    return -1
end

[ryobg]

@RealAntithesis so I assume all is ok on your side now? When I looked for valueType I ignored that it was commented out, will add it for the next release. Your FindIndex function is kinda common algorithm. Would be nice I get some feedback from JC users so we may enhance the default library with frequently used functions...

[RealAntithesis]

Thanks for that. Yep, it seems to be working now with the valueType function added to jc.lua.

Re: other common functions, I'm going to have a go at a sorting function that can sort an array of jmap objects in either ascending or descending order on the basis of a particular subrecord in the jmaps. I'm hoping the standard lua table sorting function can assist with this rather than doing it all from scratch.

[SilverIce]

Would be better to add a new function instead of adding the same to JArray, JMap and etc. I can make a PR.

local function valueType(luaVar)
  local tp = type(luaVar)

  if tp == 'number' then
    return 'number'
  elseif tp == 'string' then
    return 'string'
  --elseif tp == 'boolean' then
    --return 'boolean'
  elseif ffi.istype(CForm, luaVar) then
    return 'form'
  elseif ffi.istype(CArray, luaVar) or ffi.istype(CMap, luaVar) or ffi.istype(CFormMap, luaVar) then
    return 'object'
  end

end

The issue with Lua is that there are no distinction between float and integer values. And, when the value is being transported back from Lua to JC it's impossible to know whether the value is intended to be an integer or a float. (A reason why I had plans to unite float & integer type in JC into a number)

[ryobg]

@RealAntithesis ok, let me know later.

@SilverIce I think valueType should be consistent with the one from Papyrus i.e. return the same enumaration & integer values. I'm also confused of why you prefer this approach rather than the native func? Is it because you can insert in JArray, a Lua types?

[SilverIce]

@ryobg I have updated my answer

when the value is being transported back from Lua to JC it's impossible to know whether the value is intended to be an integer or a float. (A reason why I had plans to unite float & integer type in JC into a number)

[SilverIce]

Is it because you can insert in JArray, a Lua types?

No, it can't be done. It's consistent to work with Lua types from the Lua, in Lua way (my function mimics the type function).

[ryobg]

@SilverIce So... if you have in Lua code push into a container, you do not know is it float or a integer? What if we fix the behaviour to one of these two types (say integer) and then provide separate functions for the specific type?

[SilverIce]

@ryobg then you'd had to redo my code, remove unification, provide bunch of setInt, setFlt for a dynamic language, whole point of that language is to not have a strict typing

[SilverIce]

JSON doesn't have a distinction between int and float as well - that's why I had plans (but had to abandon due to backward compatibility) to unite float & integer type in JC into a number

[ryobg]

Well, I was wondering how to fix the communication between Lua and JC. Because what you do in Lua has and must be bound to JC so at certain point there should be some conversion. I'm just thinking aloud. Maybe your idea for custom class is better - I dunno.

[SilverIce]

@ryobg the best way would be to unite float and integer in the JC into single number type (and finally introduce boolean type), but I don't know how much projects rely on it..

JArray.valueType is a simple and straightforward solution, although it exposes true JC type. While collection[i] returns Lua variable - and this is inconsistence

[ryobg]

Maybe we better stick with just one of the valueType variants then. People are already used to it.

[SilverIce]

Well, currently when the value is being transported back to JC it's being analyzed, so for ex. 1.0 being recognized as an integer, and 2.5 as a float

[ryobg]

Good to know, but I'm reluctant to the introduce changes in that system, so if a change, it will be only the valueType function. Then we shall see what the future holds.

[RealAntithesis]

Is there a tentative date for release of the next version of jc with the valueType function? My lua code now uses this function to avoid generating errors. The next release of my mod is probably a few weeks away, so no rush.

[ryobg]

Hi back, there is no actual release targeted. Will gather some material before hand. Though looking at your requests it may be of worth :+1: Anyway, it will be more or less, if not the same, solution as yours. So I doubt you will have to worry here.