support array-like tables better

[kmike]

What do you think about adding _LuaTable.looks_like_array(max_sparse_ratio=1.0) and _LuaTable.to_list() methods?

_LuaTable.looks_like_array() may do something along the lines of https://github.com/mpx/lua-cjson/blob/d59326b2d718e1a140b9b396ffe0a557b2d93fe0/lua_cjson.c#L491

_LuaTable.to_list() may return a Python list (tbl[1] == lst[0]), replacing nil values with None.

[scoder]

Sounds good to me. AFAICT, there isn't currently a good way to convert tables with nil values. The nil values always get in the way of the iteration.

Should the to_list() method also have a sparseness limit (for safety)? And raise an exception if the limit is exceeded?

Should we name it map_to_list() to make it clear that it doesn't just copy the values? (as list(tbl.values()) would)

[scoder]

Or maybe to_list(max_gap=...)? Counting up against a global ratio might not work well if there is a large gap followed only by sequential indices. Or maybe both?

[pablomayobre]

What about specifying the length of how many items it should copy? That way you can do something like: for i=1, length do, if length is not specified copy until a nil value is reached... dont know what the original idea was but that is my 2 cents

[kmike]

Should the to_list() method also have a sparseness limit (for safety)? And raise an exception if the limit is exceeded?

Hm, default sparseness limit can get in a way here:

if tbl.looks_like_array(max_sparse_ratio=2.0):
    res = tbl.to_array()  # must write the same limit here, not DRY
else:
    res = dict(tbl)

If the intention is to raise an exception for very sparse arrays then it will help.

Should we name it map_to_list() to make it clear that it doesn't just copy the values? (as list(tbl.values()) would)

Yeah, tolist() may discard some values and insert extra None values. But I'm not sure map_to_list is much better.

Or maybe to_list(max_gap=...)? Counting up against a global ratio might not work well if there is a large gap followed only by sequential indices. Or maybe both?

When might a global ratio not work well?

What about specifying the length of how many items it should copy? That way you can do something like: for i=1, length do, if length is not specified copy until a nil value is reached... dont know what the original idea was but that is my 2 cents

The code in lua-cjson doesn't use length and does support occasional nil values. With max_sparse_ratio=1.0 it should have the same effect as stopping at the first nil value.

My main reservation now is that code which relies on looks_like_array is risky because the result is not well-defined for empty tables. From this point of view @scoder's tolist(max_sparse_ratio=.../max_gap=...) without looks_like_array method may be the best.