janet-lang / spork

Various Janet utility modules - the official "Contrib" library.
MIT License
119 stars 35 forks source link

changed JanetTArrayView to use JanetBuffer #146

Closed kamisori closed 1 year ago

bakpakin commented 1 year ago

Definitely breaking backwards compatibility but IMO worth it. The only issue with the native buffer is a 2GB limit, which may be an issue

kamisori commented 1 year ago

i see, i didnt pay attention about how large the two buffertypes could get... perhaps capa and count could be moved into a struct in a union with a sizet size? (but that would break even more ..) like so:

struct SizeInfo {
    int32_t count;
    int32_t capacity;    
};

struct JanetBuffer {
    JanetGCObject gc;

    union {
        SizeInfo s;
        size_t size;
    } sizing;

    uint8_t *data;
};

disregarding the bad var names, this way the JanetBuffer type could stay the same size. when using it as a string buffer it could behave as before, except accessing count and capa via sizing, and in a typed array it could use the 64bit size. (of course this complicates things i had in mind where you could just write into a buffer and then plug it into a ta-view -.-)

(edit: though now i think it would be better to just attach another buffer to the view)

as it is now when calling ta/properties, the buffer is always shown as empty since count stays 0. but i would rather only print the first and last couple of bytes (if at all). similar to how large arrays are printed.

another thing with potential for bugs (at least in heteroendian (is that even a word? XD) systems/networks): i moved the endianess-holding-flag from buffer to view; perhaps endianess could be resolved on the fly by the view upon access, just like the datatype, this way you could have one buffer and interpret it in many different ways. in this case i would add a function to write a copy of the buffer with the other endianess baked in for a permanent speedup

also there is definetly a bug in here, i get kinda random crashes with medium sized tarrays (10k+ doubles) while iterating over multiple such tarrays (read from different sources) in order to compare them.

kamisori commented 1 year ago

have yet to figure this out. the test i added iterates over a table, in this table are 4 identical tarrays, for iterates over the range of the arrays, inside the forloop is eachp eachk or each (tried them all but its all the same template function anyways..) iterating over the table. and after some amount of iteration (have observed anywhere between 2600 and 9600 but the higher number was achieved by calling getline before the innerloop @_@ ), the key from eachp/eachk is either garbage (usualy a very small double like 1.02255e-312) and the loops continue for a bit, or it just crashes... im certain it is my code that is doing silly things, but by now i have run out of ideas where or how to look... i hope monday will be a better day to debug this ._.

bakpakin commented 1 year ago

A number of issues looking at the code, don't use janet_buffer_init to get a garbage collected buffer. Also things like returning buffer->capacity instead of buffer->count for the buffer length seems wrong