bovee
commented
2 years ago I can reproduce this pretty reliably on my computer with e.g. r <- Reader("./dad1.uv"); r[r["time"] < 0,] and then I see a row with negative times (and several rows with negative wavelengths; the Python client has none of these erroneous rows). In each case it appears that the value is copied from a nearby row or column (e.g. many of the bad wavelengths are just copies of the intensity from the row). I also don't see any of the new metadata even though I cleared the package out of ~/R/.../entab and rebuilt after deleting the cached libentab.so and target (I do see it with the CLI and Python client).
I thought maybe this was because of some memory unsafety in the vec_to_list/vec_to_dataframe shims I had, but I refactored everything and removed all of the unsafe code in e7b6647b4d8169bf784e91dcd59f09852fb705fb and it's still happening.
I think it could still be one of a couple causes:
- Using the Rust code as a dynamic library directly is causing issues somehow? Other examples have a dynamic C wrapper that calls the static Rust library.
- There's a subtle bug in extendr? We're creating a new
Robjfor each value and it's possible that sometimes extendr is not reserving a new one, but reusing an old one because of thread safety or GC issues? Strictly speaking, it would probably be faster and better to coerce the values into the proper type in Rust usingToVectorValueinstead of having to callunliston the columns in R, but that's kind of complex to implement.
ethanbass
There's an intermittent issue when using the R bindings where fields will be out-of-order and parse speed will be abnormally slow. From #28:
I thought this might be an issue with R linking to an old version of the dynamic library, but it's also possible it's an issue with all the unsafe libR code or something else entirely.