search

traversc / qs

Quick serialization of R objects
397 stars 19 forks source link

Feature Request: function to read attributes #74

Closed randrescastaneda closed 1 year ago

randrescastaneda commented 1 year ago

First of all, thank you for this great package. It is everything I use to store and serialize R objects.

It would be great to have a function that works in a similar way to fst::metadata_fst() to read the attributes of any qs object. If it is not possible to read user-written attributes, at least it would be great to be able to read basic attributes like dim, nrows, etc, without having to load the file in memory. Sometimes the only thing that one needs is some information about the data, and not the data itself, so loading it is a waste of time and resources.

I am aware of issue #66 , but my request is different. It is to read the attributes, not to create them in any fashion. That's up to the user.

Once again, thank you so much for this great package.

PS: Are yo planning to create a package site ala pkgdown or something like it? just asking... I think such a great package should have a more user-friendly documentation. ;)

traversc commented 1 year ago

It's a good idea, but there's an issue in that the attributes are stored at the end of an object, so the function would have to at least read through the entire file.

This wouldn't take as long as deserializing the entire object, but it could still be considerable. So as long as you're aware and OK with that, I can work on implementing it.

I haven't used pkgdown before, but I'll look into it! Thanks!

randrescastaneda commented 1 year ago

I think it would be great! If I understand correctly, reading through the entire file is still faster than having to load the file into memory, specially for big files. Is that correctly?

Also, I was thinking that maybe it would be possible to save the serialized data and the attributes independently from each other (or at least allow the user to do so when saving). In this way, when reading, the attributes could be loaded separately from the data. It would something like saving two file into one...

In any case, your solution seems fantastic.

Thanks.

traversc commented 1 year ago

New version on CRAN has this feature. Check out the qattributes function.

https://cran.r-project.org/web/packages/qs/index.html

randrescastaneda commented 1 year ago

Marvelous! Thank you so much.