jeroen / protolite

Fast and Simple Object Serialization to Protocol Buffers
Other
48 stars 9 forks source link

High-performance it really ain't #1

Closed eddelbuettel closed 8 years ago

eddelbuettel commented 8 years ago

You may want to consider RApiSerialize which, after all, was written for this purpose (and inherited from another HPC application).

See the benchmark in my updated answer here.

(I see the DESCRIPTION got edited. Good. If it were to focus on, say, "smaller" or "light-weight" it may be truthful. "Fast" it ain't.)

jeroen commented 8 years ago

The serialize stuff is only a fallback for non-data objects that cannot be serialized by protobuf. The 'optimized' refers to the implementation of the protobuf serializer:

> library(ggplot2)
> data(diamonds)
> system.time(RProtoBuf::serialize_pb(diamonds, NULL))
   user  system elapsed 
  0.043   0.001   0.044 
> system.time(protolite::serialize_pb(diamonds, NULL))
   user  system elapsed 
  0.008   0.000   0.008 

The difference is even larger for character vectors:

> x <- as.character(diamonds$cut)
> system.time(protolite::serialize_pb(x, NULL))
   user  system elapsed 
  0.023   0.002   0.025 
> system.time(RProtoBuf::serialize_pb(x, NULL))
   user  system elapsed 
  6.525   0.029   6.555 
eddelbuettel commented 8 years ago

Fine. You still have no reason to use the Rcpp::Environment kludge to serialize to raw given that a dedicated C implementation exists.

But your package so I don't give a fiddle.