epinowcast / hashprng

Hash-based Matching Pseudo-Random Number Generation
https://hashprng.epinowcast.org/dev/
Other
2 stars 0 forks source link

Aims to speed up `hash_salt` by using unlist vs Reduce #13

Closed seabbs closed 1 year ago

seabbs commented 1 year ago

As discussed in #2 this PR switches out Reduce forunlist`. I am not clear if this makes sense but everything appears to work on the face of it.

pearsonca commented 1 year ago

This does seem to work, but only seems to be a ~10% improvement in that step.

seabbs commented 1 year ago

This does seem to work, but only seems to be a ~10% improvement in that step.

For sure write_bin is the real cost of that step. It looks like it can be vectorised if given the same input so perhaps that should be a focus (sort input and use as vectorised - then combine)?

pearsonca commented 1 year ago

Have checked, lapply + reassemble is ~100x slower than vectorized version.

The problem is that the ellipsis argument approach doesn't give an easily vectorizable collection as an argument. Can give that functionality up and get back substantial performance.

seabbs commented 1 year ago

What about if you take everything in using list(...) and then sort into type, use write_bin on each vectorised and then concatenate? the precise order of the args shouldn't matter? You might need a failsafe we don't know the type section that uses the current approach for safety as well.

That would assume that most common args are sortable etc. but I think they probably are? Or we could restrict hash_salt to just a few argument types that we have tested?