search

globalgov / manydata

The portal for global governance data
https://manydata.ch
GNU Affero General Public License v3.0
9 stars 0 forks source link

Consider `{data.table}`to accelerate some backend functionality as finding duplicates and removing NAs as well `{collapse}` for faster joins #288

Open henriquesposito opened 1 year ago

henriquesposito commented 1 year ago

see https://www.spsanderson.com/steveondata/posts/2023-07-18/index.html

henriquesposito commented 1 year ago

I assume we will need to refactor the code for most of the consolidate() function and for the helper functions to avoid converting back-and-fourth from data frame to tibble. Perhaps even make changes in how we resolve dates {messydates}.

henriquesposito commented 1 year ago

https://www.r-bloggers.com/2023/10/releasing-collapse-2-0-blazing-fast-joins-reshaping-and-enhanced-r/