search

Public-Health-Scotland / phsmethods

An R package to standardise methods used in Public Health Scotland (https://public-health-scotland.github.io/phsmethods/)
https://public-health-scotland.github.io/phsmethods/
54 stars 13 forks source link

fin_year performance issue #21

Closed chrisdeans closed 4 years ago

chrisdeans commented 4 years ago

fin_year is slow when dealing with moderate/large data sets. For example, it takes ~40 seconds to process 10 million dates (which is about the same number of rows as 10 years of SMR01).

It can be sped up substantially by only converting the unique dates in the set, then matching back to the original set. Testing this, I get it at ~1 second for the same 10 million rows.

davidc92 commented 4 years ago

@chrisdeans This sounds reasonable to me. I'll update the function sometime next week. 👍