Improved load of IDs to temp table, avoids skipping some IDs

PHSKC-APDE / claims_data

Process and analyze WA State Medicaid, Medicare, and All-Payer Claims Database eligibility and claims data

21 stars 4 forks source link

Improved load of IDs to temp table, avoids skipping some IDs #136

Closed alastairmatheson closed 2 years ago

alastairmatheson commented 2 years ago

The elig_timevar_collapse function had been skipping some IDs when loading to the temp table. Instead of inserting each ID though code, now using dbWriteTable to load in a single go. Extremely large numbers of IDs may encounter network issues when loading to the SQL DB, but this is yet to be shown. Added a QA check to flag discrepancies.