search

Dzeiberg / MutPred2_DB

2 stars 0 forks source link

Filtering out mutations #5

Open Dzeiberg opened 10 months ago

Dzeiberg commented 10 months ago

If not overwriting, filter out mutations that are present in all tables, additionally, add check to write_* that checks before adding.

Dzeiberg commented 10 months ago

only do filtering out in the write functions

Dzeiberg commented 10 months ago

@trberg , I added logic to filter out records right before I write to the sql table in commit 8c58a9e81b0fe6745457ad371e81634d8f9de937

I know log to the user how many rows are being skipped for each table.

This does cause Processor to take a long time to initialize, as I have to query the seq_hash,mutation pairs from all tables. If you're processing a large number of sets of mutations then this isn't significant, but for a single set of mutations this initialization time is quite long.