Closed SamBryce-Smith closed 3 years ago
final solution was this. Confusingly keep="last" sets the last as False and others True, so don't need to invert the mask to remove last exons.
# Filter out last exons for each transcript (max exon_number)
# & first exons for each transcript (exon_number == 1)
out_gr = (exons_gr.subset(lambda df: (df[region_number_col].ne(1).astype(bool)) &
(df.duplicated(subset=["transcript_id"], keep="last")), # sets last to 'False'
nb_cpu=nb_cpu
)
)
Shaves off ~ 10s single threaded, multithreaded ~ 10s slower so am deprecating multiple threads for this function.
Also renamed to more general get_internal_regions
.
closed with commit 2cc9c1109816e720ecb3a94de11e2ec846d94d87
Group by turns the filtering into a looping operation which will be slow with many Txs (ref internal exons takes ~ 37s single-threaded )- I think this could be quicker.
Possible option is to make use of duplicated just on the transcript_id column, keeping the 'last' region_number row. THis boolean mask/Series can be negated to avoid selecting the last region for each group. I suspect groupby.idxmax() may already be quite efficient though so maybe minor speedup (especially if enforce sort for safety).
i.e. something like this