A workflow for construction of Gene Expression count Matrices (GEMs). Useful for Differential Gene Expression (DGE) analysis and Gene Co-Expression Network (GCN) construction
The create-gem.py script currently uses the pandasmerge function, which is able to merge on any column. It turns out that the pandasjoin function is substantially faster, but is slightly more restrictive in that you can only join on the index.
join is essentially instantaneous, even for large matrices, where as merge becomes slow with large matrices. This is most important when you are dealing with massive matrices (greater than ~30,000 samples) where merge gets rather slow.
Description of feature
The create-gem.py script currently uses the
pandasmergefunction, which is able to merge on any column. It turns out that thepandasjoinfunction is substantially faster, but is slightly more restrictive in that you can only join on the index.joinis essentially instantaneous, even for large matrices, where asmergebecomes slow with large matrices. This is most important when you are dealing with massive matrices (greater than ~30,000 samples) where merge gets rather slow.