steineggerlab / foldcomp

Compressing protein structures effectively with torsion angles
GNU General Public License v3.0
152 stars 14 forks source link

Multithreaded database query #36

Open valentynbez opened 1 year ago

valentynbez commented 1 year ago

Hello! I use Python foldcomp interface. Sometimes I need to retrieve 10k structures from a DB. As for now, I can only do it one by one in a for loop. Could this be sped up using multithreading somehow? Thanks!

EDIT: I guess I solved it this way, querying db doesn't take much RAM

import foldcomp 
from multiprocess import Pool

def retrieve_structure(idx, afdb):
    with foldcomp.open(afdb, ids=[idx]) as db:
        for i, pdb in db:
            structure = pdb 
    return structure

with Pool(2) as p:
    structures = p.map(retrieve_structure, ids)