I think that scanning the filesystem for protocols may be relatively slow. It becomes most noticeable in long pipelines, where each command needs to repeat this scan. The simplest way to speed this up is to have the first process cache the paths to all the protocols and to make this available to other processes via the pickle.
My first step would be to run the profiler, because I don't actually know if searching the filesystem takes an significant amount of time (although I suspect it does). Whatever the profiler finds, I might be able to cache in this manner.
I think that scanning the filesystem for protocols may be relatively slow. It becomes most noticeable in long pipelines, where each command needs to repeat this scan. The simplest way to speed this up is to have the first process cache the paths to all the protocols and to make this available to other processes via the pickle.
My first step would be to run the profiler, because I don't actually know if searching the filesystem takes an significant amount of time (although I suspect it does). Whatever the profiler finds, I might be able to cache in this manner.