mmckerns
commented
5 years ago Sorry for the slow response on your question. Hopefully you were able to figure it out.
In case not... pathos.ParallelPool is a high-level interface to Parallel Python (ppft), and creates a ppft.Server that the pool farms jobs to. In ppft, you can also create and register your own Server instances, where the loglevel for the Server is handled by the logging module -- hence, you can set the logging level of the logger attached to a Server instance:
>>> import ppft as pp
>>> s = pp.Server()
>>> import logging
>>> s.logger.setLevel(logging.WARNING) If you are using the ppserver script to launch a Server, there's also a flag (-d) and a setting in the config file loglevel that can toggle the logging from logging.DEBUG to logging.WARNING -- where the latter makes the server much less verbose.
"mymethod" called in "syncoffer" function is decorated to log the statuscode of an api request and it works perfectly but pp writes in my logfile and I don't understand how and why I get these messages together with messages logged via my decorator. I was not able to find explainations or examples in your issues section neither googling, so I hope you can show me at least how to config pp to don't log because I am going to generate very large logfiles with this app Please be patient...I am totally self tought in coding
2018-10-05 10:45:34,940 pp Creating server instance (pp-1.6.4.8) 2018-10-05 10:45:34,940 pp Running on Python 2.7.12 linux2 2018-10-05 10:45:36,181 pp pp local server started with 4 workers 2018-10-05 10:45:38,178 pp Task 0 started 2018-10-05 10:45:38,180 pp Task 1 started 2018-10-05 10:45:38,182 pp Task 2 started 2018-10-05 10:45:38,183 pp Task 3 started 2018-10-05 10:45:39,158 root URL/XXXXXX STATUS 200
`from pathos.pp import ParallelPool from multiprocessing import Process , cpu_count from myfile import Myclass
myobject = Myclass()
def syncoffer( item ) : return myobject.mymethod( item )
class Parallyzed( object ) : def init( self ) : self.p = ParallelPool( cpu_count() )
def cbparallelprocess( self , func , items ) : procs = [ Process( target=func , args=( item , ) ) for item in items ] for proc in procs: proc.start() proc.join()
def sync_offer( self , items ) : return self.p.map( syncoffer , ( i for i in items ) )`