Using unpatched ThreadPoolExecutor when using gevent

[barsa-net]

I'm working on a project using gevent and I'd need to use gevent.threadpool.ThreadPoolExecutor since gevent patch the concurrent.futures.ThreadPoolExecutor class.

I'd be more than happy to open a pull request, but I'm not sure how would fit best the library.

• Simplest way would be to just add something inside _make_executor like:

      elif executor_type == 'gevent.threadpool':
          import gevent.threadpool
          _executor = gevent.threadpool.ThreadPoolExecutor

• Another way would be to enable users to pass their own implementation, adding 'custom' as EXECUTOR_TYPE and passing the actual executor class via EXECUTOR_POOL_CLASS

• Last way I can think of, would be modifying _make_executor making it gevent-aware to something like:

      if executor_type == 'thread':
          try:
              import gevent.monkey
              if gevent.monkey.is_module_patched('threading'):
                  import gevent.threadpool
                  _executor = gevent.threadpool.ThreadPoolExecutor
              else:
                  _executor = concurrent.futures.ThreadPoolExecutor
          except ImportError:
              _executor = concurrent.futures.ThreadPoolExecutor
      elif executor_type == 'process':
          _executor = concurrent.futures.ProcessPoolExecutor
      elif executor_type == 'gevent':
          # That would hopefully be the gevent-patched version, some check could be added
          _executor = concurrent.futures.ThreadPoolExecutor

There are some cavet though, gevent.threadpool.ThreadPoolExecutor uses a slightly modified Future object which doesn't have the following methods:

• cancel
• cancelled
• done
• running
• set_exception
• set_result

[dchevell]

Cool idea @barsa-net . I'm not against supporting this, though rather than explicitly hardcoding support for gevent I'd prefer to be able to just pass in one's own compatible class. Your idea of EXECUTOR_TYPE = 'custom' and EXECUTOR_POOL_CLASS = gevent.threadpool.ThreadPoolExecutor could certainly work.

I don't know what the implications would be of gevent's different Future objects. Perhaps you can test out your approach of allowing a custom executor pool class to be used, and see if you run into trouble. If it did work, then we could include gevent as a testing requirement and write some tests for that example of the feature.