search

ckan / ckanapi

A command line interface and Python module for accessing the CKAN Action API
Other
182 stars 75 forks source link

Document and expose parallel worker limits #169

Closed jqnatividad closed 3 years ago

jqnatividad commented 4 years ago

dump and load jobs can be run in parallel with multiple worker processes by using the -p parameter to specify the number of processes.

There are parameters in the code however - PARALLEL_LIMIT and MY_SITES that need to be documented so users can modify this behavior.

Presently, tweaking these values require touching code, and is not easily modifiable when ckanapi is installed from pypi.

Perhaps, MY_SITES can be added as a CLI parameter, with localhost always being included?

wardi commented 4 years ago

How about CKANAPI_PARALLEL_LIMIT and CKANAPI_MY_SITES environment variables. :+1: to localhost assumed for MY_SITES