HIINTEL / bingreward

Run daily script to generate gift card
bing.com/reward
GNU Lesser General Public License v3.0
5 stars 1 forks source link

docker container fails to run multiprocess #1

Open kenneyhe opened 7 years ago

kenneyhe commented 7 years ago

symptom: when running mpmain.py in docker container in mp, mp_faas, mp_faas_proxy

bash-4.3# python -m profile  mpmain.py
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/runpy.py", line 174, in _run_module_as_main
    "__main__", fname, loader, pkg_name)
  File "/usr/local/lib/python2.7/runpy.py", line 72, in _run_code
    exec code in run_globals
  File "/usr/local/lib/python2.7/profile.py", line 610, in <module>
    main()
  File "/usr/local/lib/python2.7/profile.py", line 603, in main
    runctx(code, globs, None, options.outfile, options.sort)
  File "/usr/local/lib/python2.7/profile.py", line 77, in runctx
    prof = prof.runctx(statement, globals, locals)
  File "/usr/local/lib/python2.7/profile.py", line 444, in runctx
    exec cmd in globals, locals
  File "mpmain.py", line 114, in <module>
    run(1, "config.xml")
  File "mpmain.py", line 65, in run
    return pool.map(helper, run_list)
  File "/usr/local/lib/python2.7/site-packages/pathos/multiprocessing.py", line 137, in map
    return _pool.map(star(f), zip(*args)) # chunksize
  File "/usr/local/lib/python2.7/site-packages/multiprocess/pool.py", line 251, in map
    return self.map_async(func, iterable, chunksize).get()
  File "/usr/local/lib/python2.7/site-packages/multiprocess/pool.py", line 567, in get
    raise self._value
cPickle.PicklingError: Can't pickle <type 'function'>: attribute lookup __builtin__.function failed
kenneyhe commented 7 years ago

Changed to using ubuntu 16.04 seem to fix it. Dockerfile below

FROM nishidayuya/docker-vagrant-ubuntu:16.04

Following label schema convention described at http://label-schema.org/rc1/

LABEL \ org.label-schema.schema-version="1.0" \ org.label-schema.name="BingRewards" \ org.label-schema.vcs-url="http://github.com/sealemar/BingRewards" \ org.label-schema.description="BingRewards is an automated point earning script that works with Bing.com to earn points that can be redeemed for giftcards." \ org.label-schema.docker.cmd="docker run -it --rm -v ${PWD}/config.xml:/usr/src/app/config.xml kenney/bingreward " \ org.label-schema.docker.maintainer="Kenney He kenneyhe@gmail.com"

RUN apt-get update RUN apt-get upgrade -y RUN apt-get install -y python-pip RUN pip install --upgrade pip COPY . /usr/src/app RUN find . -iname *pyc -exec rm {} \; WORKDIR /usr/src/app RUN cp config.xml.dist config.xml RUN chmod og-r config.xml RUN pip install -r requirements.txt

CMD ["python", "mpmain.py", "3"]