search

nansencenter / sea_ice_drift

Sea ice drift from Sentinel-1 SAR imagery using open source feature tracking
GNU General Public License v3.0
41 stars 17 forks source link

pickle error #14

Closed jthargrove closed 5 years ago

jthargrove commented 5 years ago

hi, I have successfully installed Sea Ice Drift and dependencies using the installation instructions. I also discovered the "Issues" area so you can ignore the personal email I sent you about my installation problems which it appears I solved. I am now trying to run the example. All seems to be fine until I get to sid.get_drift_PM, which gives the following error related to multiprocessing:

upm, vpm, apm, rpm, hpm, lon2pm, lat2pm = sid.get_drift_PM( ... lon1pm, lat1pm, ... lon1ft, lat1ft, ... lon2ft, lat2ft) Traceback (most recent call last): File "", line 4, in File "C:\ProgramData\Anaconda3\envs\py3drift\lib\site-packages\sea_ice_drift\seaicedrift.py", line 88, in get_drift_PM self.n2, x2, y2, **kwargs) File "C:\ProgramData\Anaconda3\envs\py3drift\lib\site-packages\sea_ice_drift\pmlib.py", line 424, in pattern_matching initargs=(c1pm1i[gpi], r1pm1i[gpi], c2fg[gpi], r2fg[gpi], brd2[gpi], img1, img2, img_size, alpha0, kwargs)) File "C:\ProgramData\Anaconda3\envs\py3drift\lib\multiprocessing\context.py", line 119, in Pool context=self.get_context()) File "C:\ProgramData\Anaconda3\envs\py3drift\lib\multiprocessing\pool.py", line 174, in init self._repopulate_pool() File "C:\ProgramData\Anaconda3\envs\py3drift\lib\multiprocessing\pool.py", line 239, in _repopulate_pool w.start() File "C:\ProgramData\Anaconda3\envs\py3drift\lib\multiprocessing\process.py", line 105, in start self._popen = self._Popen(self) File "C:\ProgramData\Anaconda3\envs\py3drift\lib\multiprocessing\context.py", line 322, in _Popen return Popen(process_obj) File "C:\ProgramData\Anaconda3\envs\py3drift\lib\multiprocessing\popen_spawn_win32.py", line 65, in init reduction.dump(process_obj, to_child) File "C:\ProgramData\Anaconda3\envs\py3drift\lib\multiprocessing\reduction.py", line 60, in dump ForkingPickler(file, protocol).dump(obj) AttributeError: Can't pickle local object 'pattern_matching.._init_pool' Traceback (most recent call last): File "", line 1, in File "C:\ProgramData\Anaconda3\envs\py3drift\lib\multiprocessing\spawn.py", line 105, in spawn_main exitcode = _main(fd) File "C:\ProgramData\Anaconda3\envs\py3drift\lib\multiprocessing\spawn.py", line 115, in _main self = reduction.pickle.load(from_parent) EOFError: Ran out of input

akorosov commented 5 years ago

Hi @jthargrove , both the pervious and this errors look weird. I'm afraid they come from the Windows version of Python. And here mu support is limited. I'm developing the code in Linux. Theoretically it should run on both platforms but ...

I can recommend to use a virtual machine with Ubuntu for installation of Python and running the code. Probably the simplest way is to install VirtualBox and Vagrant. Then you can create a simple textfile with name Vagrantfile and with the following content:

Vagrant.configure("2") do |config|
  config.vm.box = "continuumio/anaconda3"
end

and run vagrant up in the directory with this file. Vagrant will download an VM image, create a VM instance and start the VM. Then you can call vagrant ssh to login into the machine and install all the libraries as suggested in the README.

Hmmm, maybe I should add vagrantfile to this repository myself...

akorosov commented 5 years ago

Ah, it is there! There is a Vagrantfile with instructions for all software installation! I forgot I already made it :)

jthargrove commented 5 years ago

Thanks Anton. On further research, I found it's a windows-specific pickle problem as you suggested. For now, I changed a line in pmlib.py to "threads = 0" so that it avoids multi-threading, and this has allowed me to continue to evaluate the code, which now runs successfully. Porting the code to a virtual machine is something I'll leave for the future if I find that my usage demands it.

jthargrove commented 5 years ago

Thanks Anton. On further research, I found it's a windows-specific pickle problem as you suggested. For now, I changed a line in pmlib.py to "threads = 0" so that it avoids multi-threading, and this has allowed me to continue to evaluate the code, which now runs successfully. Porting the code to a virtual machine is something I'll leave for the future if I find that my usage demands it.

From: Anton Korosov [mailto:notifications@github.com] Sent: Wednesday, December 5, 2018 3:05 AM To: nansencenter/sea_ice_drift sea_ice_drift@noreply.github.com Cc: John Hargrove jhargrove@cstars.miami.edu; Mention mention@noreply.github.com Subject: Re: [nansencenter/sea_ice_drift] pickle error (#14)

Ah, it is there! There is a Vagrantfile with instructions for all software installation! I forgot I already made it :)

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnansencenter%2Fsea_ice_drift%2Fissues%2F14%23issuecomment-444394484&data=02%7C01%7Cjhargrove%40cstars.miami.edu%7C5ab57eb331064122026108d65a885748%7C2a144b72f23942d48c0e6f0f17c48e33%7C0%7C0%7C636795938962780431&sdata=Oy0KOLONzFZGcydS5bc15H2M7VAKBShiiIv4gYk3HLw%3D&reserved=0, or mute the threadhttps://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAmsTxGnStyCSW234QxwYGAnOtLawWpINks5u134mgaJpZM4ZAzkM&data=02%7C01%7Cjhargrove%40cstars.miami.edu%7C5ab57eb331064122026108d65a885748%7C2a144b72f23942d48c0e6f0f17c48e33%7C0%7C0%7C636795938962790443&sdata=d4nA9ekckrq8l4jYwGSGlaDywnopPfcQEnHBcUQpYfw%3D&reserved=0.