Open zeinabsobhani opened 5 months ago
Pandaral·lel is looking for a maintainer! If you are interested, please open an GitHub issue.
I'm having the same issues with Python 3.9 on Windows 10 Pro Pandarallel 1.6.5. In my case I have a very large spatial dataframe that is broken into chuncks using numpy array_split. The process runs through each chunk and does it's thing but the last chunk always has 2 processes frozen at 0% out of the 8.
@zeinabsobhani Could you please include a sample dataframe to accompany your functions, along with all of the imports, and most importantly, the code where you are running progress_apply
? Providing a minimal code sample to reproduce the bug would be super useful.
@tveinot Do you still have this issue? If so, please open a separate issue and include a minimal but self-contained code snippet to reproduce the bug, along with a link to obtain the aforementioned large spatial dataframe. Opening a separate issue would help keep things organized, especially once we get in the technical weeds of trying to resolve both of your issues.
@zeinabsobhani Sorry, this was a while ago for me and I had forgotten that I mentioned my issue here thinking it might be relevant. I either had an issue with a geometry causing the memory to over run (which is my personal belief), or rebuilding my environment and updating to QGIS 3.36 along with it's python components including geopandas somehow resolved it.
General
Acknowledgement
pandas
without alone (withoutpandarallel
)Bug description
Observed behavior
I have 2 function that I'm running with
parallel_apply
on my dataframe. Here are the functions:The first method runs ok with
parallel_apply
, but the second method gets stuck at 0% without raising any error.