search

arogozhnikov / python3_with_pleasure

A short guide on features of Python 3 with examples
3.63k stars 242 forks source link

Numpy advises against pickle for numpy objects #15

Closed phrz closed 6 years ago

phrz commented 6 years ago

Although the pickle example is great for showing off the increased efficiency of pickle v3, numpy outright recommends against pickling numpy arrays etc. (they explain why to use .save instead here). I worry that people may internalize some of the examples here and end up pickling numpy objects. Furthermore, pickling numpy objects as well as most other things presents a special problem to data scientists etc. who are in the middle of migrating 2-to-3: pickle v2 and v3 are mutually incompatible, unless you explicitly request backwards compatibility.

If you're open to modifying that example's code, or adding an aside that mentions the risks of pickle for security and compatibility, it may be helpful to those reading.

I really appreciate you preparing this document!

arogozhnikov commented 6 years ago

Added a comment, and a reason why this feature is important (pickle used e.g. to pass objects in multiprocessing)