Closed GindaChen closed 3 years ago
After some serious digging, I found some knobes that could be the potential cause:
pyarrow
version. Using HDFS
is successful, but pyarrow
remains a problem. This could be caused by hdfs library used by pyarrow. See #5. Indeed, it is the version unmatch. No where in the doc has explain this prior.
Recently,I had encountered with the similar problem, and pls tell me now what shall I do to take advantage of this problem? My hadoop version is 3.1.2
@IapHubery - I may have out-dated memory about this hadoop setup problem. I vaguely remember I solve this by checking all the installed packages share the same hadoop version (2.10.1). See this comment.
If in your project there is no specific hadoop version you must use, perhaps the best way to approach is to switch to Hadoop 2.10.1?
I am stuck with this error:
will throw
with the log