search

Yelp / mrjob

Run MapReduce jobs on Hadoop or Amazon Web Services
http://packages.python.org/mrjob/
Other
2.62k stars 586 forks source link

Is there any way to connect to a remote Hadoop cluster? #2180

Open FlorinAndrei opened 4 years ago

FlorinAndrei commented 4 years ago

Can't find any trace of this in the docs.

Can I run a Python script with mrjob on my laptop, and have it connect to a remote Hadoop cluster over VPN, run the mapreduce job there, get the results back on my local system? I can connect to any Hadoop TCP port just fine from my laptop.

Is there a config or code example for this?

Thanks!

JellyBai commented 4 years ago

Facing the same issue.