How can I clone the GIS-tool-for-hadoop repository in Cloudera Virtualbox VM?

[Songkorn]

Hello,

I am quite new to hadoop and github. I have read the tutorial of GIS Tools for Hadoop for Beginners. It is for the Hortonworks. Do you have any suggestion on cloning github repository for Cloudera VM?

Regards, Songkorn

[randallwhitman]

I'd guess the VM for both Hortonworks and Cloudera would have a command-prompt/shell/terminal. Not sure what if any relevant differences there could be.

[Songkorn]

Thank you very much for your reply, randallwhitman.

I have found the problem when cloning the repository as below. I am not sure how to solve this problem.

If anyone knows the way to solve this, please let me know.

================================================ [cloudera@quickstart esri-git]$ ping github.com PING github.com (192.30.253.113) 56(84) bytes of data. 64 bytes from lb-192-30-253-113-iad.github.com (192.30.253.113): icmp_seq=1 ttl=46 time=301 ms 64 bytes from lb-192-30-253-113-iad.github.com (192.30.253.113): icmp_seq=2 ttl=46 time=301 ms 64 bytes from lb-192-30-253-113-iad.github.com (192.30.253.113): icmp_seq=3 ttl=46 time=300 ms 64 bytes from lb-192-30-253-113-iad.github.com (192.30.253.113): icmp_seq=4 ttl=46 time=300 ms ^C --- github.com ping statistics --- 4 packets transmitted, 4 received, 0% packet loss, time 3891ms rtt min/avg/max/mdev = 300.730/301.041/301.397/0.246 ms

[cloudera@quickstart esri-git]$ git clone git@192.30.253.113:Esri/gis-tools-for-hadoop.git Initialized empty Git repository in /home/cloudera/esri-git/gis-tools-for-hadoop/.git/ The authenticity of host '192.30.253.113 (192.30.253.113)' can't be established. RSA key fingerprint is 16:27:ac:a5:76:28:2d:36:63:1b:56:4d:eb:df:a6:48. Are you sure you want to continue connecting (yes/no)? yes Warning: Permanently added '192.30.253.113' (RSA) to the list of known hosts. Permission denied (publickey). fatal: The remote end hung up unexpectedly [cloudera@quickstart esri-git]$

==================================================

Regards, Songkorn

[Songkorn]

Finally, I could clone the repository as below:

[cloudera@quickstart esri-git]$ git clone -v git://192.30.253.113/ESRI/gis-tools-for-hadoop.git Initialized empty Git repository in /home/cloudera/esri-git/gis-tools-for-hadoop/.git/ remote: Counting objects: 1677, done. remote: Total 1677 (delta 0), reused 0 (delta 0), pack-reused 1677 Receiving objects: 100% (1677/1677), 18.46 MiB | 2.09 MiB/s, done. Resolving deltas: 100% (787/787), done. [cloudera@quickstart esri-git]$ ls -lag total 12 drwxrwxr-x 3 cloudera 4096 Mar 2 20:51 . drwxrwxr-x 33 cloudera 4096 Mar 2 05:19 .. drwxr-xr-x 4 cloudera 4096 Mar 2 20:51 gis-tools-for-hadoop [cloudera@quickstart esri-git]$

[theronarmandm]

Hi Sonkorn;

 I recently also got started with the Esri tools for hadoop on cloudera and i'm new to both envornments.

 Did you get it right to install the esri jars and get the functions all to work ?
 If so I would be interested in finding out where you stored the Jars and how did you get the Json serdes to work.

Thanks

[randallwhitman]

https://github.com/Esri/gis-tools-for-hadoop/wiki/GIS-Tools-for-Hadoop-for-Beginners <- In that wiki page see the part about getting the IPv4 address of github.com if the VM guest does not have name resolution.

[Songkorn]

Hi,

Below lines are the steps I use ESRI ST functions in hadoop.

// Clone esri-git // Must use the below command to clone the repository // Clone ESRI repository from git https://github.com/Esri/gis-tools-for-hadoop/wiki/GIS-Tools-for-Hadoop-for-Beginners

$ git clone -v git://192.30.253.113/ESRI/gis-tools-for-hadoop.git // Using ST_Geometry in hive // https://github.com/Esri/gis-tools-for-hadoop/tree/master/samples/point-in-polygon-aggregation-hive

set hive.cli.print.header=true; add jar ${env:HOME}/songkorn/esri-git/gis-tools-for-hadoop/samples/lib/esri-geometry-api-2.0.0.jar ${env:HOME}/songkorn/esri-git/gis-tools-for-hadoop/samples/lib/spatial-sdk-hive-2.0.0.jar ${env:HOME}/songkorn/esri-git/gis-tools-for-hadoop/samples/lib/spatial-sdk-json-2.0.0.jar;

create temporary function ST_Point as 'com.esri.hadoop.hive.ST_Point'; create temporary function ST_Contains as 'com.esri.hadoop.hive.ST_Contains'; create temporary function ST_AsText as 'com.esri.hadoop.hive.ST_AsText'; create temporary function ST_Within as 'com.esri.hadoop.hive.ST_Within'; create temporary function ST_Buffer as 'com.esri.hadoop.hive.ST_Buffer';

create temporary function st_geomfromtext as 'com.esri.hadoop.hive.ST_GeomFromText'; create temporary function st_geometrytype as 'com.esri.hadoop.hive.ST_GeometryType'; create temporary function st_point as 'com.esri.hadoop.hive.ST_Point'; create temporary function st_asjson as 'com.esri.hadoop.hive.ST_AsJson'; create temporary function st_asbinary as 'com.esri.hadoop.hive.ST_AsBinary'; create temporary function st_astext as 'com.esri.hadoop.hive.ST_AsText'; create temporary function st_intersects as 'com.esri.hadoop.hive.ST_Intersects'; create temporary function st_x as 'com.esri.hadoop.hive.ST_X'; create temporary function st_y as 'com.esri.hadoop.hive.ST_Y'; create temporary function st_srid as 'com.esri.hadoop.hive.ST_SRID'; create temporary function st_linestring as 'com.esri.hadoop.hive.ST_LineString'; create temporary function st_pointn as 'com.esri.hadoop.hive.ST_PointN'; create temporary function st_startpoint as 'com.esri.hadoop.hive.ST_StartPoint'; create temporary function st_endpoint as 'com.esri.hadoop.hive.ST_EndPoint'; create temporary function st_numpoints as 'com.esri.hadoop.hive.ST_NumPoints';

Hope this helps

Songkorn

On Fri, Jun 29, 2018 at 2:47 PM, theronarmandm notifications@github.com wrote:

Hi Sonkorn;

I recently also got started with the Esri tools for hadoop on cloudera and i'm new to both envornments.

Did you get it right to install the esri jars and get the functions all to work ? If so I would be interested in finding out where you stored the Jars and how did you get the Json serdes to work.

Thanks

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/Esri/gis-tools-for-hadoop/issues/70#issuecomment-401276517, or mute the thread https://github.com/notifications/unsubscribe-auth/AE33XfT1SGWcFbGMdIc3GMf1XoKbIwLeks5uBducgaJpZM4SZ4z6 .

[theronarmandm]

Thanks this worked ....

Now i'm just trying to figure out how to add the jars and functions permanently without having to create it every time one wants to query the data

[randallwhitman]

trying to figure out how to add the jars and functions permanently

@theronarmandm I use .hiverc.