Question: Trying to use Pyasn from Jython 2.5.3, is it possible?

[r-lomba]

I'm writing a log parser using Apache Pig, and I'd like to use Pyasn to extract ASN information from IP addresses.

Normally, to use a Python extension in Apache Pig, you write a Python UDF and then register it in Pig "as jython"

So, this is what I did:

1) I successfully installed pyasn (in a previous try via pip-install, currently I have installed it manually, but still it doesn't work)

2) Wrote a Python UDF as follows:

#!/usr/bin/python

import sys
sys.path.append('/usr/lib64/python2.6/site-packages/')
sys.path.append('/usr/lib64/python2.6/site-packages/pyasn-1.6.0b1-py2.6-linux-x86_64.egg/')
sys.path.append('/usr/lib/python2.6/site-packages/')

import pyasn

@outputSchema("asn:chararray")
def asnLookup(ip):
    asndb = pyasn.pyasn('asn.dat')
    asn = asndb.lookup(ip)

    return asn

@outputSchema("asn_prefix:chararray")
def asnGetAsPrefixes(nbr):
    asndb = pyasn.pyasn('asn.dat')
    asn_prefix = asndb.get_as_prefixes(nbr)

    return asn_prefix

3) But when I try to register the UDF in Pig, I get the following exception:

grunt> register 'hdfs:///user/xxxxxx/LIB/PYASN/python_pyasn.py' using jython as pythonPyasn;
...
2017-11-23 17:18:13,236 [main] ERROR org.apache.pig.tools.grunt.Grunt - ERROR 1121: Python Error. Traceback (most recent call last):
  File "/tmp/pig6864734086775637011tmp/python_pyasn.py", line 8, in <module>
    import pyasn
  File "/usr/lib64/python2.6/site-packages/pyasn-1.6.0b1-py2.6-linux-x86_64.egg/pyasn/__init__.py", line 20
SyntaxError: future feature print_function is not defined

I believe this is an incompatibility with Jython, and I've been trying extensively to make it work using a few workarounds and searching the web for possible solutions, but no success.

Any ideas?

Thanks!

[hadiasghari]

Hi @r-lomba, the problem seems to be that the from future import print_funciton statement at the top of the file is not supported. This might be because of the Jython version -- can you try 2.7?

[r-lomba]

Thanks for the speed-of-light reply, @hadiasghari ! I'll se what I can do about upgrading jython and post my findings. Unfortunately I won't be as quick as you've been, I'll be off at this customer's in a few days. Thanks, until then

[hadiasghari]

I assume this has been since resolved.