Create Python Module then Import Module and Call Function Inside Clojure?

[israellandes]

Hi again,

my goal is to be able to run an existing python script (test.py) , inside Clojure using libpython-clj.

My test repo is here:

My understanding is you need to convert your python script into a module, import it as an module in your environment for it to be discoverable/assessable by libpython-clj

I followed this guide slightly to turn my "python-lib" folder into a module labeled "mypythonlib".

I am successfully able to get my "test.py" module/script to load in python:

$ python
Python 3.8.16 (default, Jan 17 2023, 22:25:28) [MSC v.1916 64 bit (AMD64)] :: An
aconda, Inc. on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import mypythonlib
>>> from mypythonlib import test
hello world

In my "core.clj" I have this to import module which returns:

(def mpl (py/import-module "mypythonlib"))

clj꞉clojure-app.core꞉> 
<module 'mypythonlib' from 'C:\\Users\\Is-Rael\\anaconda3\\envs\\clojure-python-app-env\\lib\\site-packages\\mypythonlib\\__init__.py'>

I then tried to apply some functions or methods from the docs to try and get the same 'hello world' to appear (not sure of proper terminology, a breakdown would be helpful too):

(py/from-import "mypythonlib" test)
; Execution error at libpython-clj2.python.ffi/check-error-throw (ffi.clj:708).
; AttributeError: module 'mypythonlib' has no attribute 'test'
; 

My test.py:

def testme():
    print("hello world")
testme()

I'm a bit confused on how to go about solving this issue of generally speaking, importing python scripts in libpython-clj so that they execute/eval in REPL. I though I had to turn them into a module and import, then call in some fashion.

Any tips and help is greatly appreciated thank you!

[cnuernber]

I would scan all the attributes of mypythonlib. The dir function should print them all out and see what attributes are there and if they make sense.

[israellandes]

I'm a bit confused on how to scan all the attributes. I used this functions in clojure to print the module dictionary:

(def mpl (py/import-module "mypythonlib"))
(module-dict mpl)

{'__name__': 'mypythonlib', '__doc__': None, '__package__': 'mypythonlib', '__loader__': <_frozen_importlib_external.SourceFileLoader object at 0x000001BBC2B3A7F0>, '__spec__': ModuleSpec(name='mypythonlib', loader=<_frozen_importlib_external.SourceFileLoader object at 0x000001BBC2B3A7F0>, origin='C:\\Users\\Is-Rael\\anaconda3\\envs\\clojure-python-app-env\\lib\\site-packages\\mypythonlib\\__init__.py', submodule_search_locations=['C:\\Users\\Is-Rael\\anaconda3\\envs\\clojure-python-app-env\\lib\\site-packages\\mypythonlib']), '__path__': ['C:\\Users\\Is-Rael\\anaconda3\\envs\\clojure-python-app-env\\lib\\site-packages\\mypythonlib'], '__file__': 'C:\\Users\\Is-Rael\\anaconda3\\envs\\clojure-python-app-env\\lib\\site-packages\\mypythonlib\\__init__.py', '__cached__': 'C:\\Users\\Is-Rael\\anaconda3\\envs\\clojure-python-app-env\\lib\\site-packages\\mypythonlib\\__pycache__\\__init__.cpython-38.pyc', '__builtins__': {'__name__': 'builtins', '__doc__': "Built-in functions, exceptions, and other objects.\n\nNoteworthy: None is the `nil' object; Ellipsis represents `...' in slices.", '__package__': '', '__loader__': <class '_frozen_importlib.BuiltinImporter'>, '__spec__': ModuleSpec(name='builtins', loader=<class '_frozen_importlib.BuiltinImporter'>), '__build_class__': <built-in function __build_class__>, '__import__': <built-in function __import__>, 'abs': <built-in function abs>, 'all': <built-in function all>, 'any': <built-in function any>, 'ascii': <built-in function ascii>, 'bin': <built-in function bin>, 'breakpoint': <built-in function breakpoint>, 'callable': <built-in function callable>, 'chr': <built-in function chr>, 'compile': <built-in function compile>, 'delattr': <built-in function delattr>, 'dir': <built-in function dir>, 'divmod': <built-in function divmod>, 'eval': <built-in function eval>, 'exec': <built-in function exec>, 'format': <built-in function format>, 'getattr': <built-in function getattr>, 'globals': <built-in function globals>, 'hasattr': <built-in function hasattr>, 'hash': <built-in function hash>, 'hex': <built-in function hex>, 'id': <built-in function id>, 'input': <built-in function input>, 'isinstance': <built-in function isinstance>, 'issubclass': <built-in function issubclass>, 'iter': <built-in function iter>, 'len': <built-in function len>, 'locals': <built-in function locals>, 'max': <built-in function max>, 'min': <built-in function min>, 'next': <built-in function next>, 'oct': <built-in function oct>, 'ord': <built-in function ord>, 'pow': <built-in function pow>, 'print': <built-in function print>, 'repr': <built-in function repr>, 'round': <built-in function round>, 'setattr': <built-in function setattr>, 'sorted': <built-in function sorted>, 'sum': <built-in function sum>, 'vars': <built-in function vars>, 'None': None, 'Ellipsis': Ellipsis, 'NotImplemented': NotImplemented, 'False': False, 'True': True, 'bool': <class 'bool'>, 'memoryview': <class 'memoryview'>, 'bytearray': <class 'bytearray'>, 'bytes': <class 'bytes'>, 'classmethod': <class 'classmethod'>, 'complex': <class 'complex'>, 'dict': <class 'dict'>, 'enumerate': <class 'enumerate'>, 'filter': <class 'filter'>, 'float': <class 'float'>, 'frozenset': <class 'frozenset'>, 'property': <class 'property'>, 'int': <class 'int'>, 'list': <class 'list'>, 'map': <class 'map'>, 'object': <class 'object'>, 'range': <class 'range'>, 'reversed': <class 'reversed'>, 'set': <class 'set'>, 'slice': <class 'slice'>, 'staticmethod': <class 'staticmethod'>, 'str': <class 'str'>, 'super': <class 'super'>, 'tuple': <class 'tuple'>, 'type': <class 'type'>, 'zip': <class 'zip'>, '__debug__': True, 'BaseException': <class 'BaseException'>, 'Exception': <class 'Exception'>, 'TypeError': <class 'TypeError'>, 'StopAsyncIteration': <class 'StopAsyncIteration'>, 'StopIteration': <class 'StopIteration'>, 'GeneratorExit': <class 'GeneratorExit'>, 'SystemExit': <class 'SystemExit'>, 'KeyboardInterrupt': <class 'KeyboardInterrupt'>, 'ImportError': <class 'ImportError'>, 'ModuleNotFoundError': <class 'ModuleNotFoundError'>, 'OSError': <class 'OSError'>, 'EnvironmentError': <class 'OSError'>, 'IOError': <class 'OSError'>, 'WindowsError': <class 'OSError'>, 'EOFError': <class 'EOFError'>, 'RuntimeError': <class 'RuntimeError'>, 'RecursionError': <class 'RecursionError'>, 'NotImplementedError': <class 'NotImplementedError'>, 'NameError': <class 'NameError'>, 'UnboundLocalError': <class 'UnboundLocalError'>, 'AttributeError': <class 'AttributeError'>, 'SyntaxError': <class 'SyntaxError'>, 'IndentationError': <class 'IndentationError'>, 'TabError': <class 'TabError'>, 'LookupError': <class 'LookupError'>, 'IndexError': <class 'IndexError'>, 'KeyError': <class 'KeyError'>, 'ValueError': <class 'ValueError'>, 'UnicodeError': <class 'UnicodeError'>, 'UnicodeEncodeError': <class 'UnicodeEncodeError'>, 'UnicodeDecodeError': <class 'UnicodeDecodeError'>, 'UnicodeTranslateError': <class 'UnicodeTranslateError'>, 'AssertionError': <class 'AssertionError'>, 'ArithmeticError': <class 'ArithmeticError'>, 'FloatingPointError': <class 'FloatingPointError'>, 'OverflowError': <class 'OverflowError'>, 'ZeroDivisionError': <class 'ZeroDivisionError'>, 'SystemError': <class 'SystemError'>, 'ReferenceError': <class 'ReferenceError'>, 'MemoryError': <class 'MemoryError'>, 'BufferError': <class 'BufferError'>, 'Warning': <class 'Warning'>, 'UserWarning': <class 'UserWarning'>, 'DeprecationWarning': <class 'DeprecationWarning'>, 'PendingDeprecationWarning': <class 'PendingDeprecationWarning'>, 'SyntaxWarning': <class 'SyntaxWarning'>, 'RuntimeWarning': <class 'RuntimeWarning'>, 'FutureWarning': <class 'FutureWarning'>, 'ImportWarning': <class 'ImportWarning'>, 'UnicodeWarning': <class 'UnicodeWarning'>, 'BytesWarning': <class 'BytesWarning'>, 'ResourceWarning': <class 'ResourceWarning'>, 'ConnectionError': <class 'ConnectionError'>, 'BlockingIOError': <class 'BlockingIOError'>, 'BrokenPipeError': <class 'BrokenPipeError'>, 'ChildProcessError': <class 'ChildProcessError'>, 'ConnectionAbortedError': <class 'ConnectionAbortedError'>, 'ConnectionRefusedError': <class 'ConnectionRefusedError'>, 'ConnectionResetError': <class 'ConnectionResetError'>, 'FileExistsError': <class 'FileExistsError'>, 'FileNotFoundError': <class 'FileNotFoundError'>, 'IsADirectoryError': <class 'IsADirectoryError'>, 'NotADirectoryError': <class 'NotADirectoryError'>, 'InterruptedError': <class 'InterruptedError'>, 'PermissionError': <class 'PermissionError'>, 'ProcessLookupError': <class 'ProcessLookupError'>, 'TimeoutError': <class 'TimeoutError'>, 'open': <built-in function open>, 'quit': Use quit() or Ctrl-Z plus Return to exit, 'exit': Use exit() or Ctrl-Z plus Return to exit, 'copyright': Copyright (c) 2001-2022 Python Software Foundation.
All Rights Reserved.

The only .py file in my module is a 'test.py' that contains this code:

def test():
    print("hello world")

My goal is to run the test.py from the module I have imported in my environment, then save the output to clojure variable.

My python function takes no arguments, and it seems the module is imported. I'm having trouble understanding how to call the function test() from clojure, or 'scan all the attributes' as you mentioned? Thanks for the help

[cnuernber]

(base) chrisn@chrisn-lp2:~/dev/clj-python/libpython-clj$ tree mypythonlib/
locales-launch: Data of en_US locale not found, generating, please wait...
mypythonlib/
├── __pycache__
│   └── test.cpython-39.pyc
└── test.py

user> (require '[libpython-clj2.python :as py])
nil
user> (py/initialize!)
Feb 27, 2023 4:29:03 PM clojure.tools.logging$eval6123$fn__6126 invoke
INFO: Detecting startup info
Feb 27, 2023 4:29:03 PM clojure.tools.logging$eval6123$fn__6126 invoke
INFO: Startup info {:lib-version "3.9", :java-library-path-addendum "/home/chrisn/miniconda3/lib", :exec-prefix "/home/chrisn/miniconda3", :executable "/home/chrisn/miniconda3/bin/python3", :libnames ("python3.9m" "python3.9"), :prefix "/home/chrisn/miniconda3", :base-prefix "/home/chrisn/miniconda3", :libname "python3.9m", :base-exec-prefix "/home/chrisn/miniconda3", :python-home "/home/chrisn/miniconda3", :version [3 9 1], :platform "linux"}
Feb 27, 2023 4:29:03 PM clojure.tools.logging$eval6123$fn__6126 invoke
INFO: Prefixing java library path: /home/chrisn/miniconda3/lib
Feb 27, 2023 4:29:03 PM clojure.tools.logging$eval6123$fn__6126 invoke
INFO: Loading python library: /home/chrisn/miniconda3/lib/libpython3.9.so
Feb 27, 2023 4:29:04 PM clojure.tools.logging$eval6123$fn__6126 invoke
INFO: Reference thread starting
:ok
user> (def m (py/import-module "mypythonlib"))
#'user/m
user> (py/dir m)
["__doc__" "__file__" "__loader__" "__name__" "__package__" "__path__" "__spec__"]
user> (def mm (py/import-module "mypythonlib.test"))
#'user/mm
user> (py/dir mm)
["__builtins__"
 "__cached__"
 "__doc__"
 "__file__"
 "__loader__"
 "__name__"
 "__package__"
 "__spec__"
 "test"]
user> (py/call-attr mm "test")
hello world
nil
user> 

I am not sure the question is but it may also depend on if this is a namespace module or old-style module. The example I wrote is a namespace module.

[israellandes]

Awesome!!! I followed this example and was able to import my python module and get my 'hello world' to print!

Thank you a ton, this example helps explain away all my questions. I'll probably be back but I hope these issues help other people solve hurdles I'm facing when leaning this library. Your time is much appreciated @cnuernber !