exaloop / codon

A high-performance, zero-overhead, extensible Python compiler using LLVM
https://docs.exaloop.io/codon
Other
14.93k stars 510 forks source link

how to know which modules are available #142

Open setop opened 1 year ago

setop commented 1 year ago
$ codon run <(echo 'import re)
# ok
$ codon run <(echo 'import logging')
63:1:8-15: error: no module named 'logging'

Is there a way know which modules are available with codon ?

Corollary question : how to improve this list ? Typically, logging would be really useful :)

setop commented 1 year ago

I'd like to be able to run python script as-is. Like elaborate with python interpreter then run in prod with codon.

python codon
form 1 import json from python import json
form 2 from json import loads ?

I'm investigating a way to change import on the fly so that if the python form does not works, it tried the codon form.

I tried

from python import builtins
original_import = builtins.__import__

def _import(*args, **kwargs):
    print("hack_my_import", args, kwargs)
    return original_import(*args, *kwargs)

builtins.__import__ = _import

But got error: 'pyobj' object has no attribute '__import__'.

inumanag commented 1 year ago

Hi @setop

For now, you can explore stdlib directory to see what modules we currently support.

Note that Codon is compiled language; Python is interpreted. Same goes for our Python bridge: it is invoked during the runtime. This means that you cannot use Python bridge to dynamically load Codon features (vice-versa works, though).

cvmiller commented 1 year ago

I too am running into modules missing. Looking in the stdlib, I see that they are in fact, missing.

Is there a way to compile python libraries into stdlib, so as to get wider programaic support? For example I am seeing these errors in trying to compile my open source project which is a simple IPv6 webserver in python, from:

https://github.com/cvmiller/ipv6-httpd

$ . .bashrc
cvmiller@codon:~$ codon build ipv6-httpd.py 
ipv6-httpd.py:24:8-14: error: no module named 'socket'
ipv6-httpd.py:25:6-17: error: no module named 'http.server'
ipv6-httpd.py:25:6-17: error: no module named 'http.server'
ipv6-httpd.py:27:8-14: error: no module named 'signal'
ipv6-httpd.py:35:5-16: error: cannot import name '_exit' from 'os.__init__'
ipv6-httpd.py:38:1-45: error: no module named 'signal'
ipv6-httpd.py:41:17-41: error: name 'SimpleHTTPRequestHandler' is not defined
ipv6-httpd.py:55:20-30: error: name 'HTTPServer' is not defined
ipv6-httpd.py:59:5-18: error: name 'server' is not defined
ipv6-httpd.py:63:5-16: error: cannot import name '_exit' from 'os.__init__'

thanks

elisbyberi commented 1 year ago

@cvmiller Here is an explanation of why compiling CPython with Codon is unproductive: https://github.com/exaloop/codon/issues/314#issuecomment-1493088061

By using @python decorator, you can execute CPython code from Codon:

# https://github.com/cvmiller/ipv6-httpd/blob/master/LICENSE
@python
def run():
    # port webserver listens to
    listen_port = 8080

    import socket
    from http.server import HTTPServer, SimpleHTTPRequestHandler

    import signal
    import os

    # signal handlder for SIGINT
    def sigint_handler(signal, frame):
        shutdown_requested = True
        print("\nCaught SIGINT, dying")

        exit(0)

    # register SIGINT signal handler
    signal.signal(signal.SIGINT, sigint_handler)

    class MyHandler(SimpleHTTPRequestHandler):
        def do_GET(self):
            # if path is /ip then print client IP address (v4 or v6)
            if self.path == '/ip':
                self.send_response(200)
                self.send_header('Content-type', 'text/html')
                self.end_headers()
                answer = 'Your IP address is ' + self.client_address[0]
                # convert string 'answer' to bytes for buffer output
                self.wfile.write(str.encode(answer))
                return
            else:
                return SimpleHTTPRequestHandler.do_GET(self)

    class HTTPServerV6(HTTPServer):
        address_family = socket.AF_INET6

    def main():
        global server
        server = HTTPServerV6(('::', listen_port), MyHandler)
        print('Listening on port:' + str(listen_port) + '\nPress ^C to quit')
        server.serve_forever()

        exit(0)

    main()

if __name__ == '__main__':
    run()
cvmiller commented 1 year ago

Thanks. I understand that it is possible to call interpreted Python from Codon, which is a great feature. I was just wondering if there were plans to add sockets (and networking) to the Codon stdlib.

jsdjfksnv commented 4 months ago

Hi @setop

For now, you can explore stdlib directory to see what modules we currently support.

Note that Codon is compiled language; Python is interpreted. Same goes for our Python bridge: it is invoked during the runtime. This means that you cannot use Python bridge to dynamically load Codon features (vice-versa works, though).

Hello, in stdlib. in exactly which file can I find which modules are supported?