exaloop / codon

A high-performance, zero-overhead, extensible Python compiler using LLVM
https://docs.exaloop.io/codon
Other
15.18k stars 522 forks source link

[FFI] inconsistent between compile with dlopen/without dlopen #162

Closed iuridiniz closed 1 year ago

iuridiniz commented 1 year ago

Hi,

I was doing some shenanigans with Codon and FFI and I got an inconsistent. Here I'm publish a minimal code to reproduce.

If I use FFI with dlopen, this code compiles perfectly (using libsqlite3.sobecause I think everyone has it, but it is not restrict to it)

# without dlopen

LIBRARY="/usr/lib/x86_64-linux-gnu/libsqlite3.so"
from C import LIBRARY.sqlite3_open(cobj, cobj) -> int
from C import LIBRARY.sqlite3_close(cobj) -> int

db = cobj()
if sqlite3_open(":memory:".c_str(), __ptr__(db)) != 0:
    print("Failed to open database")
    exit(1)

sqlite3_close(db)

Shell output:

iuri@GTR:~/Sources/python-projects/codon-tests$ codon --version
0.15.3
iuri@GTR:~/Sources/python-projects/codon-tests$ codon build -l sqlite3 test_ffi_with_dlopen.codon 
iuri@GTR:~/Sources/python-projects/codon-tests$ echo $?
0
iuri@GTR:~/Sources/python-projects/codon-tests$ codon build test_ffi_with_dlopen.codon 
iuri@GTR:~/Sources/python-projects/codon-tests$ echo $?
0

But If I try to compile a variant of it without dlopen, I got compilation error

# without dlopen

from C import sqlite3_open(cobj, cobj) -> int
from C import sqlite3_close(cobj) -> int

db = cobj()
if sqlite3_open(":memory:".c_str(), __ptr__(db)) != 0:
    print("Failed to open database")
    exit(1)

sqlite3_close(db)

Shell output:

iuri@GTR:~/Sources/python-projects/codon-tests$ codon --version
0.15.3
iuri@GTR:~/Sources/python-projects/codon-tests$ codon build -l sqlite3 test_ffi_without_dlopen.codon 
test_ffi_without_dlopen.codon:7:4-49: error: 'Ptr[Ptr[byte]]' does not match expected type 'Ptr[byte]'
iuri@GTR:~/Sources/python-projects/codon-tests$ echo $?
1

To workaround it I need to replace the function signature:

# without dlopen, using Ptr[Ptr[byte]] instead of cobj

from C import sqlite3_open(cobj, Ptr[Ptr[byte]]) -> int
from C import sqlite3_close(cobj) -> int

db = cobj()
if sqlite3_open(":memory:".c_str(), __ptr__(db)) != 0:
    print("Failed to open database")
    exit(1)

sqlite3_close(db)

Shell output:

iuri@GTR:~/Sources/python-projects/codon-tests$ codon --version
0.15.3
iuri@GTR:~/Sources/python-projects/codon-tests$ codon build -l sqlite3 test_ffi_without_dlopen_workaroud.codon 
iuri@GTR:~/Sources/python-projects/codon-tests$ echo $?
0

My PC:

iuri@GTR:~/Sources/python-projects/codon-tests$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 22.04.1 LTS
Release:        22.04
Codename:       jammy

iuri@GTR:~/Sources/python-projects/codon-tests$ uname -a
Linux GTR 5.15.0-56-generic #62-Ubuntu SMP Tue Nov 22 19:54:14 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux

iuri@GTR:~/Sources/python-projects/codon-tests$ gcc --version
gcc (Ubuntu 11.3.0-1ubuntu1~22.04) 11.3.0
Copyright (C) 2021 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

iuri@GTR:~/Sources/python-projects/codon-tests$ ld --version
GNU ld (GNU Binutils for Ubuntu) 2.38
Copyright (C) 2022 Free Software Foundation, Inc.
Este programa é um software livre; você pode redistribuí-lo sob os termos
da Licença Pública Geral GNU versão 3 ou (a seu critério) uma versão posterior.
Esse programa possui absolutamente nenhuma garantia.

Best regards

arshajii commented 1 year ago

@inumanag it seems like the type checker does not enforce types on dynamically linked functions. The same issue is present here: https://github.com/exaloop/codon/issues/135 -- could you take a look at this one?