IsakTheHacker
commented
1 year ago Related: #995
Open bitxeno opened 1 year ago
IsakTheHacker
commented
1 year ago Related: #995
Saeppo
commented
1 year ago This worked for me.
python {
version 3.8
pip {
install "pydantic<2"
}
}
mhsmith
commented
1 year ago Originally posted by @caarmen in https://github.com/chaquo/chaquopy/issues/995#issuecomment-1817681622
Would like to be able to use pydantic 2. Some libs seem to require it (ex: ariadne-codegen: https://github.com/mirumee/ariadne-codegen/pull/186 )
mhsmith
commented
1 year ago Originally posted by @IsakTheHacker in #995
I need the pydantic package for my app which depends on the native pydantic-core package written in Rust. I have tried following the guide for building Cryptography version 39 and read #657. It is hard to understand exactly what I need to do to set up Rust support when there is no official documentation. For example, do I need to cross-compile Python as described here or is that specific to the Cryptogaphy package?
mhsmith
commented
1 year ago @IsakTheHacker: I'll try to centralize all the notes about Rust in a single place: #1030.
svdgoor
commented
6 months ago Is there a method to manually build and insert the module? I don't need the latest version each time, but since openai depends on it I cannot build my app.
I need to build for Android and iOS. Neither are currently functioning.
mhsmith
commented
6 months ago Since openai is still compatible with pydantic version 1, there is an easy workaround for that:
pip {
install("pydantic<2")
install("openai")
}Originally posted by @maciekglowka in https://github.com/chaquo/chaquopy/issues/1261#issuecomment-2485352472
Would this approach [from #1261] work for crates that don't support ABI3 ? I am currently trying to build pydantic core, but due to a jiter dependency I can't use this feature on PyO3.
[otherwise I keep getting linking errors with libgcc as previously]
Originally posted by @emanuele-f in https://github.com/chaquo/chaquopy/issues/1261#issuecomment-2485705552
Hello, no, for non-ABI3 modules currently you will still need to cross compile the Python interpreter as explained in https://github.com/chaquo/chaquopy/issues/657
Originally posted by @maciekglowka in https://github.com/chaquo/chaquopy/issues/1261#issuecomment-2485712741
Shoot....thanks for the quick answer though. I was worried that'd be the case (after spending half a day again trying different configurations).
I managed to even work around the jiter thing (it has cfg flags). But then pydantic itself import loads of non-abi3 types, so...
mhsmith
commented
1 week ago @maciekglowka: Are the libgcc errors the same as in #1142? If so, that probably indicates that the build is using the Linux compiler rather than the Android one. The current NDK has replaced GCC with Clang, so I don't think libgcc even exists for Android anymore.
maciekglowka
commented
1 week ago @mhsmith that's the issue. No libgcc to link to (as it's not in the android toolchain anymore).
I've tried to use rust's nightly feature to build the std lib from source (https://doc.rust-lang.org/cargo/reference/unstable.html#build-std) but no success yet. The other approach would be to compile libgcc for the target. I think numba package does it, but I do not fully understand it's shell script yet :)
mhsmith
commented
1 week ago I've tried to use rust's nightly feature to build the std lib from source (https://doc.rust-lang.org/cargo/reference/unstable.html#build-std) but no success yet.
You mean the Rust std lib? Why would you need to rebuild that? If Rust has Android support, doesn't it provide a pre-compiled copy?
The other approach would be to compile libgcc for the target. I think numba package does it, but I do not fully understand it's shell script yet :)
No, that's the wrong approach. Libgcc is very closely coupled to the exact compiler version, and it can't be used with a different one. Clang has its own equivalent called libcompiler_rt.
But all this is a compiler implementation detail which you shouldn't normally be aware of. These are static libraries, included implicitly by the linker, so we don't even need to worry about distributing them.
Numba is an exception because it does low-level JIT compilation, so its recipe will need to be significantly changed to work with the current NDK.
Like I said, the fact that you're seeing a reference to libgcc anywhere in the build probably indicates that a Linux compiler has been used at some point. I suggest you look through the build log to try and find out how this happened. It's often caused by build scripts which hard-code "cc" or "gcc" rather than using the CC environment variable.
maciekglowka
commented
1 week ago You mean the Rust std lib? Why would you need to rebuild that? If Rust has Android support, doesn't it provide a pre-compiled copy?
Thought that was something worth a shot (to avoid linking to the standard std-lib):
https://github.com/rust-mobile/ndk/issues/149#issuecomment-963971717
Ok, thanks - I will look for the wrong toolchain used by PyO3
emanuele-f
commented
1 week ago You should provide the env variables for the cross compilation, either by editing the build-wheel script or by creating a patch under the pydantic package.
The following worked before, but may need adaptations now that the abi3 modules compilation support was merged:
os.environ["RUSTFLAGS"] = f"-C linker={os.environ['CC']}"
os.environ["CARGO_BUILD_TARGET"] = os.environ['CHAQUOPY_TRIPLET']
# https://pyo3.rs/v0.15.2/building_and_distribution.html
os.environ["PYO3_PYTHON"] = f"python{os.environ['CHAQUOPY_PYTHON']}"
os.environ["PYO3_CROSS_PYTHON_VERSION"] = os.environ['CHAQUOPY_PYTHON']
os.environ["PYO3_CROSS_LIB_DIR"] = f"{os.environ['RECIPE_DIR']}/../../../../build/{os.environ['CHAQUOPY_ABI']}/sysroot/usr/lib"
#print(os.environ)Thanks for the help! However I think I'd give up for now, that doesn't feel feasible for me. I've tested so many configurations that I am confused what gave which output :) [anyway the furthest I could ever get was the dreaded sys config data or whatever it's properly called)
I follow this link https://github.com/chaquo/chaquopy/issues/657 to build pydantic v2.4.2，but download package step report error:
how to fix this?