Build Errors on MacBook M1 with Metal Feature - Githubissues

EricLBuehler / mistral.rs

Blazingly fast LLM inference.

MIT License

4.47k stars 311 forks source link

Build Errors on MacBook M1 with Metal Feature #35

Closed biels closed 7 months ago

biels commented 7 months ago

Command used:

cargo build --release --features metal

Environment

Hardware: MacBook M1 OS: macOS Rust Toolchain: nightly-aarch64-apple-darwin

Errors

During the build process with the Metal feature enabled, several type mismatch errors and a trait bound error were encountered, detailed as follows:

Mismatched Types: Expected u32, found Tensor in various parts of the codebase, indicating a discrepancy between the expected integer type and the tensor data structure being used instead. This issue was observed in the context of token IDs and log probability tokens.

Incorrect Argument Types: In several instances, function calls expected a &Tensor, but a &Vec was provided, highlighting a conflict between expected tensor references and vector references.

Trait Bound Not Satisfied: The Serialize trait is not implemented for the TopLogprob structure, which is necessary for serialization operations required by the code.

Error log (some of the errors):

error[E0308]: mismatched types
   --> mistralrs-core/src/engine/mod.rs:137:17
    |
136 |             let is_done = deref_refcell!(seq).is_done(
    |                                               ------- arguments to this method are incorrect
137 |                 next_token_id,
    |                 ^^^^^^^^^^^^^ expected `u32`, found `Tensor`
    |
note: method defined here
   --> mistralrs-core/src/sequence.rs:159:12
    |
159 |     pub fn is_done(&self, tok: u32, eos_tok: u32, max_model_len: usize) -> Option<StopReason> {
    |            ^^^^^^^        --------

error[E0308]: mismatched types
   --> mistralrs-core/src/engine/mod.rs:156:40
    |
156 |                     tokenizer.decode(&[logprob.token], false),
    |                                        ^^^^^^^^^^^^^ expected `u32`, found `Tensor`

error[E0308]: mismatched types
   --> mistralrs-core/src/pipeline/gemma.rs:399:35
    |
399 |             .sample(&logits, Some(&ctxt))?)
    |                              ---- ^^^^^ expected `&Tensor`, found `&Vec<u32>`
    |                              |
    |                              arguments to this enum variant are incorrect
    |
    = note: expected reference `&candle_core::Tensor`
               found reference `&Vec<u32>`

Steps to Reproduce

The environment is set up on a MacBook M1 with the necessary Rust toolchain and cargo versions. Execute the build command with the Metal feature enabled as provided above. Observe the compilation errors listed.

EricLBuehler commented 7 months ago

Hi @biels, thanks for raising the issue. This stemmed from an incorrect dependency, however it should be fixed now. Could you please try it again with a cargo update and git pull?

biels commented 7 months ago

Hi @EricLBuehler, I still see latest commit as [this one] (https://github.com/EricLBuehler/mistral.rs/commit/27cff959657343df64b220150272004bb8b15a9d), maybe forgot to push?

EricLBuehler commented 7 months ago

No, the change is actually in another repo. It is ready to use now.

EricLBuehler commented 7 months ago

@biels, were you able to successfully build?

biels commented 7 months ago

Hey, could you include a link to the repo where it is?

EricLBuehler commented 7 months ago

Sorry for the confusion, the repo is a dependency of this one. It was fixed by updating the dependencies, so no work is required on your part except for cargo update and cargo run ....

biels commented 7 months ago

@EricLBuehler still not compiling:

This is what is coming up now after pull + cargo update and cargo run ....

   Compiling esaxx-rs v0.1.10
error[E0407]: method `alloc_impl` is not a member of trait `BackendDevice`
    --> /Users/biel/.cargo/git/checkouts/candle-c6a149c3b35a488f/2c2b418/candle-core/src/metal_backend/mod.rs:1646:5
     |
1646 | /     fn alloc_impl(
1647 | |         &self,
1648 | |         shape: &Shape,
1649 | |         dtype: DType,
...    |
1665 | |         }
1666 | |     }
     | |_____^ not a member of trait `BackendDevice`

error[E0046]: not all trait items implemented, missing: `alloc_uninit`, `storage_from_cpu_storage_owned`
    --> /Users/biel/.cargo/git/checkouts/candle-c6a149c3b35a488f/2c2b418/candle-core/src/metal_backend/mod.rs:1598:1
     |
1598 | impl BackendDevice for MetalDevice {
     | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ missing `alloc_uninit`, `storage_from_cpu_storage_owned` in implementation
     |
    ::: /Users/biel/.cargo/git/checkouts/candle-c6a149c3b35a488f/2c2b418/candle-core/src/backend.rs:134:5
     |
134  |     unsafe fn alloc_uninit(&self, _shape: &Shape, _dtype: DType) -> Result<Self::Storage>;
     |     -------------------------------------------------------------------------------------- `alloc_uninit` from trait
...
138  |     fn storage_from_cpu_storage_owned(&self, _: CpuStorage) -> Result<Self::Storage>;
     |     --------------------------------------------------------------------------------- `storage_from_cpu_storage_owned` from trait

   Compiling minimal-lexical v0.2.1
   Compiling bit-vec v0.6.3
   Compiling rustls v0.22.3
   Compiling strsim v0.11.0
   Compiling number_prefix v0.4.0
error[E0599]: no method named `alloc_impl` found for reference `&metal_backend::device::MetalDevice` in the current scope
    --> /Users/biel/.cargo/git/checkouts/candle-c6a149c3b35a488f/2c2b418/candle-core/src/metal_backend/mod.rs:1643:14
     |
1643 |         self.alloc_impl(shape, dtype, Some(0))
     |              ^^^^^^^^^^ method not found in `&MetalDevice`

   Compiling heck v0.5.0
   Compiling fastrand v2.0.2
error[E0599]: no method named `alloc_impl` found for struct `CpuDevice` in the current scope
    --> /Users/biel/.cargo/git/checkouts/candle-c6a149c3b35a488f/2c2b418/candle-core/src/metal_backend/mod.rs:1663:61
     |
1663 |             let cpu_storage = crate::cpu_backend::CpuDevice.alloc_impl(shape, dtype, init_value)?;
     |                                                             ^^^^^^^^^^ method not found in `CpuDevice`
     |
    ::: /Users/biel/.cargo/git/checkouts/candle-c6a149c3b35a488f/2c2b418/candle-core/src/cpu_backend/mod.rs:30:1
     |
30   | pub struct CpuDevice;
     | -------------------- method `alloc_impl` not found for this struct

error[E0599]: no method named `alloc_impl` found for reference `&metal_backend::device::MetalDevice` in the current scope
    --> /Users/biel/.cargo/git/checkouts/candle-c6a149c3b35a488f/2c2b418/candle-core/src/metal_backend/mod.rs:1669:14
     |
1669 |         self.alloc_impl(shape, dtype, Some(1))
     |              ^^^^^^^^^^ method not found in `&MetalDevice`

EricLBuehler commented 7 months ago

@biels, the metal backend should be fixed now. Could you try it again?

biels commented 7 months ago

@EricLBuehler this is what's happening now (In the end I log the versions I have for reference):

warning: `mistralrs-core` (lib) generated 2 warnings
   Compiling mistralrs-pyo3 v0.1.0 (/Users/biel/sandbox/rust/mistral.rs/mistralrs-pyo3)
error: linking with `cc` failed: exit status: 1
  |
  = note: env -u IPHONEOS_DEPLOYMENT_TARGET -u TVOS_DEPLOYMENT_TARGET LC_ALL="C" PATH="/Users/biel/.rustup/toolchains/nightly-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/bin:/Users/biel/.cargo/bin:/Users/biel/.bun/bin:/usr/local/opt/openssl@3/bin:/usr/local/opt/mysql@5.7/bin:/usr/local/opt/curl/bin:/usr/local/sbin:/Users/biel/.nodenv/shims:/Users/biel/.rbenv/shims:/Users/biel/.local/bin:/Users/biel/Library/Python/3.8/bin:/Library/PostgreSQL/15/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/Users/biel/sandbox/sciter-js-sdk-main/bin/macosx:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/local/bin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/bin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/appleinternal/bin:/Library/Apple/usr/bin:/Applications/Wireshark.app/Contents/MacOS:/Users/biel/.cargo/bin:/Users/biel/Library/Application Support/JetBrains/Toolbox/scripts:/Users/biel/.dotfiles/plugins/git:/node_modules/.bin/" VSLANG="1033" ZERO_AR_DATE="1" "cc" "-Wl,-exported_symbols_list,/var/folders/08/b840r_kn2wqcglzgxy4nly7c0000gn/T/rustc2aabtW/list" "-arch" "arm64" "/var/folders/08/b840r_kn2wqcglzgxy4nly7c0000gn/T/rustc2aabtW/symbols.o" "/Users/biel/sandbox/rust/mistral.rs/target/release/deps/mistralrs.mistralrs.5ac1da7054f9a315-cgu.00.rcgu.o" "/Users/biel/sandbox/rust/mistral.rs/target/release/deps/mistralrs.mistralrs.5ac1da7054f9a315-cgu.01.rcgu.o" "/Users/biel/sandbox/rust/mistral.rs/target/release/deps/mistralrs.mistralrs.5ac1da7054f9a315-cgu.02.rcgu.o" "/Users/biel/sandbox/rust/mistral.rs/target/release/deps/mistralrs.mistralrs.5ac1da7054f9a315-cgu.03.rcgu.o" "/Users/biel/sandbox/rust/mistral.rs/target/release/deps/mistralrs.mistralrs.5ac1da7054f9a315-cgu.04.rcgu.o" "/Users/biel/sandbox/rust/mistral.rs/target/release/deps/mistralrs.mistralrs.5ac1da7054f9a315-cgu.05.rcgu.o" "/Users/biel/sandbox/rust/mistral.rs/target/release/deps/mistralrs.mistralrs.5ac1da7054f9a315-cgu.06.rcgu.o" "/Users/biel/sandbox/rust/mistral.rs/target/release/deps/mistralrs.mistralrs.5ac1da7054f9a315-cgu.07.rcgu.o" "/Users/biel/sandbox/rust/mistral.rs/target/release/deps/mistralrs.mistralrs.5ac1da7054f9a315-cgu.08.rcgu.o" "/Users/biel/sandbox/rust/mistral.rs/target/release/deps/mistralrs.mistralrs.5ac1da7054f9a315-cgu.09.rcgu.o" "/Users/biel/sandbox/rust/mistral.rs/target/release/deps/mistralrs.mistralrs.5ac1da7054f9a315-cgu.10.rcgu.o" "/Users/biel/sandbox/rust/mistral.rs/target/release/deps/mistralrs.mistralrs.5ac1da7054f9a315-cgu.11.rcgu.o" "/Users/biel/sandbox/rust/mistral.rs/target/release/deps/mistralrs.mistralrs.5ac1da7054f9a315-cgu.12.rcgu.o" "/Users/biel/sandbox/rust/mistral.rs/target/release/deps/mistralrs.mistralrs.5ac1da7054f9a315-cgu.13.rcgu.o" "/Users/biel/sandbox/rust/mistral.rs/target/release/deps/mistralrs.mistralrs.5ac1da7054f9a315-cgu.14.rcgu.o" "/Users/biel/sandbox/rust/mistral.rs/target/release/deps/mistralrs.mistralrs.5ac1da7054f9a315-cgu.15.rcgu.o" "/Users/biel/sandbox/rust/mistral.rs/target/release/deps/mistralrs.zzpma2ratpjqanc.rcgu.o" "-L" "/Users/biel/sandbox/rust/mistral.rs/target/release/deps" "-L" "/Users/biel/sandbox/rust/mistral.rs/target/release/build/objc_exception-5b3d15de4c6412dd/out" "-L" "/Users/biel/sandbox/rust/mistral.rs/target/release/build/esaxx-rs-0cf973329aa6949a/out" "-L" "/Users/biel/sandbox/rust/mistral.rs/target/release/build/onig_sys-28c35b83bc6266c0/out" "-L" "/Users/biel/sandbox/rust/mistral.rs/target/release/build/ring-69ff6d9a9f99decb/out" "-L" "/Users/biel/.rustup/toolchains/nightly-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib" "/Users/biel/sandbox/rust/mistral.rs/target/release/deps/libpyo3-826a4fb06548ad87.rlib" "/Users/biel/sandbox/rust/mistral.rs/target/release/deps/libmemoffset-d593da914c1dc4bf.rlib" "/Users/biel/sandbox/rust/mistral.rs/target/release/deps/libpyo3_ffi-8b05d50596e6662e.rlib" "/Users/biel/sandbox/rust/mistral.rs/target/release/deps/libunindent-268c8cbb6eb37565.rlib" "/Users/biel/sandbox/rust/mistral.rs/target/release/deps/libmistralrs-1fb67cd69bf270b2.rlib" "/Users/biel/sandbox/rust/mistral.rs/target/release/deps/libmistralrs_core-259224dfba461941.rlib" "/Users/biel/sandbox/rust/mistral.rs/target/release/deps/libchrono-f66ef094e5c9e146.rlib" "/Users/biel/sandbox/rust/mistral.rs/target/release/deps/libiana_time_zone-b42484eda538b788.rlib" "/Users/biel/sandbox/rust/mistral.rs/target/release/deps/libtqdm-679945f9ecfe2e32.rlib" "/Users/biel/sandbox/rust/mistral.rs/target/release/deps/libcrossterm-1172e9ff9f8e3d01.rlib" "/Users/biel/sandbox/rust/mistral.rs/target/release/deps/libsignal_hook_mio-76c2ac6908f3b5c5.rlib" "/Users/biel/sandbox/rust/mistral.rs/target/release/deps/libsignal_hook-6e6a03fc84dd4afc.rlib" "/Users/biel/sandbox/rust/mistral.rs/target/release/deps/libsignal_hook_registry-1465d1a5fe0e9249.rlib" "/Users/biel/sandbox/rust/mistral.rs/target/release/deps/libmio-543d88a64b5210a2.rlib" "/Users/biel/sandbox/rust/mistral.rs/target/release/deps/libparking_lot-9e35089a135b21cd.rlib" "/Users/biel/sandbox/rust/mistral.rs/target/release/deps/libparking_lot_core-0972a66868ddb0af.rlib" "/Users/biel/sandbox/rust/mistral.rs/target/release/deps/liblock_api-cacdb5f197e90721.rlib" "/Users/biel/sandbox/rust/mistral.rs/target/release/deps/libscopeguard-91dc0781da109ffe.rlib" "/Users/biel/sandbox/rust/mistral.rs/target/release/deps/librange_checked-949b9a3cf2151f49.rlib" "/Users/biel/sandbox/rust/mistral.rs/target/release/deps/libminijinja-3c1065eecbed3053.rlib" "/Users/biel/sandbox/rust/mistral.rs/target/release/deps/libhf_hub-432c74415d5b81ce.rlib" "/Users/biel/sandbox/rust/mistral.rs/target/release/deps/libdirs-fda6e8a009e2db88.rlib" "/Users/biel/sandbox/rust/mistral.rs/target/release/deps/libdirs_sys-a47f0a2fec7aed6f.rlib" "/Users/biel/sandbox/rust/mistral.rs/target/release/deps/liboption_ext-c3c7f91528f07548.rlib" "/Users/biel/sandbox/rust/mistral.rs/target/release/deps/libureq-a89f8d41a717e0ad.rlib" "/Users/biel/sandbox/rust/mistral.rs/target/release/deps/libnative_tls-ca98391fbabc96c5.rlib" "/Users/biel/sandbox/rust/mistral.rs/target/release/deps/libtempfile-b2de976fe47ff1c1.rlib" "/Users/biel/sandbox/rust/mistral.rs/target/release/deps/libfastrand-586a292ddec586e5.rlib" "/Users/biel/sandbox/rust/mistral.rs/target/release/deps/librustix-675255cec4b9e07c.rlib" "/Users/biel/sandbox/rust/mistral.rs/target/release/deps/liberrno-5508772baf9866ef.rlib" "/Users/biel/sandbox/rust/mistral.rs/target/release/deps/libsecurity_framework-e5a78f08b8a3e887.rlib" "/Users/biel/sandbox/rust/mistral.rs/target/release/deps/libsecurity_framework_sys-eed0e2ee037db629.rlib" "/Users/biel/sandbox/rust/mistral.rs/target/release/deps/libwebpki_roots-82ad18a43ca9e74e.rlib" "/Users/biel/sandbox/rust/mistral.rs/target/release/deps/librustls-b1e983f3aef4a17c.rlib" "/Users/biel/sandbox/rust/mistral.rs/target/release/deps/libsubtle-c5d28f95036d97d4.rlib" "/Users/biel/sandbox/rust/mistral.rs/target/release/deps/libwebpki-80e3ed459d597482.rlib" "/Users/biel/sandbox/rust/mistral.rs/target/release/deps/libring-8dd8c31c04f248bc.rlib" "/Users/biel/sandbox/rust/mistral.rs/target/release/deps/libspin-bed49a621c290946.rlib" "/Users/biel/sandbox/rust/mistral.rs/target/release/deps/libuntrusted-7bb200392b791665.rlib" "/Users/biel/sandbox/rust/mistral.rs/target/release/deps/libzeroize-8a5881d4ef715f8e.rlib" "/Users/biel/sandbox/rust/mistral.rs/target/release/deps/librustls_pki_types-d0dd10dfac8db95c.rlib" "/Users/biel/sandbox/rust/mistral.rs/target/release/deps/libflate2-c364e86190afbd3a.rlib" "/Users/biel/sandbox/rust/mistral.rs/target/release/deps/libminiz_oxide-d539d231d55e52c0.rlib" "/Users/biel/sandbox/rust/mistral.rs/target/release/deps/libadler-2e6eb746297f4769.rlib" "/Users/biel/sandbox/rust/mistral.rs/target/release/deps/libbase64-48b4ba9f79ec44c6.rlib" "/Users/biel/sandbox/rust/mistral.rs/target/release/deps/liburl-84b5b5de186f261a.rlib" "/Users/biel/sandbox/rust/mistral.rs/target/release/deps/libidna-3f947e6ff360281d.rlib" "/Users/biel/sandbox/rust/mistral.rs/target/release/deps/libunicode_normalization-5d42d9620870d934.rlib" "/Users/biel/sandbox/rust/mistral.rs/target/release/deps/libtinyvec-02a6868cc15d1951.rlib" "/Users/biel/sandbox/rust/mistral.rs/target/release/deps/libtinyvec_macros-3c3f6f842e297927.rlib" "/Users/biel/sandbox/rust/mistral.rs/target/release/deps/libunicode_bidi-cd7ebcbe70afc5b9.rlib" "/Users/biel/sandbox/rust/mistral.rs/target/release/deps/libform_urlencoded-58f298d9ded69e39.rlib" "/Users/biel/sandbox/rust/mistral.rs/target/release/deps/libpercent_encoding-cf271079b5ee72fd.rlib" "/Users/biel/sandbox/rust/mistral.rs/target/release/deps/libanyhow-15015ad532d912fe.rlib" "/Users/biel/sandbox/rust/mistral.rs/target/release/deps/libcandle_transformers-f92d18c7fea3dcc7.rlib" "/Users/biel/sandbox/rust/mistral.rs/target/release/deps/libserde_plain-a81a5f8cd26d3ae0.rlib" "/Users/biel/sandbox/rust/mistral.rs/target/release/deps/libfancy_regex-1e2b7d319abdb280.rlib" "/Users/biel/sandbox/rust/mistral.rs/target/release/deps/libbit_set-675e85f7cc0c9f0e.rlib" "/Users/biel/sandbox/rust/mistral.rs/target/release/deps/libbit_vec-d3dc162e7431bb31.rlib" "/Users/biel/sandbox/rust/mistral.rs/target/release/deps/libtracing-0e68a51320853c75.rlib" "/Users/biel/sandbox/rust/mistral.rs/target/release/deps/libpin_project_lite-64fb84ace3799e87.rlib" "/Users/biel/sandbox/rust/mistral.rs/target/release/deps/libtracing_core-7480267416fe9d83.rlib" "/Users/biel/sandbox/rust/mistral.rs/target/release/deps/libcandle_sampling-0248f418dde61168.rlib" "/Users/biel/sandbox/rust/mistral.rs/target/release/deps/libtokenizers-38a0d0dd2adcf6e0.rlib" "/Users/biel/sandbox/rust/mistral.rs/target/release/deps/libesaxx_rs-b8b978c77d16eaef.rlib" "/Users/biel/sandbox/rust/mistral.rs/target/release/deps/libregex-853b54dc8eb06d41.rlib" "/Users/biel/sandbox/rust/mistral.rs/target/release/deps/libregex_automata-700d6b6916322cfd.rlib" "/Users/biel/sandbox/rust/mistral.rs/target/release/deps/libaho_corasick-9c827f0e458750fb.rlib" "/Users/biel/sandbox/rust/mistral.rs/target/release/deps/libregex_syntax-dfd21c053e540886.rlib" "/Users/biel/sandbox/rust/mistral.rs/target/release/deps/libunicode_normalization_alignments-d8638c95b32e0d22.rlib" "/Users/biel/sandbox/rust/mistral.rs/target/release/deps/libsmallvec-3bffd666f46be7e6.rlib" "/Users/biel/sandbox/rust/mistral.rs/target/release/deps/libspm_precompiled-95204aaab838ef6f.rlib" "/Users/biel/sandbox/rust/mistral.rs/target/release/deps/libbase64-9f2a3895ed0e3f34.rlib" "/Users/biel/sandbox/rust/mistral.rs/target/release/deps/libunicode_segmentation-3cca92bb7c45bcba.rlib" "/Users/biel/sandbox/rust/mistral.rs/target/release/deps/libnom-cfbd0499ab823c97.rlib" "/Users/biel/sandbox/rust/mistral.rs/target/release/deps/libmemchr-0043c5f6f5c75b73.rlib" "/Users/biel/sandbox/rust/mistral.rs/target/release/deps/libunicode_categories-e48ed94708af97e8.rlib" "/Users/biel/sandbox/rust/mistral.rs/target/release/deps/libitertools-a762dd3266b5f942.rlib" "/Users/biel/sandbox/rust/mistral.rs/target/release/deps/libmonostate-aaf49745cfd004b3.rlib" "/Users/biel/sandbox/rust/mistral.rs/target/release/deps/libmacro_rules_attribute-466bbad81c6492ba.rlib" "/Users/biel/sandbox/rust/mistral.rs/target/release/deps/libindicatif-ac1129cc9c50febc.rlib" "/Users/biel/sandbox/rust/mistral.rs/target/release/deps/libportable_atomic-8f3d91cad3f79c2f.rlib" "/Users/biel/sandbox/rust/mistral.rs/target/release/deps/libnumber_prefix-b1f32dad741c592d.rlib" "/Users/biel/sandbox/rust/mistral.rs/target/release/deps/libconsole-7b8898a2c27235b7.rlib" "/Users/biel/sandbox/rust/mistral.rs/target/release/deps/libunicode_width-2c5d8676025a818f.rlib" "/Users/biel/sandbox/rust/mistral.rs/target/release/deps/librayon_cond-aa3cc7b98ec77c46.rlib" "/Users/biel/sandbox/rust/mistral.rs/target/release/deps/libitertools-dd105620bb905c8c.rlib" "/Users/biel/sandbox/rust/mistral.rs/target/release/deps/libonig-54f4e7ff73c489d8.rlib" "/Users/biel/sandbox/rust/mistral.rs/target/release/deps/libbitflags-c5795c861ae0760a.rlib" "/Users/biel/sandbox/rust/mistral.rs/target/release/deps/libonig_sys-587aa365624116c0.rlib" "/Users/biel/sandbox/rust/mistral.rs/target/release/deps/libderive_builder-ff491e2d3c30c316.rlib" "/Users/biel/sandbox/rust/mistral.rs/target/release/deps/liblazy_static-936d4e0e3ce61c35.rlib" "/Users/biel/sandbox/rust/mistral.rs/target/release/deps/libmistralrs_lora-704f271acb91946c.rlib" "/Users/biel/sandbox/rust/mistral.rs/target/release/deps/libcandle_nn-a1216476fa653f5d.rlib" "/Users/biel/sandbox/rust/mistral.rs/target/release/deps/libindexmap-1d092b2a635ffab6.rlib" "/Users/biel/sandbox/rust/mistral.rs/target/release/deps/libequivalent-51f76f73e8c7c5fe.rlib" "/Users/biel/sandbox/rust/mistral.rs/target/release/deps/libhashbrown-1bc1b53cd3311b0b.rlib" "/Users/biel/sandbox/rust/mistral.rs/target/release/deps/libcandle_core-d0efc12c50bd9834.rlib" "/Users/biel/sandbox/rust/mistral.rs/target/release/deps/libnum_cpus-60dbd145680f086b.rlib" "/Users/biel/sandbox/rust/mistral.rs/target/release/deps/libmemmap2-463f3d8f0624efb2.rlib" "/Users/biel/sandbox/rust/mistral.rs/target/release/deps/libzip-28d93ad2cb26dbf8.rlib" "/Users/biel/sandbox/rust/mistral.rs/target/release/deps/libcrc32fast-8785c27a11676433.rlib" "/Users/biel/sandbox/rust/mistral.rs/target/release/deps/libyoke-c3ca67a150b36cd9.rlib" "/Users/biel/sandbox/rust/mistral.rs/target/release/deps/libzerofrom-c7a48bf3dee12ff4.rlib" "/Users/biel/sandbox/rust/mistral.rs/target/release/deps/libstable_deref_trait-f756c0405b7b5a9a.rlib" "/Users/biel/sandbox/rust/mistral.rs/target/release/deps/libsafetensors-0e6058cd622175ea.rlib" "/Users/biel/sandbox/rust/mistral.rs/target/release/deps/libserde_json-1f57b557d2f05db3.rlib" "/Users/biel/sandbox/rust/mistral.rs/target/release/deps/libitoa-938b23dfd7edab47.rlib" "/Users/biel/sandbox/rust/mistral.rs/target/release/deps/libryu-c1096adc1d191a03.rlib" "/Users/biel/sandbox/rust/mistral.rs/target/release/deps/libcandle_metal_kernels-4cb12f08ac177af1.rlib" "/Users/biel/sandbox/rust/mistral.rs/target/release/deps/libmetal-ed1bb646c369bc28.rlib" "/Users/biel/sandbox/rust/mistral.rs/target/release/deps/libblock-4f56f50e7c5cd7d3.rlib" "/Users/biel/sandbox/rust/mistral.rs/target/release/deps/libcore_graphics_types-f42896ae9b8be4d1.rlib" "/Users/biel/sandbox/rust/mistral.rs/target/release/deps/libcore_foundation-a762e3eea86d83ac.rlib" "/Users/biel/sandbox/rust/mistral.rs/target/release/deps/libcore_foundation_sys-ec9f96a6a0600a7c.rlib" "/Users/biel/sandbox/rust/mistral.rs/target/release/deps/libforeign_types-d7a00dca4501f556.rlib" "/Users/biel/sandbox/rust/mistral.rs/target/release/deps/libforeign_types_shared-8989736b46ca3c94.rlib" "/Users/biel/sandbox/rust/mistral.rs/target/release/deps/libobjc-818f48eb0327b14b.rlib" "/Users/biel/sandbox/rust/mistral.rs/target/release/deps/libmalloc_buf-e99feb60557b373b.rlib" "/Users/biel/sandbox/rust/mistral.rs/target/release/deps/liblog-14d6a9a64ae5c066.rlib" "/Users/biel/sandbox/rust/mistral.rs/target/release/deps/libgemm-75c36d9f11dc1a7b.rlib" "/Users/biel/sandbox/rust/mistral.rs/target/release/deps/libgemm_c32-cc4c69fe42d27e91.rlib" "/Users/biel/sandbox/rust/mistral.rs/target/release/deps/libgemm_c64-7aa3f14d92742c66.rlib" "/Users/biel/sandbox/rust/mistral.rs/target/release/deps/libgemm_f64-68bcda7244d6e886.rlib" "/Users/biel/sandbox/rust/mistral.rs/target/release/deps/libgemm_f16-e4c6ae6b93586446.rlib" "/Users/biel/sandbox/rust/mistral.rs/target/release/deps/libgemm_f32-be2eca26f0ebace7.rlib" "/Users/biel/sandbox/rust/mistral.rs/target/release/deps/libgemm_common-f6b61ac162851389.rlib" "/Users/biel/sandbox/rust/mistral.rs/target/release/deps/libonce_cell-518a437aa7d3fb76.rlib" "/Users/biel/sandbox/rust/mistral.rs/target/release/deps/libpulp-8345fc5488ce3537.rlib" "/Users/biel/sandbox/rust/mistral.rs/target/release/deps/libnum_complex-02c1e3fa81e539a9.rlib" "/Users/biel/sandbox/rust/mistral.rs/target/release/deps/libdyn_stack-37ac862d87809816.rlib" "/Users/biel/sandbox/rust/mistral.rs/target/release/deps/libreborrow-390c7dd0d8c5eb20.rlib" "/Users/biel/sandbox/rust/mistral.rs/target/release/deps/libsysctl-7a960126d00aa88b.rlib" "/Users/biel/sandbox/rust/mistral.rs/target/release/deps/libthiserror-ea6970cb8874e68d.rlib" "/Users/biel/sandbox/rust/mistral.rs/target/release/deps/libbitflags-56958ad175f68b6d.rlib" "/Users/biel/sandbox/rust/mistral.rs/target/release/deps/librayon-0ea67e9388255925.rlib" "/Users/biel/sandbox/rust/mistral.rs/target/release/deps/librayon_core-03de8bb0b1452eea.rlib" "/Users/biel/sandbox/rust/mistral.rs/target/release/deps/libcrossbeam_deque-8baf6fd05f0e0882.rlib" "/Users/biel/sandbox/rust/mistral.rs/target/release/deps/libcrossbeam_epoch-535c2a45860ca40d.rlib" "/Users/biel/sandbox/rust/mistral.rs/target/release/deps/libcrossbeam_utils-c837359d3398b750.rlib" "/Users/biel/sandbox/rust/mistral.rs/target/release/deps/libeither-a4b9903931ad5e50.rlib" "/Users/biel/sandbox/rust/mistral.rs/target/release/deps/libserde-af709a1a8c2cc646.rlib" "/Users/biel/sandbox/rust/mistral.rs/target/release/deps/libbyteorder-10bba7f2cbe3001e.rlib" "/Users/biel/sandbox/rust/mistral.rs/target/release/deps/libhalf-9434fdff9cb404c0.rlib" "/Users/biel/sandbox/rust/mistral.rs/target/release/deps/librand_distr-68475a41249a0f61.rlib" "/Users/biel/sandbox/rust/mistral.rs/target/release/deps/librand-01a1a11006be08b9.rlib" "/Users/biel/sandbox/rust/mistral.rs/target/release/deps/librand_chacha-017200293211e85d.rlib" "/Users/biel/sandbox/rust/mistral.rs/target/release/deps/libppv_lite86-ab9c3930759091a9.rlib" "/Users/biel/sandbox/rust/mistral.rs/target/release/deps/librand_core-b7f2d1439be73569.rlib" "/Users/biel/sandbox/rust/mistral.rs/target/release/deps/libgetrandom-d91e688cc6c18854.rlib" "/Users/biel/sandbox/rust/mistral.rs/target/release/deps/liblibc-37ad1e326b9cbf00.rlib" "/Users/biel/sandbox/rust/mistral.rs/target/release/deps/libcfg_if-237ca97c10329bbb.rlib" "/Users/biel/sandbox/rust/mistral.rs/target/release/deps/libnum_traits-ac3c272d6566d6bb.rlib" "/Users/biel/sandbox/rust/mistral.rs/target/release/deps/libbytemuck-ee4042e4a5bbadfa.rlib" "/Users/biel/.rustup/toolchains/nightly-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib/libstd-245dba7eb3f71de0.rlib" "/Users/biel/.rustup/toolchains/nightly-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib/libpanic_unwind-0822f2bc685668d4.rlib" "/Users/biel/.rustup/toolchains/nightly-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib/libobject-c3c52b4cd12b325e.rlib" "/Users/biel/.rustup/toolchains/nightly-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib/libmemchr-bbc63654593d603d.rlib" "/Users/biel/.rustup/toolchains/nightly-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib/libaddr2line-146b40bfd24f73ec.rlib" "/Users/biel/.rustup/toolchains/nightly-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib/libgimli-e54256b8e7060959.rlib" "/Users/biel/.rustup/toolchains/nightly-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib/librustc_demangle-829b5d694ad5530d.rlib" "/Users/biel/.rustup/toolchains/nightly-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib/libstd_detect-dab8ae8c738c5735.rlib" "/Users/biel/.rustup/toolchains/nightly-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib/libhashbrown-c5bcbc0b523c4f2e.rlib" "/Users/biel/.rustup/toolchains/nightly-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib/librustc_std_workspace_alloc-d1f6bccd33b13ea1.rlib" "/Users/biel/.rustup/toolchains/nightly-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib/libminiz_oxide-f8f8344f4c41febf.rlib" "/Users/biel/.rustup/toolchains/nightly-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib/libadler-8edc7cc732dcf852.rlib" "/Users/biel/.rustup/toolchains/nightly-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib/libunwind-e0758bcf1624ad58.rlib" "/Users/biel/.rustup/toolchains/nightly-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib/libcfg_if-6d7cccd8fa746af7.rlib" "/Users/biel/.rustup/toolchains/nightly-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib/liblibc-e49f559b9aa8fc23.rlib" "/Users/biel/.rustup/toolchains/nightly-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib/liballoc-501df7350853671f.rlib" "/Users/biel/.rustup/toolchains/nightly-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib/librustc_std_workspace_core-a4ecc63307a2b2e4.rlib" "/Users/biel/.rustup/toolchains/nightly-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib/libcore-4b44a1965098f9de.rlib" "/Users/biel/.rustup/toolchains/nightly-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib/libcompiler_builtins-6edf6b06661533da.rlib" "-framework" "Security" "-lc++" "-framework" "Metal" "-lSystem" "-framework" "MetalPerformanceShaders" "-lSystem" "-framework" "CoreGraphics" "-framework" "CoreFoundation" "-lobjc" "-liconv" "-lSystem" "-lc" "-lm" "-L" "/Users/biel/.rustup/toolchains/nightly-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib" "-o" "/Users/biel/sandbox/rust/mistral.rs/target/release/deps/libmistralrs.dylib" "-Wl,-dead_strip" "-dynamiclib" "-Wl,-dylib" "-nodefaultlibs"
  = note: ld: warning: ignoring duplicate libraries: '-lSystem'
          Undefined symbols for architecture arm64:
            "_PyBaseObject_Type", referenced from:
                mistralrs::_::_$LT$impl$u20$mistralrs..ModelKind$GT$::__pymethod_Normal__::hb8cd43d4da88386e (.llvm.6094665132293408195) in mistralrs.mistralrs.5ac1da7054f9a315-cgu.00.rcgu.o
                mistralrs::_::_$LT$impl$u20$mistralrs..ModelKind$GT$::__pymethod_XLoraNormal__::h418a1e6cb7b170e2 (.llvm.6094665132293408195) in mistralrs.mistralrs.5ac1da7054f9a315-cgu.00.rcgu.o
                mistralrs::_::_$LT$impl$u20$mistralrs..ModelKind$GT$::__pymethod_XLoraGGUF__::hdb8f79aeca77dff5 (.llvm.6094665132293408195) in mistralrs.mistralrs.5ac1da7054f9a315-cgu.00.rcgu.o
                mistralrs::_::_$LT$impl$u20$mistralrs..ModelKind$GT$::__pymethod_XLoraGGML__::hbdc9deb32848e856 (.llvm.6094665132293408195) in mistralrs.mistralrs.5ac1da7054f9a315-cgu.00.rcgu.o
                mistralrs::_::_$LT$impl$u20$mistralrs..ModelKind$GT$::__pymethod_QuantizedGGUF__::h948a5791be867434 (.llvm.6094665132293408195) in mistralrs.mistralrs.5ac1da7054f9a315-cgu.00.rcgu.o
                mistralrs::_::_$LT$impl$u20$mistralrs..ModelKind$GT$::__pymethod_QuantizedGGML__::h6effa4951b762554 (.llvm.6094665132293408195) in mistralrs.mistralrs.5ac1da7054f9a315-cgu.00.rcgu.o
                mistralrs::_::_$LT$impl$u20$mistralrs..DType$GT$::__pymethod_U8__::h34bbb82fc3c2f139 (.llvm.6094665132293408195) in mistralrs.mistralrs.5ac1da7054f9a315-cgu.00.rcgu.o
                ...
            "_PyBool_Type", referenced from:
                pyo3::types::boolobject::_$LT$impl$u20$pyo3..conversion..FromPyObject$u20$for$u20$bool$GT$::extract::h9a42ba0a1f2118d6 in libpyo3-826a4fb06548ad87.rlib[15](pyo3-826a4fb06548ad87.pyo3.1cceb9e92f39dd66-cgu.12.rcgu.o)
            "_PyBytes_AsString", referenced from:
                pyo3::types::string::PyString::to_string_lossy::h04a943aca21dc817 in libpyo3-826a4fb06548ad87.rlib[8](pyo3-826a4fb06548ad87.pyo3.1cceb9e92f39dd66-cgu.05.rcgu.o)
            "_PyBytes_Size", referenced from:
                pyo3::types::string::PyString::to_string_lossy::h04a943aca21dc817 in libpyo3-826a4fb06548ad87.rlib[8](pyo3-826a4fb06548ad87.pyo3.1cceb9e92f39dd66-cgu.05.rcgu.o)
            "_PyDict_Next", referenced from:
                pyo3::types::dict::PyDictIterator::next_unchecked::h3e63d35efbd4eea8 in libpyo3-826a4fb06548ad87.rlib[17](pyo3-826a4fb06548ad87.pyo3.1cceb9e92f39dd66-cgu.14.rcgu.o)
            "_PyErr_Fetch", referenced from:
                pyo3::err::PyErr::take::h8333268145de9d80 in libpyo3-826a4fb06548ad87.rlib[4](pyo3-826a4fb06548ad87.pyo3.1cceb9e92f39dd66-cgu.01.rcgu.o)
                pyo3::err::err_state::lazy_into_normalized_ffi_tuple::ha4d7907a47e72650 (.llvm.18419523723726193394) in libpyo3-826a4fb06548ad87.rlib[17](pyo3-826a4fb06548ad87.pyo3.1cceb9e92f39dd66-cgu.14.rcgu.o)
            "_PyErr_GivenExceptionMatches", referenced from:
                pyo3::types::module::PyModule::index::hbfa5d2c6e001f805 in libpyo3-826a4fb06548ad87.rlib[12](pyo3-826a4fb06548ad87.pyo3.1cceb9e92f39dd66-cgu.09.rcgu.o)
            "_PyErr_NewExceptionWithDoc", referenced from:
                pyo3::err::PyErr::new_type::h9199e0276f95a157 in libpyo3-826a4fb06548ad87.rlib[4](pyo3-826a4fb06548ad87.pyo3.1cceb9e92f39dd66-cgu.01.rcgu.o)
            "_PyErr_NormalizeException", referenced from:
                pyo3::err::err_state::PyErrState::normalize::hf6a009eaad60d703 in libpyo3-826a4fb06548ad87.rlib[17](pyo3-826a4fb06548ad87.pyo3.1cceb9e92f39dd66-cgu.14.rcgu.o)
                pyo3::err::err_state::lazy_into_normalized_ffi_tuple::ha4d7907a47e72650 (.llvm.18419523723726193394) in libpyo3-826a4fb06548ad87.rlib[17](pyo3-826a4fb06548ad87.pyo3.1cceb9e92f39dd66-cgu.14.rcgu.o)
            "_PyErr_Print", referenced from:
                pyo3::err::panic_after_error::h0174fe97ff0121e6 in libpyo3-826a4fb06548ad87.rlib[4](pyo3-826a4fb06548ad87.pyo3.1cceb9e92f39dd66-cgu.01.rcgu.o)
            "_PyErr_PrintEx", referenced from:
                pyo3::err::PyErr::print_panic_and_unwind::h128e2dc68f948392 in libpyo3-826a4fb06548ad87.rlib[4](pyo3-826a4fb06548ad87.pyo3.1cceb9e92f39dd66-cgu.01.rcgu.o)
                pyo3::err::PyErr::print::h89851c9fe1a94dc3 in libpyo3-826a4fb06548ad87.rlib[4](pyo3-826a4fb06548ad87.pyo3.1cceb9e92f39dd66-cgu.01.rcgu.o)
            "_PyErr_Restore", referenced from:
                _$LT$pyo3..exceptions..PyPendingDeprecationWarning$u20$as$u20$core..fmt..Display$GT$::fmt::h4c527b2e474517c2 in libpyo3-826a4fb06548ad87.rlib[3](pyo3-826a4fb06548ad87.pyo3.1cceb9e92f39dd66-cgu.00.rcgu.o)
                _$LT$pyo3..exceptions..PyPendingDeprecationWarning$u20$as$u20$core..fmt..Display$GT$::fmt::h4c527b2e474517c2 in libpyo3-826a4fb06548ad87.rlib[3](pyo3-826a4fb06548ad87.pyo3.1cceb9e92f39dd66-cgu.00.rcgu.o)
                pyo3::err::PyErr::print_panic_and_unwind::h128e2dc68f948392 in libpyo3-826a4fb06548ad87.rlib[4](pyo3-826a4fb06548ad87.pyo3.1cceb9e92f39dd66-cgu.01.rcgu.o)
                pyo3::err::PyErr::print_panic_and_unwind::h128e2dc68f948392 in libpyo3-826a4fb06548ad87.rlib[4](pyo3-826a4fb06548ad87.pyo3.1cceb9e92f39dd66-cgu.01.rcgu.o)
                pyo3::err::PyErr::print::h89851c9fe1a94dc3 in libpyo3-826a4fb06548ad87.rlib[4](pyo3-826a4fb06548ad87.pyo3.1cceb9e92f39dd66-cgu.01.rcgu.o)
                pyo3::pyclass::create_type_object::no_constructor_defined::h7dcd3f5a3676ea53 in libpyo3-826a4fb06548ad87.rlib[7](pyo3-826a4fb06548ad87.pyo3.1cceb9e92f39dd66-cgu.04.rcgu.o)
                pyo3::pyclass::create_type_object::no_constructor_defined::h7dcd3f5a3676ea53 in libpyo3-826a4fb06548ad87.rlib[7](pyo3-826a4fb06548ad87.pyo3.1cceb9e92f39dd66-cgu.04.rcgu.o)
                pyo3::pyclass::create_type_object::no_constructor_defined::h7dcd3f5a3676ea53 in libpyo3-826a4fb06548ad87.rlib[7](pyo3-826a4fb06548ad87.pyo3.1cceb9e92f39dd66-cgu.04.rcgu.o)
                ...
            "_PyErr_SetObject", referenced from:
                pyo3::err::err_state::lazy_into_normalized_ffi_tuple::ha4d7907a47e72650 (.llvm.18419523723726193394) in libpyo3-826a4fb06548ad87.rlib[17](pyo3-826a4fb06548ad87.pyo3.1cceb9e92f39dd66-cgu.14.rcgu.o)
            "_PyErr_SetString", referenced from:
                pyo3::err::err_state::lazy_into_normalized_ffi_tuple::ha4d7907a47e72650 (.llvm.18419523723726193394) in libpyo3-826a4fb06548ad87.rlib[17](pyo3-826a4fb06548ad87.pyo3.1cceb9e92f39dd66-cgu.14.rcgu.o)
            "_PyErr_WriteUnraisable", referenced from:
                pyo3::impl_::trampoline::trampoline_unraisable::hf59229d0dce783be in mistralrs.mistralrs.5ac1da7054f9a315-cgu.07.rcgu.o
                _$LT$pyo3..exceptions..PyPendingDeprecationWarning$u20$as$u20$core..fmt..Display$GT$::fmt::h4c527b2e474517c2 in libpyo3-826a4fb06548ad87.rlib[3](pyo3-826a4fb06548ad87.pyo3.1cceb9e92f39dd66-cgu.00.rcgu.o)
                _$LT$pyo3..exceptions..PyPendingDeprecationWarning$u20$as$u20$core..fmt..Display$GT$::fmt::h4c527b2e474517c2 in libpyo3-826a4fb06548ad87.rlib[3](pyo3-826a4fb06548ad87.pyo3.1cceb9e92f39dd66-cgu.00.rcgu.o)
                _$LT$pyo3..types..module..PyModule$u20$as$u20$core..fmt..Display$GT$::fmt::h4c77f4f8bf899af4 in libpyo3-826a4fb06548ad87.rlib[12](pyo3-826a4fb06548ad87.pyo3.1cceb9e92f39dd66-cgu.09.rcgu.o)
                _$LT$pyo3..types..module..PyModule$u20$as$u20$core..fmt..Display$GT$::fmt::h4c77f4f8bf899af4 in libpyo3-826a4fb06548ad87.rlib[12](pyo3-826a4fb06548ad87.pyo3.1cceb9e92f39dd66-cgu.09.rcgu.o)
            "_PyExc_AttributeError", referenced from:
                pyo3::types::module::PyModule::index::hbfa5d2c6e001f805 in libpyo3-826a4fb06548ad87.rlib[12](pyo3-826a4fb06548ad87.pyo3.1cceb9e92f39dd66-cgu.09.rcgu.o)
            "_PyExc_BaseException", referenced from:
                pyo3::sync::GILOnceCell$LT$T$GT$::init::hadfa538172d7c7bd in libpyo3-826a4fb06548ad87.rlib[6](pyo3-826a4fb06548ad87.pyo3.1cceb9e92f39dd66-cgu.03.rcgu.o)
            "_PyExc_ImportError", referenced from:
                core::ops::function::FnOnce::call_once$u7b$$u7b$vtable.shim$u7d$$u7d$::hbda58aa353a8fa17 in libpyo3-826a4fb06548ad87.rlib[13](pyo3-826a4fb06548ad87.pyo3.1cceb9e92f39dd66-cgu.10.rcgu.o)
            "_PyExc_OverflowError", referenced from:
                core::ops::function::FnOnce::call_once$u7b$$u7b$vtable.shim$u7d$$u7d$::hef6d0253f700a929 in libpyo3-826a4fb06548ad87.rlib[14](pyo3-826a4fb06548ad87.pyo3.1cceb9e92f39dd66-cgu.11.rcgu.o)
            "_PyExc_RuntimeError", referenced from:
                core::ops::function::FnOnce::call_once$u7b$$u7b$vtable.shim$u7d$$u7d$::h78911697c2b91c3c in libpyo3-826a4fb06548ad87.rlib[4](pyo3-826a4fb06548ad87.pyo3.1cceb9e92f39dd66-cgu.01.rcgu.o)
                core::ops::function::FnOnce::call_once$u7b$$u7b$vtable.shim$u7d$$u7d$::h78911697c2b91c3c in libpyo3-826a4fb06548ad87.rlib[13](pyo3-826a4fb06548ad87.pyo3.1cceb9e92f39dd66-cgu.10.rcgu.o)
            "_PyExc_SystemError", referenced from:
                core::ops::function::FnOnce::call_once$u7b$$u7b$vtable.shim$u7d$$u7d$::h5d05c243b0c0f901 in mistralrs.mistralrs.5ac1da7054f9a315-cgu.00.rcgu.o
                core::ops::function::FnOnce::call_once$u7b$$u7b$vtable.shim$u7d$$u7d$::h5d05c243b0c0f901 (.llvm.583042539351141793) in mistralrs.mistralrs.5ac1da7054f9a315-cgu.07.rcgu.o
                core::ops::function::FnOnce::call_once$u7b$$u7b$vtable.shim$u7d$$u7d$::h5d05c243b0c0f901 (.llvm.16673687617079832658) in mistralrs.mistralrs.5ac1da7054f9a315-cgu.13.rcgu.o
                core::ops::function::FnOnce::call_once$u7b$$u7b$vtable.shim$u7d$$u7d$::h2101c1204cdb6375 in libpyo3-826a4fb06548ad87.rlib[4](pyo3-826a4fb06548ad87.pyo3.1cceb9e92f39dd66-cgu.01.rcgu.o)
                core::ops::function::FnOnce::call_once$u7b$$u7b$vtable.shim$u7d$$u7d$::h2101c1204cdb6375 in libpyo3-826a4fb06548ad87.rlib[6](pyo3-826a4fb06548ad87.pyo3.1cceb9e92f39dd66-cgu.03.rcgu.o)
                core::ops::function::FnOnce::call_once$u7b$$u7b$vtable.shim$u7d$$u7d$::h2101c1204cdb6375 in libpyo3-826a4fb06548ad87.rlib[7](pyo3-826a4fb06548ad87.pyo3.1cceb9e92f39dd66-cgu.04.rcgu.o)
                core::ops::function::FnOnce::call_once$u7b$$u7b$vtable.shim$u7d$$u7d$::h2101c1204cdb6375 (.llvm.5751547217046959959) in libpyo3-826a4fb06548ad87.rlib[8](pyo3-826a4fb06548ad87.pyo3.1cceb9e92f39dd66-cgu.05.rcgu.o)
                ...
            "_PyExc_TypeError", referenced from:
                core::ops::function::FnOnce::call_once$u7b$$u7b$vtable.shim$u7d$$u7d$::h97e35e7e149ad451 in mistralrs.mistralrs.5ac1da7054f9a315-cgu.13.rcgu.o
                core::ops::function::FnOnce::call_once$u7b$$u7b$vtable.shim$u7d$$u7d$::hae02a95e0a790b96 (.llvm.9008972100799772007) in libpyo3-826a4fb06548ad87.rlib[4](pyo3-826a4fb06548ad87.pyo3.1cceb9e92f39dd66-cgu.01.rcgu.o)
                core::ops::function::FnOnce::call_once$u7b$$u7b$vtable.shim$u7d$$u7d$::ha25df84fbb704f5e in libpyo3-826a4fb06548ad87.rlib[7](pyo3-826a4fb06548ad87.pyo3.1cceb9e92f39dd66-cgu.04.rcgu.o)
                core::ops::function::FnOnce::call_once$u7b$$u7b$vtable.shim$u7d$$u7d$::h139fcaf62bc86822 (.llvm.9797303095938556425) in libpyo3-826a4fb06548ad87.rlib[15](pyo3-826a4fb06548ad87.pyo3.1cceb9e92f39dd66-cgu.12.rcgu.o)
                core::ops::function::FnOnce::call_once$u7b$$u7b$vtable.shim$u7d$$u7d$::ha25df84fbb704f5e in libpyo3-826a4fb06548ad87.rlib[15](pyo3-826a4fb06548ad87.pyo3.1cceb9e92f39dd66-cgu.12.rcgu.o)
                pyo3::impl_::extract_argument::argument_extraction_error::h14ee3f0463d48df7 in libpyo3-826a4fb06548ad87.rlib[15](pyo3-826a4fb06548ad87.pyo3.1cceb9e92f39dd66-cgu.12.rcgu.o)
                core::ops::function::FnOnce::call_once$u7b$$u7b$vtable.shim$u7d$$u7d$::h139fcaf62bc86822 in libpyo3-826a4fb06548ad87.rlib[17](pyo3-826a4fb06548ad87.pyo3.1cceb9e92f39dd66-cgu.14.rcgu.o)
                ...
            "_PyExc_ValueError", referenced from:
                core::ops::function::FnOnce::call_once$u7b$$u7b$vtable.shim$u7d$$u7d$::h566297d2468b9892 in mistralrs.mistralrs.5ac1da7054f9a315-cgu.00.rcgu.o
                core::ops::function::FnOnce::call_once$u7b$$u7b$vtable.shim$u7d$$u7d$::ha05fe3ca248698f3 in mistralrs.mistralrs.5ac1da7054f9a315-cgu.00.rcgu.o
                core::ops::function::FnOnce::call_once$u7b$$u7b$vtable.shim$u7d$$u7d$::h566297d2468b9892 in mistralrs.mistralrs.5ac1da7054f9a315-cgu.02.rcgu.o
                core::ops::function::FnOnce::call_once$u7b$$u7b$vtable.shim$u7d$$u7d$::ha05fe3ca248698f3 in mistralrs.mistralrs.5ac1da7054f9a315-cgu.02.rcgu.o
                core::ops::function::FnOnce::call_once$u7b$$u7b$vtable.shim$u7d$$u7d$::hb1c484087bd98350 in mistralrs.mistralrs.5ac1da7054f9a315-cgu.02.rcgu.o
                core::ops::function::FnOnce::call_once$u7b$$u7b$vtable.shim$u7d$$u7d$::h566297d2468b9892 in mistralrs.mistralrs.5ac1da7054f9a315-cgu.11.rcgu.o
                core::ops::function::FnOnce::call_once$u7b$$u7b$vtable.shim$u7d$$u7d$::ha05fe3ca248698f3 in mistralrs.mistralrs.5ac1da7054f9a315-cgu.11.rcgu.o
                ...
            "_PyException_GetCause", referenced from:
                pyo3::err::PyErr::cause::he49e01e5a9e1cade in libpyo3-826a4fb06548ad87.rlib[4](pyo3-826a4fb06548ad87.pyo3.1cceb9e92f39dd66-cgu.01.rcgu.o)
                pyo3::err::PyErr::cause::he49e01e5a9e1cade in libpyo3-826a4fb06548ad87.rlib[4](pyo3-826a4fb06548ad87.pyo3.1cceb9e92f39dd66-cgu.01.rcgu.o)
            "_PyException_GetTraceback", referenced from:
                pyo3::err::PyErr::from_value::h5c20fc5f9956402e in libpyo3-826a4fb06548ad87.rlib[4](pyo3-826a4fb06548ad87.pyo3.1cceb9e92f39dd66-cgu.01.rcgu.o)
            "_PyException_SetCause", referenced from:
                pyo3::impl_::pyclass::lazy_type_object::wrap_in_runtime_error::ha5ba3d6682787a79 in libpyo3-826a4fb06548ad87.rlib[13](pyo3-826a4fb06548ad87.pyo3.1cceb9e92f39dd66-cgu.10.rcgu.o)
                pyo3::impl_::extract_argument::argument_extraction_error::h14ee3f0463d48df7 in libpyo3-826a4fb06548ad87.rlib[15](pyo3-826a4fb06548ad87.pyo3.1cceb9e92f39dd66-cgu.12.rcgu.o)
            "_PyException_SetTraceback", referenced from:
                pyo3::err::PyErr::into_value::h193a2cb44ca517e6 in libpyo3-826a4fb06548ad87.rlib[4](pyo3-826a4fb06548ad87.pyo3.1cceb9e92f39dd66-cgu.01.rcgu.o)
            "_PyFloat_AsDouble", referenced from:
                pyo3::types::floatob::_$LT$impl$u20$pyo3..conversion..FromPyObject$u20$for$u20$f64$GT$::extract::h50af161b147081a6 in libpyo3-826a4fb06548ad87.rlib[6](pyo3-826a4fb06548ad87.pyo3.1cceb9e92f39dd66-cgu.03.rcgu.o)
                pyo3::types::floatob::_$LT$impl$u20$pyo3..conversion..FromPyObject$u20$for$u20$f32$GT$::extract::hbd51b1a4f433abbc in libpyo3-826a4fb06548ad87.rlib[6](pyo3-826a4fb06548ad87.pyo3.1cceb9e92f39dd66-cgu.03.rcgu.o)
            "_PyFloat_Type", referenced from:
                pyo3::types::floatob::_$LT$impl$u20$pyo3..conversion..FromPyObject$u20$for$u20$f64$GT$::extract::h50af161b147081a6 in libpyo3-826a4fb06548ad87.rlib[6](pyo3-826a4fb06548ad87.pyo3.1cceb9e92f39dd66-cgu.03.rcgu.o)
                pyo3::types::floatob::_$LT$impl$u20$pyo3..conversion..FromPyObject$u20$for$u20$f32$GT$::extract::hbd51b1a4f433abbc in libpyo3-826a4fb06548ad87.rlib[6](pyo3-826a4fb06548ad87.pyo3.1cceb9e92f39dd66-cgu.03.rcgu.o)
            "_PyGILState_Ensure", referenced from:
                pyo3::gil::GILGuard::acquire::h9b35e5b340abfdbb in libpyo3-826a4fb06548ad87.rlib[18](pyo3-826a4fb06548ad87.pyo3.1cceb9e92f39dd66-cgu.15.rcgu.o)
            "_PyGILState_Release", referenced from:
                _$LT$pyo3..err..PyErr$u20$as$u20$core..fmt..Debug$GT$::fmt::h97d9b277989d63c4 in libpyo3-826a4fb06548ad87.rlib[4](pyo3-826a4fb06548ad87.pyo3.1cceb9e92f39dd66-cgu.01.rcgu.o)
                _$LT$pyo3..err..PyErr$u20$as$u20$core..fmt..Debug$GT$::fmt::h97d9b277989d63c4 in libpyo3-826a4fb06548ad87.rlib[4](pyo3-826a4fb06548ad87.pyo3.1cceb9e92f39dd66-cgu.01.rcgu.o)
                _$LT$pyo3..gil..GILGuard$u20$as$u20$core..ops..drop..Drop$GT$::drop::hd4a0aab517f5b599 in libpyo3-826a4fb06548ad87.rlib[18](pyo3-826a4fb06548ad87.pyo3.1cceb9e92f39dd66-cgu.15.rcgu.o)
            "_PyInterpreterState_Get", referenced from:
                pyo3::impl_::pymodule::ModuleDef::make_module::h97e88f3ebb53a443 in libpyo3-826a4fb06548ad87.rlib[13](pyo3-826a4fb06548ad87.pyo3.1cceb9e92f39dd66-cgu.10.rcgu.o)
            "_PyInterpreterState_GetID", referenced from:
                pyo3::impl_::pymodule::ModuleDef::make_module::h97e88f3ebb53a443 in libpyo3-826a4fb06548ad87.rlib[13](pyo3-826a4fb06548ad87.pyo3.1cceb9e92f39dd66-cgu.10.rcgu.o)
            "_PyIter_Next", referenced from:
                _$LT$$RF$pyo3..types..iterator..PyIterator$u20$as$u20$core..iter..traits..iterator..Iterator$GT$::next::ha3feafea70c2e468 in libpyo3-826a4fb06548ad87.rlib[11](pyo3-826a4fb06548ad87.pyo3.1cceb9e92f39dd66-cgu.08.rcgu.o)
            "_PyList_Append", referenced from:
                pyo3::types::list::PyList::append::inner::he8eff9a29e4ee1a0 in libpyo3-826a4fb06548ad87.rlib[13](pyo3-826a4fb06548ad87.pyo3.1cceb9e92f39dd66-cgu.10.rcgu.o)
            "_PyList_New", referenced from:
                pyo3::types::module::PyModule::index::hbfa5d2c6e001f805 in libpyo3-826a4fb06548ad87.rlib[12](pyo3-826a4fb06548ad87.pyo3.1cceb9e92f39dd66-cgu.09.rcgu.o)
            "_PyLong_AsLong", referenced from:
                pyo3::conversions::std::num::_$LT$impl$u20$pyo3..conversion..FromPyObject$u20$for$u20$u32$GT$::extract::h5e02bc6836cb95de in libpyo3-826a4fb06548ad87.rlib[14](pyo3-826a4fb06548ad87.pyo3.1cceb9e92f39dd66-cgu.11.rcgu.o)
                pyo3::conversions::std::num::_$LT$impl$u20$pyo3..conversion..FromPyObject$u20$for$u20$isize$GT$::extract::h19c1a6b9eb2b367e in libpyo3-826a4fb06548ad87.rlib[14](pyo3-826a4fb06548ad87.pyo3.1cceb9e92f39dd66-cgu.11.rcgu.o)
            "_PyLong_AsUnsignedLongLong", referenced from:
                pyo3::conversions::std::num::_$LT$impl$u20$pyo3..conversion..FromPyObject$u20$for$u20$u64$GT$::extract::h5bdd9507188d2707 in libpyo3-826a4fb06548ad87.rlib[14](pyo3-826a4fb06548ad87.pyo3.1cceb9e92f39dd66-cgu.11.rcgu.o)
            "_PyLong_FromLong", referenced from:
                pyo3::conversions::std::num::_$LT$impl$u20$pyo3..conversion..IntoPy$LT$pyo3..instance..Py$LT$pyo3..types..any..PyAny$GT$$GT$$u20$for$u20$isize$GT$::into_py::h2b11672e0e75c815 in libpyo3-826a4fb06548ad87.rlib[14](pyo3-826a4fb06548ad87.pyo3.1cceb9e92f39dd66-cgu.11.rcgu.o)
            "_PyLong_FromSsize_t", referenced from:
                pyo3::impl_::pyclass::get_sequence_item_from_mapping::h10b17877b15cbe14 in libpyo3-826a4fb06548ad87.rlib[10](pyo3-826a4fb06548ad87.pyo3.1cceb9e92f39dd66-cgu.07.rcgu.o)
                pyo3::impl_::pyclass::assign_sequence_item_from_mapping::hb05d7c7ecccf2a57 in libpyo3-826a4fb06548ad87.rlib[10](pyo3-826a4fb06548ad87.pyo3.1cceb9e92f39dd66-cgu.07.rcgu.o)
            "_PyLong_FromUnsignedLongLong", referenced from:
                pyo3::conversions::std::num::_$LT$impl$u20$pyo3..conversion..IntoPy$LT$pyo3..instance..Py$LT$pyo3..types..any..PyAny$GT$$GT$$u20$for$u20$usize$GT$::into_py::h873f93d6323cafd9 in libpyo3-826a4fb06548ad87.rlib[14](pyo3-826a4fb06548ad87.pyo3.1cceb9e92f39dd66-cgu.11.rcgu.o)
            "_PyMapping_Keys", referenced from:
                mistralrs::ChatCompletionRequest::new::hd539ce096d525341 in mistralrs.mistralrs.5ac1da7054f9a315-cgu.00.rcgu.o
            "_PyMapping_Size", referenced from:
                mistralrs::ChatCompletionRequest::new::hd539ce096d525341 in mistralrs.mistralrs.5ac1da7054f9a315-cgu.00.rcgu.o
            "_PyMapping_Values", referenced from:
                mistralrs::ChatCompletionRequest::new::hd539ce096d525341 in mistralrs.mistralrs.5ac1da7054f9a315-cgu.00.rcgu.o
            "_PyMem_Malloc", referenced from:
                core::ops::function::FnOnce::call_once$u7b$$u7b$vtable.shim$u7d$$u7d$::h5888067ad10ee069 in libpyo3-826a4fb06548ad87.rlib[7](pyo3-826a4fb06548ad87.pyo3.1cceb9e92f39dd66-cgu.04.rcgu.o)
                pyo3::pyclass::create_type_object::PyTypeBuilder::type_doc::_$u7b$$u7b$closure$u7d$$u7d$::hb1bc6788a5debeab in libpyo3-826a4fb06548ad87.rlib[7](pyo3-826a4fb06548ad87.pyo3.1cceb9e92f39dd66-cgu.04.rcgu.o)
            "_PyModule_Create2", referenced from:
                pyo3::sync::GILOnceCell$LT$T$GT$::init::hc0a18d1bb83d0f90 in libpyo3-826a4fb06548ad87.rlib[6](pyo3-826a4fb06548ad87.pyo3.1cceb9e92f39dd66-cgu.03.rcgu.o)
            "_PyNumber_Index", referenced from:
                pyo3::conversions::std::num::_$LT$impl$u20$pyo3..conversion..FromPyObject$u20$for$u20$u32$GT$::extract::h5e02bc6836cb95de in libpyo3-826a4fb06548ad87.rlib[14](pyo3-826a4fb06548ad87.pyo3.1cceb9e92f39dd66-cgu.11.rcgu.o)
                pyo3::conversions::std::num::_$LT$impl$u20$pyo3..conversion..FromPyObject$u20$for$u20$isize$GT$::extract::h19c1a6b9eb2b367e in libpyo3-826a4fb06548ad87.rlib[14](pyo3-826a4fb06548ad87.pyo3.1cceb9e92f39dd66-cgu.11.rcgu.o)
                pyo3::conversions::std::num::_$LT$impl$u20$pyo3..conversion..FromPyObject$u20$for$u20$u64$GT$::extract::h5bdd9507188d2707 in libpyo3-826a4fb06548ad87.rlib[14](pyo3-826a4fb06548ad87.pyo3.1cceb9e92f39dd66-cgu.11.rcgu.o)
            "_PyObject_Call", referenced from:
                pyo3::instance::Py$LT$T$GT$::call_method::h10ae0414e8ac5041 in mistralrs.mistralrs.5ac1da7054f9a315-cgu.13.rcgu.o
                pyo3::instance::Py$LT$T$GT$::call::hfb5cd76c2093d45d in mistralrs.mistralrs.5ac1da7054f9a315-cgu.13.rcgu.o
            "_PyObject_DelItem", referenced from:
                pyo3::impl_::pyclass::assign_sequence_item_from_mapping::hb05d7c7ecccf2a57 in libpyo3-826a4fb06548ad87.rlib[10](pyo3-826a4fb06548ad87.pyo3.1cceb9e92f39dd66-cgu.07.rcgu.o)
            "_PyObject_Free", referenced from:
                core::ops::function::FnOnce::call_once$u7b$$u7b$vtable.shim$u7d$$u7d$::h5888067ad10ee069 in libpyo3-826a4fb06548ad87.rlib[7](pyo3-826a4fb06548ad87.pyo3.1cceb9e92f39dd66-cgu.04.rcgu.o)
                pyo3::pyclass::create_type_object::PyTypeBuilder::type_doc::_$u7b$$u7b$closure$u7d$$u7d$::hb1bc6788a5debeab in libpyo3-826a4fb06548ad87.rlib[7](pyo3-826a4fb06548ad87.pyo3.1cceb9e92f39dd66-cgu.04.rcgu.o)
            "_PyObject_GC_UnTrack", referenced from:
                pyo3::impl_::pyclass::tp_dealloc_with_gc::h464ed8e56c39491f in mistralrs.mistralrs.5ac1da7054f9a315-cgu.07.rcgu.o
                pyo3::impl_::pyclass::tp_dealloc_with_gc::h50afd5b924f94779 in mistralrs.mistralrs.5ac1da7054f9a315-cgu.07.rcgu.o
                pyo3::impl_::pyclass::tp_dealloc_with_gc::h7066cb8cb1ce472e in mistralrs.mistralrs.5ac1da7054f9a315-cgu.07.rcgu.o
                pyo3::impl_::pyclass::tp_dealloc_with_gc::h9a044185c76b30ec in mistralrs.mistralrs.5ac1da7054f9a315-cgu.07.rcgu.o
                pyo3::impl_::pyclass::tp_dealloc_with_gc::ha632fa38b11c3c7a in mistralrs.mistralrs.5ac1da7054f9a315-cgu.07.rcgu.o
                pyo3::impl_::pyclass::tp_dealloc_with_gc::hbdec8dd03bb3e9a6 in mistralrs.mistralrs.5ac1da7054f9a315-cgu.07.rcgu.o
                pyo3::impl_::pyclass::tp_dealloc_with_gc::hd266ac15105812cc in mistralrs.mistralrs.5ac1da7054f9a315-cgu.07.rcgu.o
                ...
            "_PyObject_GenericGetDict", referenced from:
                pyo3::pyclass::create_type_object::create_type_object::inner::hd5338cffe7cc32e5 in libpyo3-826a4fb06548ad87.rlib[7](pyo3-826a4fb06548ad87.pyo3.1cceb9e92f39dd66-cgu.04.rcgu.o)
            "_PyObject_GenericSetDict", referenced from:
                pyo3::pyclass::create_type_object::create_type_object::inner::hd5338cffe7cc32e5 in libpyo3-826a4fb06548ad87.rlib[7](pyo3-826a4fb06548ad87.pyo3.1cceb9e92f39dd66-cgu.04.rcgu.o)
            "_PyObject_GetAttr", referenced from:
                pyo3::instance::Py$LT$T$GT$::call_method::h10ae0414e8ac5041 in mistralrs.mistralrs.5ac1da7054f9a315-cgu.13.rcgu.o
                pyo3::types::any::PyAny::_getattr::hb5ecbb6d068b2203 (.llvm.13472963216848620627) in libpyo3-826a4fb06548ad87.rlib[12](pyo3-826a4fb06548ad87.pyo3.1cceb9e92f39dd66-cgu.09.rcgu.o)
            "_PyObject_GetItem", referenced from:
                pyo3::impl_::pyclass::get_sequence_item_from_mapping::h10b17877b15cbe14 in libpyo3-826a4fb06548ad87.rlib[10](pyo3-826a4fb06548ad87.pyo3.1cceb9e92f39dd66-cgu.07.rcgu.o)
            "_PyObject_GetIter", referenced from:
                pyo3::types::iterator::PyIterator::from_object::ha4dac5691c91cf41 in libpyo3-826a4fb06548ad87.rlib[11](pyo3-826a4fb06548ad87.pyo3.1cceb9e92f39dd66-cgu.08.rcgu.o)
            "_PyObject_Repr", referenced from:
                _$LT$pyo3..exceptions..PyPendingDeprecationWarning$u20$as$u20$core..fmt..Debug$GT$::fmt::hc688821381a5e213 in libpyo3-826a4fb06548ad87.rlib[3](pyo3-826a4fb06548ad87.pyo3.1cceb9e92f39dd66-cgu.00.rcgu.o)
                _$LT$pyo3..types..traceback..PyTraceback$u20$as$u20$core..fmt..Debug$GT$::fmt::h02831b59a59ceb5f in libpyo3-826a4fb06548ad87.rlib[13](pyo3-826a4fb06548ad87.pyo3.1cceb9e92f39dd66-cgu.10.rcgu.o)
            "_PyObject_SetAttr", referenced from:
                pyo3::types::any::PyAny::setattr::inner::h7713ae2839a1bede in libpyo3-826a4fb06548ad87.rlib[12](pyo3-826a4fb06548ad87.pyo3.1cceb9e92f39dd66-cgu.09.rcgu.o)
            "_PyObject_SetAttrString", referenced from:
                pyo3::impl_::pyclass::lazy_type_object::initialize_tp_dict::ha4381268de11d07c in libpyo3-826a4fb06548ad87.rlib[13](pyo3-826a4fb06548ad87.pyo3.1cceb9e92f39dd66-cgu.10.rcgu.o)
            "_PyObject_SetItem", referenced from:
                pyo3::impl_::pyclass::assign_sequence_item_from_mapping::hb05d7c7ecccf2a57 in libpyo3-826a4fb06548ad87.rlib[10](pyo3-826a4fb06548ad87.pyo3.1cceb9e92f39dd66-cgu.07.rcgu.o)
            "_PyObject_Str", referenced from:
                _$LT$pyo3..exceptions..PyPendingDeprecationWarning$u20$as$u20$core..fmt..Display$GT$::fmt::h4c527b2e474517c2 in libpyo3-826a4fb06548ad87.rlib[3](pyo3-826a4fb06548ad87.pyo3.1cceb9e92f39dd66-cgu.00.rcgu.o)
                pyo3::err::PyErr::_take::_$u7b$$u7b$closure$u7d$$u7d$::h33f071e8931afab1 in libpyo3-826a4fb06548ad87.rlib[4](pyo3-826a4fb06548ad87.pyo3.1cceb9e92f39dd66-cgu.01.rcgu.o)
                _$LT$pyo3..types..module..PyModule$u20$as$u20$core..fmt..Display$GT$::fmt::h4c77f4f8bf899af4 in libpyo3-826a4fb06548ad87.rlib[12](pyo3-826a4fb06548ad87.pyo3.1cceb9e92f39dd66-cgu.09.rcgu.o)
            "_PySequence_Check", referenced from:
                pyo3::types::sequence::_$LT$impl$u20$pyo3..conversion..FromPyObject$u20$for$u20$alloc..vec..Vec$LT$T$GT$$GT$::extract::h44948943e2b3eab8 in mistralrs.mistralrs.5ac1da7054f9a315-cgu.13.rcgu.o
                pyo3::types::sequence::_$LT$impl$u20$pyo3..conversion..FromPyObject$u20$for$u20$alloc..vec..Vec$LT$T$GT$$GT$::extract::hf8b106d805de6c60 in mistralrs.mistralrs.5ac1da7054f9a315-cgu.13.rcgu.o
            "_PySequence_GetItem", referenced from:
                mistralrs::ChatCompletionRequest::new::hd539ce096d525341 in mistralrs.mistralrs.5ac1da7054f9a315-cgu.00.rcgu.o
                mistralrs::ChatCompletionRequest::new::hd539ce096d525341 in mistralrs.mistralrs.5ac1da7054f9a315-cgu.00.rcgu.o
            "_PySequence_Size", referenced from:
                pyo3::types::sequence::_$LT$impl$u20$pyo3..conversion..FromPyObject$u20$for$u20$alloc..vec..Vec$LT$T$GT$$GT$::extract::h44948943e2b3eab8 in mistralrs.mistralrs.5ac1da7054f9a315-cgu.13.rcgu.o
                pyo3::types::sequence::_$LT$impl$u20$pyo3..conversion..FromPyObject$u20$for$u20$alloc..vec..Vec$LT$T$GT$$GT$::extract::hf8b106d805de6c60 in mistralrs.mistralrs.5ac1da7054f9a315-cgu.13.rcgu.o
            "_PyTuple_New", referenced from:
                pyo3::types::tuple::PyTuple::new::h85059b13432604b6 in mistralrs.mistralrs.5ac1da7054f9a315-cgu.08.rcgu.o
                pyo3::types::tuple::_$LT$impl$u20$pyo3..conversion..IntoPy$LT$pyo3..instance..Py$LT$pyo3..types..any..PyAny$GT$$GT$$u20$for$u20$$LP$T0$C$$RP$$GT$::into_py::hd2cedfa68dde8f20 in libpyo3-826a4fb06548ad87.rlib[11](pyo3-826a4fb06548ad87.pyo3.1cceb9e92f39dd66-cgu.08.rcgu.o)
                core::ops::function::FnOnce::call_once$u7b$$u7b$vtable.shim$u7d$$u7d$::h96d76610e1f054c3 in libpyo3-826a4fb06548ad87.rlib[14](pyo3-826a4fb06548ad87.pyo3.1cceb9e92f39dd66-cgu.11.rcgu.o)
            "_PyType_FromSpec", referenced from:
                pyo3::pyclass::create_type_object::create_type_object::inner::hd5338cffe7cc32e5 in libpyo3-826a4fb06548ad87.rlib[7](pyo3-826a4fb06548ad87.pyo3.1cceb9e92f39dd66-cgu.04.rcgu.o)
            "_PyType_GenericAlloc", referenced from:
                _$LT$pyo3..pyclass_init..PyNativeTypeInitializer$LT$T$GT$$u20$as$u20$pyo3..pyclass_init..PyObjectInit$LT$T$GT$$GT$::into_new_object::inner::hacc097fac45e573f in libpyo3-826a4fb06548ad87.rlib[17](pyo3-826a4fb06548ad87.pyo3.1cceb9e92f39dd66-cgu.14.rcgu.o)
            "_PyType_IsSubtype", referenced from:
                mistralrs::_::_$LT$impl$u20$mistralrs..ModelKind$GT$::__pymethod___default___pyo3__repr______::h5d5bca718395e087 in mistralrs.mistralrs.5ac1da7054f9a315-cgu.00.rcgu.o
                mistralrs::_::_$LT$impl$u20$mistralrs..ModelKind$GT$::__pymethod___default___pyo3__int______::hbe34ef8cbd90e361 in mistralrs.mistralrs.5ac1da7054f9a315-cgu.00.rcgu.o
                mistralrs::_::_$LT$impl$u20$mistralrs..ModelKind$GT$::__pymethod___default___pyo3__richcmp______::h3da2b9c488e5751e in mistralrs.mistralrs.5ac1da7054f9a315-cgu.00.rcgu.o
                mistralrs::_::_$LT$impl$u20$mistralrs..DType$GT$::__pymethod___default___pyo3__repr______::h37d2e32aae4d9c5d in mistralrs.mistralrs.5ac1da7054f9a315-cgu.00.rcgu.o
                mistralrs::_::_$LT$impl$u20$mistralrs..DType$GT$::__pymethod___default___pyo3__int______::h7094c562ef2c251f in mistralrs.mistralrs.5ac1da7054f9a315-cgu.00.rcgu.o
                mistralrs::_::_$LT$impl$u20$mistralrs..DType$GT$::__pymethod___default___pyo3__richcmp______::h8e14f40e05da3268 in mistralrs.mistralrs.5ac1da7054f9a315-cgu.00.rcgu.o
                mistralrs::_::_$LT$impl$u20$pyo3..impl_..pyclass..PyMethods$LT$mistralrs..Runner$GT$$u20$for$u20$pyo3..impl_..pyclass..PyClassImplCollector$LT$mistralrs..Runner$GT$$GT$::py_methods::ITEMS::trampoline::h43ec32f420acaf63 (.llvm.6094665132293408195) in mistralrs.mistralrs.5ac1da7054f9a315-cgu.00.rcgu.o
                mistralrs::_::_$LT$impl$u20$pyo3..impl_..pyclass..PyMethods$LT$mistralrs..Runner$GT$$u20$for$u20$pyo3..impl_..pyclass..PyClassImplCollector$LT$mistralrs..Runner$GT$$GT$::py_methods::ITEMS::trampoline::h43ec32f420acaf63 (.llvm.6094665132293408195) in mistralrs.mistralrs.5ac1da7054f9a315-cgu.00.rcgu.o
                ...
            "_PyUnicode_AsEncodedString", referenced from:
                pyo3::types::string::PyString::to_string_lossy::h04a943aca21dc817 in libpyo3-826a4fb06548ad87.rlib[8](pyo3-826a4fb06548ad87.pyo3.1cceb9e92f39dd66-cgu.05.rcgu.o)
            "_PyUnicode_AsUTF8AndSize", referenced from:
                pyo3::impl_::extract_argument::FunctionDescription::extract_arguments_fastcall::h80cf6097bb8b4b18 in mistralrs.mistralrs.5ac1da7054f9a315-cgu.07.rcgu.o
                pyo3::impl_::extract_argument::FunctionDescription::extract_arguments_tuple_dict::he2845f7354825c5e in mistralrs.mistralrs.5ac1da7054f9a315-cgu.07.rcgu.o
                pyo3::conversions::std::string::_$LT$impl$u20$pyo3..conversion..FromPyObject$u20$for$u20$alloc..string..String$GT$::extract::h130406c72aee78ba in libpyo3-826a4fb06548ad87.rlib[8](pyo3-826a4fb06548ad87.pyo3.1cceb9e92f39dd66-cgu.05.rcgu.o)
                pyo3::types::string::PyString::to_string_lossy::h04a943aca21dc817 in libpyo3-826a4fb06548ad87.rlib[8](pyo3-826a4fb06548ad87.pyo3.1cceb9e92f39dd66-cgu.05.rcgu.o)
                pyo3::conversions::std::string::_$LT$impl$u20$pyo3..conversion..FromPyObject$u20$for$u20$$RF$str$GT$::extract::haf730fda7404349a in libpyo3-826a4fb06548ad87.rlib[14](pyo3-826a4fb06548ad87.pyo3.1cceb9e92f39dd66-cgu.11.rcgu.o)
            "_PyUnicode_FromStringAndSize", referenced from:
                core::ops::function::FnOnce::call_once$u7b$$u7b$vtable.shim$u7d$$u7d$::h2101c1204cdb6375 in libpyo3-826a4fb06548ad87.rlib[4](pyo3-826a4fb06548ad87.pyo3.1cceb9e92f39dd66-cgu.01.rcgu.o)
                _$LT$pyo3..err..PyDowncastErrorArguments$u20$as$u20$pyo3..err..err_state..PyErrArguments$GT$::arguments::hd934f8233648133b in libpyo3-826a4fb06548ad87.rlib[4](pyo3-826a4fb06548ad87.pyo3.1cceb9e92f39dd66-cgu.01.rcgu.o)
                core::ops::function::FnOnce::call_once$u7b$$u7b$vtable.shim$u7d$$u7d$::h2101c1204cdb6375 in libpyo3-826a4fb06548ad87.rlib[6](pyo3-826a4fb06548ad87.pyo3.1cceb9e92f39dd66-cgu.03.rcgu.o)
                core::ops::function::FnOnce::call_once$u7b$$u7b$vtable.shim$u7d$$u7d$::h2101c1204cdb6375 in libpyo3-826a4fb06548ad87.rlib[7](pyo3-826a4fb06548ad87.pyo3.1cceb9e92f39dd66-cgu.04.rcgu.o)
                core::ops::function::FnOnce::call_once$u7b$$u7b$vtable.shim$u7d$$u7d$::h2101c1204cdb6375 (.llvm.5751547217046959959) in libpyo3-826a4fb06548ad87.rlib[8](pyo3-826a4fb06548ad87.pyo3.1cceb9e92f39dd66-cgu.05.rcgu.o)
                core::ops::function::FnOnce::call_once$u7b$$u7b$vtable.shim$u7d$$u7d$::hbd24e7d3ce54b7e3 (.llvm.5751547217046959959) in libpyo3-826a4fb06548ad87.rlib[8](pyo3-826a4fb06548ad87.pyo3.1cceb9e92f39dd66-cgu.05.rcgu.o)
                pyo3::conversions::std::string::_$LT$impl$u20$pyo3..conversion..IntoPy$LT$pyo3..instance..Py$LT$pyo3..types..any..PyAny$GT$$GT$$u20$for$u20$alloc..string..String$GT$::into_py::he188bd8f9d6d999b in libpyo3-826a4fb06548ad87.rlib[8](pyo3-826a4fb06548ad87.pyo3.1cceb9e92f39dd66-cgu.05.rcgu.o)
                ...
            "_PyUnicode_InternInPlace", referenced from:
                pyo3::types::string::PyString::intern::h3bdfd9b76f97db2e in libpyo3-826a4fb06548ad87.rlib[8](pyo3-826a4fb06548ad87.pyo3.1cceb9e92f39dd66-cgu.05.rcgu.o)
            "_Py_IsInitialized", referenced from:
                parking_lot::once::Once::call_once_force::_$u7b$$u7b$closure$u7d$$u7d$::h92dcd03e9bdaf03b (.llvm.14350639940780853391) in libpyo3-826a4fb06548ad87.rlib[18](pyo3-826a4fb06548ad87.pyo3.1cceb9e92f39dd66-cgu.15.rcgu.o)
                core::ops::function::FnOnce::call_once$u7b$$u7b$vtable.shim$u7d$$u7d$::h7b43ca3828124bbf (.llvm.14350639940780853391) in libpyo3-826a4fb06548ad87.rlib[18](pyo3-826a4fb06548ad87.pyo3.1cceb9e92f39dd66-cgu.15.rcgu.o)
            "__Py_Dealloc", referenced from:
                pyo3::instance::Py$LT$T$GT$::call_method::h10ae0414e8ac5041 in mistralrs.mistralrs.5ac1da7054f9a315-cgu.13.rcgu.o
                pyo3::instance::Py$LT$T$GT$::call::hfb5cd76c2093d45d in mistralrs.mistralrs.5ac1da7054f9a315-cgu.13.rcgu.o
                pyo3::impl_::pyclass::get_sequence_item_from_mapping::h10b17877b15cbe14 in libpyo3-826a4fb06548ad87.rlib[10](pyo3-826a4fb06548ad87.pyo3.1cceb9e92f39dd66-cgu.07.rcgu.o)
                pyo3::impl_::pyclass::assign_sequence_item_from_mapping::hb05d7c7ecccf2a57 in libpyo3-826a4fb06548ad87.rlib[10](pyo3-826a4fb06548ad87.pyo3.1cceb9e92f39dd66-cgu.07.rcgu.o)
                pyo3::conversions::std::num::_$LT$impl$u20$pyo3..conversion..FromPyObject$u20$for$u20$u32$GT$::extract::h5e02bc6836cb95de in libpyo3-826a4fb06548ad87.rlib[14](pyo3-826a4fb06548ad87.pyo3.1cceb9e92f39dd66-cgu.11.rcgu.o)
                pyo3::conversions::std::num::_$LT$impl$u20$pyo3..conversion..FromPyObject$u20$for$u20$isize$GT$::extract::h19c1a6b9eb2b367e in libpyo3-826a4fb06548ad87.rlib[14](pyo3-826a4fb06548ad87.pyo3.1cceb9e92f39dd66-cgu.11.rcgu.o)
                pyo3::conversions::std::num::_$LT$impl$u20$pyo3..conversion..FromPyObject$u20$for$u20$u64$GT$::extract::h5bdd9507188d2707 in libpyo3-826a4fb06548ad87.rlib[14](pyo3-826a4fb06548ad87.pyo3.1cceb9e92f39dd66-cgu.11.rcgu.o)
                ...
            "__Py_FalseStruct", referenced from:
                mistralrs::_::_$LT$impl$u20$mistralrs..ModelKind$GT$::__pymethod___default___pyo3__richcmp______::h3da2b9c488e5751e in mistralrs.mistralrs.5ac1da7054f9a315-cgu.00.rcgu.o
                mistralrs::_::_$LT$impl$u20$mistralrs..ModelKind$GT$::__pymethod___default___pyo3__richcmp______::h3da2b9c488e5751e in mistralrs.mistralrs.5ac1da7054f9a315-cgu.00.rcgu.o
                mistralrs::_::_$LT$impl$u20$mistralrs..ModelKind$GT$::__pymethod___default___pyo3__richcmp______::h3da2b9c488e5751e in mistralrs.mistralrs.5ac1da7054f9a315-cgu.00.rcgu.o
                mistralrs::_::_$LT$impl$u20$mistralrs..ModelKind$GT$::__pymethod___default___pyo3__richcmp______::h3da2b9c488e5751e in mistralrs.mistralrs.5ac1da7054f9a315-cgu.00.rcgu.o
                mistralrs::_::_$LT$impl$u20$mistralrs..DType$GT$::__pymethod___default___pyo3__richcmp______::h8e14f40e05da3268 in mistralrs.mistralrs.5ac1da7054f9a315-cgu.00.rcgu.o
                mistralrs::_::_$LT$impl$u20$mistralrs..DType$GT$::__pymethod___default___pyo3__richcmp______::h8e14f40e05da3268 in mistralrs.mistralrs.5ac1da7054f9a315-cgu.00.rcgu.o
                mistralrs::_::_$LT$impl$u20$mistralrs..DType$GT$::__pymethod___default___pyo3__richcmp______::h8e14f40e05da3268 in mistralrs.mistralrs.5ac1da7054f9a315-cgu.00.rcgu.o
                mistralrs::_::_$LT$impl$u20$mistralrs..DType$GT$::__pymethod___default___pyo3__richcmp______::h8e14f40e05da3268 in mistralrs.mistralrs.5ac1da7054f9a315-cgu.00.rcgu.o
                ...
            "__Py_NoneStruct", referenced from:
                mistralrs::_::_$LT$impl$u20$pyo3..impl_..pyclass..PyMethods$LT$mistralrs..ChatCompletionRequest$GT$$u20$for$u20$pyo3..impl_..pyclass..PyClassImplCollector$LT$mistralrs..ChatCompletionRequest$GT$$GT$::py_methods::ITEMS::trampoline::h3a81c2c867ee463d (.llvm.6094665132293408195) in mistralrs.mistralrs.5ac1da7054f9a315-cgu.00.rcgu.o
                mistralrs::loaders::NormalLoader::new::hd4cafb295b00e04d in mistralrs.mistralrs.5ac1da7054f9a315-cgu.01.rcgu.o
                mistralrs::loaders::NormalLoader::new::hd4cafb295b00e04d in mistralrs.mistralrs.5ac1da7054f9a315-cgu.01.rcgu.o
                mistralrs::loaders::NormalLoader::new::hd4cafb295b00e04d in mistralrs.mistralrs.5ac1da7054f9a315-cgu.01.rcgu.o
                mistralrs::loaders::NormalLoader::new::hd4cafb295b00e04d in mistralrs.mistralrs.5ac1da7054f9a315-cgu.01.rcgu.o
                mistralrs::loaders::NormalLoader::new::hd4cafb295b00e04d in mistralrs.mistralrs.5ac1da7054f9a315-cgu.01.rcgu.o
                mistralrs::loaders::NormalLoader::load::h065024c212950bac in mistralrs.mistralrs.5ac1da7054f9a315-cgu.01.rcgu.o
                mistralrs::loaders::NormalLoader::load::h065024c212950bac in mistralrs.mistralrs.5ac1da7054f9a315-cgu.01.rcgu.o
                mistralrs::loaders::NormalLoader::load::h065024c212950bac in mistralrs.mistralrs.5ac1da7054f9a315-cgu.01.rcgu.o
                mistralrs::loaders::NormalLoader::load::h065024c212950bac in mistralrs.mistralrs.5ac1da7054f9a315-cgu.01.rcgu.o
                ...
            "__Py_NotImplementedStruct", referenced from:
                mistralrs::_::_$LT$impl$u20$mistralrs..ModelKind$GT$::__pymethod___default___pyo3__richcmp______::h3da2b9c488e5751e in mistralrs.mistralrs.5ac1da7054f9a315-cgu.00.rcgu.o
                mistralrs::_::_$LT$impl$u20$mistralrs..ModelKind$GT$::__pymethod___default___pyo3__richcmp______::h3da2b9c488e5751e in mistralrs.mistralrs.5ac1da7054f9a315-cgu.00.rcgu.o
                mistralrs::_::_$LT$impl$u20$mistralrs..ModelKind$GT$::__pymethod___default___pyo3__richcmp______::h3da2b9c488e5751e in mistralrs.mistralrs.5ac1da7054f9a315-cgu.00.rcgu.o
                mistralrs::_::_$LT$impl$u20$mistralrs..ModelKind$GT$::__pymethod___default___pyo3__richcmp______::h3da2b9c488e5751e in mistralrs.mistralrs.5ac1da7054f9a315-cgu.00.rcgu.o
                mistralrs::_::_$LT$impl$u20$mistralrs..DType$GT$::__pymethod___default___pyo3__richcmp______::h8e14f40e05da3268 in mistralrs.mistralrs.5ac1da7054f9a315-cgu.00.rcgu.o
                mistralrs::_::_$LT$impl$u20$mistralrs..DType$GT$::__pymethod___default___pyo3__richcmp______::h8e14f40e05da3268 in mistralrs.mistralrs.5ac1da7054f9a315-cgu.00.rcgu.o
                mistralrs::_::_$LT$impl$u20$mistralrs..DType$GT$::__pymethod___default___pyo3__richcmp______::h8e14f40e05da3268 in mistralrs.mistralrs.5ac1da7054f9a315-cgu.00.rcgu.o
                mistralrs::_::_$LT$impl$u20$mistralrs..DType$GT$::__pymethod___default___pyo3__richcmp______::h8e14f40e05da3268 in mistralrs.mistralrs.5ac1da7054f9a315-cgu.00.rcgu.o
                ...
            "__Py_TrueStruct", referenced from:
                mistralrs::_::_$LT$impl$u20$mistralrs..ModelKind$GT$::__pymethod___default___pyo3__richcmp______::h3da2b9c488e5751e in mistralrs.mistralrs.5ac1da7054f9a315-cgu.00.rcgu.o
                mistralrs::_::_$LT$impl$u20$mistralrs..ModelKind$GT$::__pymethod___default___pyo3__richcmp______::h3da2b9c488e5751e in mistralrs.mistralrs.5ac1da7054f9a315-cgu.00.rcgu.o
                mistralrs::_::_$LT$impl$u20$mistralrs..ModelKind$GT$::__pymethod___default___pyo3__richcmp______::h3da2b9c488e5751e in mistralrs.mistralrs.5ac1da7054f9a315-cgu.00.rcgu.o
                mistralrs::_::_$LT$impl$u20$mistralrs..ModelKind$GT$::__pymethod___default___pyo3__richcmp______::h3da2b9c488e5751e in mistralrs.mistralrs.5ac1da7054f9a315-cgu.00.rcgu.o
                mistralrs::_::_$LT$impl$u20$mistralrs..DType$GT$::__pymethod___default___pyo3__richcmp______::h8e14f40e05da3268 in mistralrs.mistralrs.5ac1da7054f9a315-cgu.00.rcgu.o
                mistralrs::_::_$LT$impl$u20$mistralrs..DType$GT$::__pymethod___default___pyo3__richcmp______::h8e14f40e05da3268 in mistralrs.mistralrs.5ac1da7054f9a315-cgu.00.rcgu.o
                mistralrs::_::_$LT$impl$u20$mistralrs..DType$GT$::__pymethod___default___pyo3__richcmp______::h8e14f40e05da3268 in mistralrs.mistralrs.5ac1da7054f9a315-cgu.00.rcgu.o
                mistralrs::_::_$LT$impl$u20$mistralrs..DType$GT$::__pymethod___default___pyo3__richcmp______::h8e14f40e05da3268 in mistralrs.mistralrs.5ac1da7054f9a315-cgu.00.rcgu.o
                ...
          ld: symbol(s) not found for architecture arm64
          clang: error: linker command failed with exit code 1 (use -v to see invocation)

error: could not compile `mistralrs-pyo3` (lib) due to 1 previous error
biel@BIEL-Y9WC2 mistral.rs % python3 --version

Python 3.9.6
biel@BIEL-Y9WC2 mistral.rs % rustup target add aarch64-apple-darwin

info: component 'rust-std' for target 'aarch64-apple-darwin' is up to date
biel@BIEL-Y9WC2 mistral.rs % xcode-select --install

xcode-select: note: Command line tools are already installed. Use "Software Update" in System Settings or the softwareupdate command line interface to install updates
biel@BIEL-Y9WC2 mistral.rs %

EricLBuehler commented 7 months ago

According to the pyo3 faq and this issue, it looks like it can be resolved by removing the extension-modules feature flag. The extension-modules feature is needed only on Unix targets for compatibility as it disables linking to libpython. However, this might just be your problem.

I've pushed a commit which removes that feature flag so it should try to link to libpython now. Can you try building again after git pull?

biels commented 7 months ago

Okay, I was able to build now. I'll let you know if it works completely in a moment.

EricLBuehler commented 7 months ago

@biels, were you able to get it to work?

EricLBuehler commented 7 months ago

Closing, but please feel free to reeopen

ianchen06 commented 6 months ago

Hi @biels May I ask how did you compile it on macos?

I just tried the latest commit on master branch b3d130d but I am still getting the same errors with mistralrs-pyo3

EricLBuehler commented 6 months ago

Hi @ianchen06, as of 9ec90a4 this should work now.