mac os run yi trasnformer failed

[arthasyou]

The Error show as bellow

thread 'main' panicked at /Users/you/.cargo/registry/src/index.crates.io-6f17d22bba15001f/foreign-types-shared-0.3.1/src/lib.rs:72:9: assertion failed: !ptr.is_null() stack backtrace: 0: rust_begin_unwind at /rustc/cc66ad468955717ab92600c770da8c1601a4ff33/library/std/src/panicking.rs:595:5 1: core::panicking::panic_fmt at /rustc/cc66ad468955717ab92600c770da8c1601a4ff33/library/core/src/panicking.rs:67:14 2: core::panicking::panic at /rustc/cc66ad468955717ab92600c770da8c1601a4ff33/library/core/src/panicking.rs:117:5 3: foreign_types_shared::ForeignTypeRef::from_ptr at /Users/you/.cargo/registry/src/index.crates.io-6f17d22bba15001f/foreign-types-shared-0.3.1/src/lib.rs:72:9 4: ::deref at /Users/you/.cargo/registry/src/index.crates.io-6f17d22bba15001f/metal-0.27.0/src/buffer.rs:13:1 5: <&metal::buffer::Buffer as candle_metal_kernels::EncoderParam>::set_param at /Users/you/.cargo/git/checkouts/candle-0c2b4fa9e5801351/fa3ea98/candle-metal-kernels/src/lib.rs:80:43 6: candle_metal_kernels::set_param at /Users/you/.cargo/git/checkouts/candle-0c2b4fa9e5801351/fa3ea98/candle-metal-kernels/src/lib.rs:42:5 7: candle_metal_kernels::call_affine at /Users/you/.cargo/git/checkouts/candle-0c2b4fa9e5801351/fa3ea98/candle-metal-kernels/src/lib.rs:731:5 8: ::affine at /Users/you/.cargo/git/checkouts/candle-0c2b4fa9e5801351/fa3ea98/candle-core/src/metal_backend.rs:358:13 9: candle_core::storage::Storage::affine at /Users/you/.cargo/git/checkouts/candle-0c2b4fa9e5801351/fa3ea98/candle-core/src/storage.rs:76:31 10: candle_core::tensor::Tensor::affine at /Users/you/.cargo/git/checkouts/candle-0c2b4fa9e5801351/fa3ea98/candle-core/src/tensor.rs:649:23 11: <candle_core::tensor::Tensor as core::ops::arith::Mul>::mul at /Users/you/.cargo/git/checkouts/candle-0c2b4fa9e5801351/fa3ea98/candle-core/src/tensor.rs:2642:17 12: llm_rust::models::yi::Attention::forward at ./src/models/yi.rs:290:32 13: llm_rust::models::yi::DecoderLayer::forward at ./src/models/yi.rs:351:18 14: llm_rust::models::yi::Model::forward at ./src/models/yi.rs:433:18 15: llm_rust::train::run at ./src/train.rs:69:22 16: llm_rust::main at ./src/main.rs:16:5 17: core::ops::function::FnOnce::call_once at /rustc/cc66ad468955717ab92600c770da8c1601a4ff33/library/core/src/ops/function.rs:250:5

[Narsil]

Could you share what you are running ? I can't reproduce atm.

[arthasyou]

thx Narsil! You can check my llm demo as below https://github.com/arthasyou/llm_rust/tree/yi

[ivarflakstad]

At the very least I can tell you that the MTLBuffer candle is trying to use is null. Meaning it has either not been allocated or deallocated early.

Does the same happen if you run the yi example that is in candle-examples?