Closed mtkgeek closed 1 month ago
Hi @mtkgeek, I just merged #489 which should fix this. As you are using the Python API, can you please follow the instructions to install from source, and try it again?
Oh nice. I'm using the pip version. I tried installing from source but keep running into errors when I run pip install maturin[patchelf]
.
Building wheels for collected packages: patchelf
Building wheel for patchelf (pyproject.toml) ... error
error: subprocess-exited-with-error
× Building wheel for patchelf (pyproject.toml) did not run successfully.
│ exit code: 1
╰─> [9 lines of output]
Traceback (most recent call last):
File "/private/var/folders/7m/2srhc2n972gfvzq9jtj9n7rr0000gn/T/pip-build-env-x0rlsvm9/overlay/lib/python3.10/site-packages/skbuild/setuptools_wrap.py", line 643, in setup
cmkr = cmaker.CMaker(cmake_executable)
File "/private/var/folders/7m/2srhc2n972gfvzq9jtj9n7rr0000gn/T/pip-build-env-x0rlsvm9/overlay/lib/python3.10/site-packages/skbuild/cmaker.py", line 145, in __init__
self.cmake_version = get_cmake_version(self.cmake_executable)
File "/private/var/folders/7m/2srhc2n972gfvzq9jtj9n7rr0000gn/T/pip-build-env-x0rlsvm9/overlay/lib/python3.10/site-packages/skbuild/cmaker.py", line 102, in get_cmake_version
raise SKBuildError(msg) from err
Problem with the CMake installation, aborting build. CMake executable is cmake
[end of output]
note: This error originates from a subprocess, and is likely not a problem with pip.
ERROR: Failed building wheel for patchelf
Failed to build patchelf
ERROR: ERROR: Failed to build installable wheels for some pyproject.toml based projects (patchelf
I guess I'll just wait for these changes to make it to pypi.
@mtkgeek I just released v0.1.23 which includes this fix. Can you please try it again after a pip reinstall?
Thanks for the update @EricLBuehler. Now pip install doesn't work though
ERROR
Compiling png v0.17.13
Compiling candle-core v0.6.0 (https://github.com/EricLBuehler/candle.git?rev=b438cba3f8742519d6f2154209e096c123fc3eb9#b438cba3)
Compiling hyper v1.3.1
error[E0308]: mismatched types
--> /Users/me/.cargo/git/checkouts/candle-c6a149c3b35a488f/b438cba/candle-core/src/quantized/metal.rs:132:39
|
132 | let src = crate::Storage::Cpu(src);
| ------------------- ^^^ expected `CpuStorage`, found `&CpuStorage`
| |
| arguments to this enum variant are incorrect
|
note: tuple variant defined here
--> /Users/me/.cargo/git/checkouts/candle-c6a149c3b35a488f/b438cba/candle-core/src/storage.rs:10:5
|
10 | Cpu(CpuStorage),
| ^^^
help: consider using clone here
|
132 | let src = crate::Storage::Cpu(src.clone());
| ++++++++
error[E0599]: no method named `as_slice` found for enum `Storage` in the current scope
--> /Users/me/.cargo/git/checkouts/candle-c6a149c3b35a488f/b438cba/candle-core/src/quantized/metal.rs:136:36
|
136 | storage.from_float(src.as_slice::<f32>()?)?;
| ^^^^^^^^ method not found in `Storage`
|
::: /Users/me/.cargo/git/checkouts/candle-c6a149c3b35a488f/b438cba/candle-core/src/storage.rs:9:1
|
9 | pub enum Storage {
| ---------------- method `as_slice` not found for this enum
|
note: the method `as_slice` exists on the type `CpuStorage`
--> /Users/me/.cargo/git/checkouts/candle-c6a149c3b35a488f/b438cba/candle-core/src/cpu_backend/mod.rs:1541:5
|
1541 | pub fn as_slice<D: WithDType>(&self) -> Result<&[D]> {
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Compiling signal-hook-mio v0.2.3
Compiling derive_builder_macro v0.20.0
Compiling darling_macro v0.11.0
Some errors have detailed explanations: E0308, E0599.
For more information about an error, try `rustc --explain E0308`.
Compiling itertools v0.11.0
error: could not compile `candle-core` (lib) due to 2 previous errors
warning: build failed, waiting for other jobs to finish...
💥 maturin failed
Caused by: Failed to build a native library through cargo
Caused by: Cargo build finished with "exit status: 101"
@mtkgeek thank you for reporting this, can you please try it again after you update via pip - it should be fixed?
@EricLBuehler the new version installs but the original problem still persists. PanicException: Need preprocessor config
when I run the code.
@EricLBuehler the new version installs but the original problem still persists.
PanicException: Need preprocessor config
when I run the code.
Can you please paste the output of ls /Users/path/to/my/phi3v_model
?
Sure @EricLBuehler . It's
config.json model.safetensors.index.json
model-00001-of-00002.safetensors tokenizer.json
model-00002-of-00002.safetensors tokenizer_config.json
Ah ok! I think I must have forgotten to update the docs. As documented here now, if you add preprocessor_config.json
this should work.
Thanks, it works now. I also wanted to point out that I can see Loading model
/Users/path/to/my/phi3v_modelon cpu.
is this normal, I was expecting "on metal" or "on mps" ?
Ok, that is good that it works. Regarding the device, can you close this issue and open another for that?
Hi, I'm trying to run the phi3-vision model using local files. I followed the instructions but ran into this error
panicked at mistralrs-core/src/pipeline/vision.rs:192:22: Need preprocessor config
Even though the instructions do not say a
preprocessor config
file is needed, I added that as well but still ran into the same error. I'm on a Mac btwMY CODE