Hey folks, thanks for making this library! I'm looking forward to using it in my own code. When I try to use apple metal, inference moves quickly once the model is loaded, but the load times are very long. Any idea what's going on? I'm on an M1 macbook pro with the M1 pro. Thanks!
Hey folks, thanks for making this library! I'm looking forward to using it in my own code. When I try to use apple metal, inference moves quickly once the model is loaded, but the load times are very long. Any idea what's going on? I'm on an M1 macbook pro with the M1 pro. Thanks!
This function uses the
os.walk()
function to recursively traverse the directory tree rooted at the given path. It then iterates through each subdirectory and its contents, printing out any files found. The function can be customized to filter or limit the output as needed. llama_print_timings: load time = 276625.80 ms llama_print_timings: sample time = 78.89 ms / 112 runs ( 0.70 ms per token) llama_print_timings: prompt eval time = 2083.72 ms / 9 tokens ( 231.52 ms per token) llama_print_timings: eval time = 5143.88 ms / 111 runs ( 46.34 ms per token) llama_print_timings: total time = 7317.77 ms Embeddings: [0.3125355 -0.49622384 0.8355821 -5.7579308 0.7423292 0.9061695 1.0729696 0.55204344 -0.4947705 0.76542914 -0.08652469 -0.92549866 0.56039613 1.3558147 1.5783771 2.0912528 0.83712846 2.230342 1.7294012 0.91197115 2.3878574 0.17538485 1.0380409 -1.9442692 -0.75153935 -0.46291435 -1.0816864 -1.6969684 2.5978549 0.07971512 2.4507911 -0.43762016 -1.121216 3.2154222 -1.623437 2.3818705 -0.54442763 -0.036091648 0.44333118 1.4332048 -2.0323622 -4.9638834 -0.10573294 -0.6578927 -2.525819 -1.7579216 -0.082311146 -1.282202 -0.3188231 -0.30106577 0.18109736 -1.5109167 -0.46574533 -0.7783518 1.4447253 -0.07444402 0.92188036 -1.6830924 0.9693521 0.6487904 0.00760857 -1.8059875 0.028990952 5.1491437 0.30388048 0.8748815 1.0947195 -0.81968737 2.2706778 -2.470005 -1.365771 1.4964849 0.9554498 0.9874882 -0.21673784 -0.9545497 2.3141744 -0.13471882 -0.72715926 3.1383698 1.1182431 -0.19498359 1.4391023 -1.9303225 0.10249053 0.14131173 -0.8227461 -0.73139864 0.05322086 -0.9561012 0.12730123 2.825649 -1.9981018 0.1599991 0.60884464 -0.4810178 -1.1864207 0.40985224 0.018653302 -0.80309135 1.0827796 -0.9229907 2.0702777 -0.97203946 -2.9991443 -1.6874504 1.6150079 -0.31088957 -2.1397588 -0.3066519 0.3554739 1.0634847 -0.5357307 -0.30370006 -0.3489394 0.13483998 0.28904593 -0.11815561 0.8544113 -4.3085527 0.82415545 0.07161723 -0.1693164 1.4458497 0.8364489 3.3607686 2.5930157 -1.0991863]