search

ml-explore / mlx

MLX: An array framework for Apple silicon
https://ml-explore.github.io/mlx/
MIT License
14.83k stars 845 forks source link

Change initial memory limits and add memory size to device info #1064

Closed awni closed 2 weeks ago

awni commented 2 weeks ago
  1. Add a memory_size field to the device_info dictionary
  2. Change the allocator memory limits to be more sensible. Previously on devices with large machines the memory limit is set to well over the total physical memory which is not a very sensible default.

LLM generation / transformer training timings did not change on M2 Ultra.