search

karpathy / llama2.c

Inference Llama 2 in one file of pure C
MIT License
16.92k stars 1.99k forks source link

Can this be compiled to run on Windows 10, or Windows XP? #516

Open MartialTerran opened 2 months ago

MartialTerran commented 2 months ago

Assuming that a Windows 10 machine can run this code, can this run.c lllama2 be modified to be complied to run inference mode on a Windows 10 machine? I asked Google Gemini 1.5 Pro this question and this is Gemini's response:

Compiling and Running the Code on Windows 10 While the provided code appears to be primarily written in C, it includes some POSIX-specific elements like mmap and munmap which are not directly available on Windows 10. Therefore, running this code directly on Windows 10 would require some modifications and additional libraries. Here's a potential approach:

  1. Replacing POSIX Functions: Memory Mapping: Use the Windows API functions CreateFileMapping, MapViewOfFile, and UnmapViewOfFile to achieve similar functionality as mmap and munmap. Libraries like Boost.Interprocess could also provide cross-platform memory mapping solutions. Other Functions: Review the code for other potential POSIX-specific functions and find equivalent Windows API functions or libraries.
  2. Compiler and Libraries: Compiler: Use a C compiler compatible with Windows, such as Microsoft Visual C++, MinGW, or Clang. Libraries: Ensure any required libraries (e.g., for linear algebra operations) have Windows versions or alternatives.
  3. Code Structure: Conditional Compilation: The code already utilizes #if defined _WIN32 blocks. Adapt these sections to incorporate the necessary Windows-specific implementations. Consider creating separate source files for platform-specific code to maintain clarity.
  4. Building and Running: Use the chosen compiler and build system to compile the code into an executable. Run the executable from the command line, providing the required arguments (checkpoint path, options). Additional Considerations: OpenMP: The code utilizes OpenMP for parallelization. Ensure you have a compatible OpenMP library and compiler flags set up for Windows. Testing: Thoroughly test the modified code on Windows 10 to ensure functionality and performance. Tools and Libraries: Microsoft Visual Studio MinGW Clang Boost C++ Libraries OpenMP library for Windows Disclaimer: Modifying the code to run on Windows 10 requires some C programming and Windows API knowledge. This response provides a general direction, and specific implementation details may vary.
jameswdelancey commented 2 months ago

lol. yea absolutely. there is a small fix you need to compile it to run models over 4gig, like llama 7b. Just compile it with ming64 and run it first.

jameswdelancey commented 2 months ago

https://github.com/karpathy/llama2.c/pull/513

MartialTerran commented 2 months ago

Hi. According to Google Gemini Pro, compiling a Llama2.c model on a Windows 10 machine is "difficult" also because of high RAM memory requirements. And "Dependency Hell" due to "libraries optimized for Linux". I apprehend that the windows-compile task is beyond my experience and practical ability. Can Windows run the Linux version of Llama2.c within a Windows builtin Linux environment "MingW64 ~" prompt? Is that the same method as what you originally proposed? Can you point to simple instructions printed online?

On Sat, May 11, 2024, 12:03 AM James Delancey @.***> wrote:

lol. yea absolutely. there is a small fix you need to compile it to run models over 4gig, like llama 7b. Just compile it with ming64 and run it first.

— Reply to this email directly, view it on GitHub https://github.com/karpathy/llama2.c/issues/516#issuecomment-2105527029, or unsubscribe https://github.com/notifications/unsubscribe-auth/BHT6KT2R3CQSZGJRQKGVUEDZBWKAPAVCNFSM6AAAAABHJXVADWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCMBVGUZDOMBSHE . You are receiving this because you authored the thread.Message ID: @.***>