liuliu / ccv

C-based/Cached/Core Computer Vision Library, A Modern Computer Vision Library
http://libccv.org
Other
7.07k stars 1.71k forks source link

Metal FlashAttention #241

Closed philipturner closed 1 year ago

philipturner commented 1 year ago

I tried everything possible, but C++ code will not compile with CCV. It keeps getting stuck on the linking phase, where any function implemented in a .mm or .cpp file does not link. I'd rather fix the C++ issues than rewrite in Objective-C, which was succeeded by Swift 10 years ago.

Build output ``` (base) philipturner@m1-max-mbp draw-things % bazel run Apps/DrawThings:CLI --verbose_failures INFO: Invocation ID: d513d98d-cf8d-493c-a70d-4fa24838340a INFO: Analyzed target //Apps/DrawThings:CLI (1 packages loaded, 290 targets configured). INFO: Found 1 target... INFO: From Compiling lib/nnc/mps/ccv_nnc_mps.m: external/ccv/lib/nnc/mps/ccv_nnc_mps.m:437:25: warning: 'optimizationProfile' is deprecated: first deprecated in macOS 14.0 [-Wdeprecated-declarations] compilationDescriptor.optimizationProfile = MPSGraphOptimizationProfilePerformance; ^~~~~~~~~~~~~~~~~~~ MPSGraph will automatically provide the best performance and power efficiency with MPSGraphOptimizationLevel1. /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX14.0.sdk/System/Library/Frameworks/MetalPerformanceShadersGraph.framework/Headers/MPSGraph.h:103:62: note: 'optimizationProfile' has been explicitly marked deprecated here @property (readwrite, nonatomic) MPSGraphOptimizationProfile optimizationProfile ^ 1 warning generated. ERROR: /Users/philipturner/Documents/DrawThings/draw-things/Apps/DrawThings/BUILD:80:13: Linking Apps/DrawThings/CLI failed: (Exit 1): cc_wrapper.sh failed: error executing command (from target //Apps/DrawThings:CLI) (cd /private/var/tmp/_bazel_philipturner/823bc4b5b9713ec4f2d8de0426485f9b/sandbox/darwin-sandbox/3424/execroot/__main__ && \ exec env - \ APPLE_SDK_PLATFORM=MacOSX \ APPLE_SDK_VERSION_OVERRIDE=14.0 \ PATH=/opt/homebrew/opt/llvm@14/bin:/opt/homebrew/opt/llvm/bin:/Users/philipturner/miniforge3/bin:/Users/philipturner/miniforge3/condabin:/Users/philipturner/perl5/bin:/opt/homebrew/bin:/Library/Frameworks/Python.framework/Versions/3.10/bin:/opt/homebrew/bin:/opt/homebrew/sbin:/Library/Frameworks/Python.framework/Versions/3.8/bin:/Library/Frameworks/Python.framework/Versions/3.9/bin:/usr/local/bin:/System/Cryptexes/App/usr/bin:/usr/bin:/bin:/usr/sbin:/sbin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/local/bin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/bin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/appleinternal/bin:/Library/Apple/usr/bin:/Library/TeX/texbin:/usr/local/share/dotnet:~/.dotnet/tools:/usr/local/go/bin:/Users/philipturner/.rvm/bin \ XCODE_VERSION_OVERRIDE=15.0.0.15A5160n \ ZERO_AR_DATE=1 \ external/local_config_cc/cc_wrapper.sh @bazel-out/darwin_arm64-fastbuild/bin/Apps/DrawThings/CLI-2.params) # Configuration: 579382b61398843cb15c81a7965d8f238ed4bdebc19b6673391790d8c99e0e68 # Execution platform: @local_config_platform//:host Use --sandbox_debug to see verbose messages from the sandbox and retain the sandbox build root for debugging ld: warning: Could not find or use auto-linked framework 'CoreAudioTypes' Undefined symbols for architecture arm64: "_ccv_nnc_init_mfa_context", referenced from: ___ccv_nnc_default_mfa_context_block_invoke in libnnc_mps_compat.a(ccv_nnc_mps.o) "_ccv_nnc_mfa_context_supported", referenced from: _ccv_nnc_init_stream_context in libnnc_mps_compat.a(ccv_nnc_mps.o) ld: symbol(s) not found for architecture arm64 clang: error: linker command failed with exit code 1 (use -v to see invocation) Error in child process '/usr/bin/xcrun'. 1 Target //Apps/DrawThings:CLI failed to build INFO: Elapsed time: 0.467s, Critical Path: 0.32s INFO: 69 processes: 66 disk cache hit, 3 internal. FAILED: Build did NOT complete successfully ERROR: Build failed. Not running target ```

ccv_nnc_mfa.hpp contains some skeleton function declarations for GEMM and Attention. These are drafts that show how the functions will eventually be used.

Instead of requiring libMFA to be present at build time, this PR isolates libMFA as much as possible. libMFA is a standalone file with a ridiculous build dependency (Xcode 14.2 exactly) and will likely be hosted on GitHub releases. It also needs to exist at a pre-defined path. I'm not sure whether CCV can allocate a stable path in the file system. This route should be compatible with iOS, where you can set an app ENV VAR pointing to the resources directory.

The error messages use ANSI escape sequences, to blend in with compiler error messages from Clang.

liuliu commented 1 year ago

This is already merged.