tensorflow / tensorflow

An Open Source Machine Learning Framework for Everyone
https://tensorflow.org
Apache License 2.0
186.09k stars 74.27k forks source link

Unable to Force-load TensorFlowLiteSelectTfOps.framework, created with Selective Build, in iOS #67790

Open tanpengshi opened 5 months ago

tanpengshi commented 5 months ago

IDE: Xcode 15 Platform: iOS17 TensorFlow version: r2.9

I am developing both iOS and Android apps that are running with TensorFlow Lite model. Because my model uses LSTM, I have to make use of TFSelectOps.

In addition, because the TensorFlowLiteSelectTFOps library is large in memory size, I have to do a selective build. After much effort, I have succeeded in making my TensorFlowLite model running smoothly on the Android app based on the selectively built libraries.

On the iOS however, I used a bazel build:

bash tensorflow/lite/ios/build_frameworks.sh \ --input_models=model1.tflite,model2.tflite \ --target_archs=x86_64,armv7,arm64

to generate the:

  1. TensorFlowLiteSelectTfOps.framework
  2. TensorFlowLiteC.framework

After that I edited the TensorFlowLiteSwift.podspec to include the framework:

Pod::Spec.new do |s|
  s.name             = 'TensorFlowLiteSwift'
  s.version          = '2.14.0'
  s.authors          = 'Google Inc.'
  s.license          = { :type => 'Apache' }
  s.homepage         = 'https://github.com/tensorflow/tensorflow'
  s.source           = { :git => 'https://github.com/tensorflow/tensorflow.git', :tag => "v#{s.version}" }
  s.summary          = 'TensorFlow Lite for Swift'
  s.description      = <<-DESC

  TensorFlow Lite is TensorFlow's lightweight solution for Swift developers. It
  enables low-latency inference of on-device machine learning models with a
  small binary size and fast performance supporting hardware acceleration.
                       DESC

  s.ios.deployment_target = '17.0'

  s.module_name = 'TensorFlowLite'
  s.static_framework = true

  tfl_dir = 'tensorflow/lite/'
  swift_dir = tfl_dir + 'swift/'

  s.default_subspec = 'Core'

  s.subspec 'Core' do |core|
    # Adjust the path to point to your custom frameworks
    core.vendored_frameworks = [
      'frameworks/TensorFlowLiteC.framework',
      'frameworks/TensorFlowLiteSelectTfOps.framework'
    ]

    core.source_files = swift_dir + 'Sources/*.swift'
    core.exclude_files = swift_dir + 'Sources/{CoreML,Metal}Delegate.swift'

    core.test_spec 'Tests' do |ts|
      ts.source_files = swift_dir + 'Tests/*.swift'
      ts.exclude_files = swift_dir + 'Tests/MetalDelegateTests.swift'
      ts.resources = [
        tfl_dir + 'testdata/add.bin',
        tfl_dir + 'testdata/add_quantized.bin',
      ]
    end
  end

  s.subspec 'CoreML' do |coreml|
    coreml.source_files = swift_dir + 'Sources/CoreMLDelegate.swift'
    coreml.dependency 'TensorFlowLiteSwift/Core', "#{s.version}"
  end

  s.subspec 'Metal' do |metal|
    metal.source_files = swift_dir + 'Sources/MetalDelegate.swift'
    metal.dependency 'TensorFlowLiteSwift/Core', "#{s.version}"

    metal.test_spec 'Tests' do |ts|
      ts.source_files = swift_dir + 'Tests/{Interpreter,MetalDelegate}Tests.swift'
      ts.resources = [
        tfl_dir + 'testdata/add.bin',
        tfl_dir + 'testdata/add_quantized.bin',
        tfl_dir + 'testdata/multi_add.bin',
      ]
    end
  end
end

I then do in the Podfile:

pod 'TensorFlowLiteSwift', :path => '../../local-podspecs/TensorFlowLiteSwift.podspec'

Then in terminal:

pod install

In my Other Linker Flags, i put: -force_load $(SRCROOT)/local-podspecs/frameworks/TensorFlowLiteSelectTfOps.framework/TensorFlowLiteSelectTfOps

But when I do build, i get lots of errors below:

FacialRecognition Undefined symbol: google::protobuf::TextFormat::PrintToString(google::protobuf::Message const&, std::1::basic_string<char, std::__1::char_traits, std::1::allocator>*)

Undefined symbol: google::protobuf::TextFormat::Parse(google::protobuf::io::ZeroCopyInputStream, google::protobuf::Message)

Undefined symbol: google::protobuf::DoubleValue::MergePartialFromCodedStream(google::protobuf::io::CodedInputStream*)

Undefined symbol: google::protobuf::DoubleValue::MergeFrom(google::protobuf::DoubleValue const&)

Undefined symbol: google::protobuf::DoubleValue::DoubleValue(google::protobuf::DoubleValue const&)

Undefined symbol: google::protobuf::MessageLite::ParseFromArray(void const*, int)

Undefined symbol: google::protobuf::MessageLite::ParseFromString(std::1::basic_string<char, std::__1::char_traits, std::1::allocator> const&)

Undefined symbol: google::protobuf::MessageLite::ParseFromCodedStream(google::protobuf::io::CodedInputStream*)

Undefined symbol: google::protobuf::RepeatedField::Swap(google::protobuf::RepeatedField*)

Undefined symbol: google::protobuf::RepeatedField::Reserve(int)

Undefined symbol: google::protobuf::RepeatedField::~RepeatedField()

Undefined symbol: google::protobuf::RepeatedField::Swap(google::protobuf::RepeatedField*)

Undefined symbol: google::protobuf::RepeatedField::Reserve(int)

Undefined symbol: google::protobuf::RepeatedField::~RepeatedField()

Undefined symbol: google::protobuf::RepeatedField::Swap(google::protobuf::RepeatedField*)

Undefined symbol: google::protobuf::RepeatedField::Reserve(int)

Undefined symbol: google::protobuf::RepeatedField::~RepeatedField()

Undefined symbol: google::protobuf::RepeatedField::Swap(google::protobuf::RepeatedField*)

Undefined symbol: google::protobuf::RepeatedField::Reserve(int)

Undefined symbol: google::protobuf::RepeatedField::~RepeatedField()

Undefined symbol: google::protobuf::RepeatedField::Swap(google::protobuf::RepeatedField*)

Undefined symbol: google::protobuf::RepeatedField::Reserve(int)

Undefined symbol: google::protobuf::RepeatedField::~RepeatedField()

Undefined symbol: google::protobuf::RepeatedField::Swap(google::protobuf::RepeatedField*)

Undefined symbol: google::protobuf::RepeatedField::Reserve(int)

Undefined symbol: google::protobuf::RepeatedField::~RepeatedField()

Undefined symbol: google::protobuf::RepeatedField::Swap(google::protobuf::RepeatedField*)

Undefined symbol: google::protobuf::RepeatedField::Reserve(int)

Undefined symbol: google::protobuf::RepeatedField::~RepeatedField()

Undefined symbol: google::protobuf::FieldDescriptor::TypeOnceInit(google::protobuf::FieldDescriptor const*)

Undefined symbol: google::protobuf::FieldDescriptor::kCppTypeToName

Undefined symbol: google::protobuf::FieldDescriptor::kTypeToCppTypeMap

Undefined symbol: google::protobuf::UnknownFieldSet::ClearFallback()

Undefined symbol: google::protobuf::UnknownFieldSet::MergeFrom(google::protobuf::UnknownFieldSet const&)

Undefined symbol: google::protobuf::RepeatedPtrField<std::1::basic_string<char, std::__1::char_traits, std::1::allocator>>::~RepeatedPtrField()

Undefined symbol: google::protobuf::_Any_defaultinstance

Undefined symbol: google::protobuf::_BoolValue_defaultinstance

Undefined symbol: google::protobuf::io::CodedInputStream::SkipFallback(int, int)

Undefined symbol: google::protobuf::io::CodedInputStream::ReadTagFallback(unsigned int)

Undefined symbol: google::protobuf::io::CodedInputStream::ReadVarint32Fallback(unsigned int)

Undefined symbol: google::protobuf::io::CodedInputStream::ReadVarint64Fallback()

Undefined symbol: google::protobuf::io::CodedInputStream::GetDirectBufferPointer(void const*, int)

Undefined symbol: google::protobuf::io::CodedInputStream::default_recursionlimit

Undefined symbol: google::protobuf::io::CodedInputStream::ReadLittleEndian32Fallback(unsigned int*)

Undefined symbol: google::protobuf::io::CodedInputStream::ReadLittleEndian64Fallback(unsigned long long*)

Undefined symbol: google::protobuf::io::CodedInputStream::ReadVarintSizeAsIntFallback()

Undefined symbol: google::protobuf::io::CodedInputStream::DecrementRecursionDepthAndPopLimit(int)

Undefined symbol: google::protobuf::io::CodedInputStream::IncrementRecursionDepthAndPushLimit(int)

Undefined symbol: google::protobuf::io::CodedInputStream::ReadRaw(void*, int)

Undefined symbol: google::protobuf::io::CodedInputStream::Refresh()

Undefined symbol: google::protobuf::io::CodedInputStream::PopLimit(int)

Undefined symbol: google::protobuf::io::CodedInputStream::PushLimit(int)

Undefined symbol: google::protobuf::io::CodedInputStream::~CodedInputStream()

Undefined symbol: google::protobuf::io::ArrayOutputStream::ArrayOutputStream(void*, int, int)

Undefined symbol: google::protobuf::io::CodedOutputStream::EnableAliasing(bool)

Undefined symbol: google::protobuf::io::CodedOutputStream::WriteVarint32SlowPath(unsigned int)

Undefined symbol: google::protobuf::io::CodedOutputStream::WriteVarint64SlowPath(unsigned long long)

Undefined symbol: google::protobuf::io::CodedOutputStream::WriteStringWithSizeToArray(std::1::basic_string<char, std::__1::char_traits, std::1::allocator> const&, unsigned char*)

Undefined symbol: google::protobuf::io::CodedOutputStream::CodedOutputStream(google::protobuf::io::ZeroCopyOutputStream*)

Undefined symbol: google::protobuf::io::CodedOutputStream::~CodedOutputStream()

Undefined symbol: google::protobuf::io::ZeroCopyOutputStream::WriteAliasedRaw(void const*, int)

Undefined symbol: google::protobuf::_DoubleValue_defaultinstance

Undefined symbol: google::protobuf::Any::MergePartialFromCodedStream(google::protobuf::io::CodedInputStream*)

Undefined symbol: google::protobuf::Any::Clear()

Undefined symbol: google::protobuf::Any::MergeFrom(google::protobuf::Any const&)

Undefined symbol: google::protobuf::Any::Any(google::protobuf::Any const&)

Undefined symbol: google::protobuf::DoubleValue google::protobuf::Arena::CreateMaybeMessage(google::protobuf::Arena)

Undefined symbol: google::protobuf::Any google::protobuf::Arena::CreateMaybeMessage(google::protobuf::Arena)

Undefined symbol: google::protobuf::BoolValue google::protobuf::Arena::CreateMaybeMessage(google::protobuf::Arena)

Undefined symbol: google::protobuf::Message::DiscardUnknownFields()

Undefined symbol: google::protobuf::Message::CheckTypeAndMergeFrom(google::protobuf::MessageLite const&)

Undefined symbol: google::protobuf::Message::CopyFrom(google::protobuf::Message const&)

Undefined symbol: google::protobuf::Message::MergeFrom(google::protobuf::Message const&)

Undefined symbol: google::protobuf::internal::LogMessage::LogMessage(google::protobuf::LogLevel, char const*, int)

Undefined symbol: google::protobuf::internal::LogMessage::~LogMessage()

Undefined symbol: google::protobuf::internal::LogMessage::operator<<(char const*)

Undefined symbol: google::protobuf::internal::LogMessage::operator<<(std::1::basic_string<char, std::__1::char_traits, std::1::allocator> const&)

Undefined symbol: google::protobuf::internal::LogMessage::operator<<(long long)

Undefined symbol: google::protobuf::internal::NameOfEnum(google::protobuf::EnumDescriptor const*, int)

Undefined symbol: google::protobuf::internal::WireFormat::SerializeUnknownFields(google::protobuf::UnknownFieldSet const&, google::protobuf::io::CodedOutputStream*)

Undefined symbol: google::protobuf::internal::WireFormat::ComputeUnknownFieldsSize(google::protobuf::UnknownFieldSet const&)

Undefined symbol: google::protobuf::internal::WireFormat::SerializeUnknownFieldsToArray(google::protobuf::UnknownFieldSet const&, unsigned char*)

Undefined symbol: google::protobuf::internal::WireFormat::SkipField(google::protobuf::io::CodedInputStream, unsigned int, google::protobuf::UnknownFieldSet)

Undefined symbol: google::protobuf::internal::GenericSwap(google::protobuf::MessageLite, google::protobuf::MessageLite)

Undefined symbol: google::protobuf::internal::InitSCCImpl(google::protobuf::internal::SCCInfoBase*)

Undefined symbol: google::protobuf::internal::LogFinisher::operator=(google::protobuf::internal::LogMessage&)

Undefined symbol: google::protobuf::internal::MapFieldBase::SetMapDirty()

Undefined symbol: google::protobuf::internal::MapFieldBase::~MapFieldBase()

Undefined symbol: google::protobuf::internal::OnShutdownRun(void ()(void const), void const*)

Undefined symbol: google::protobuf::internal::ReflectionOps::Merge(google::protobuf::Message const&, google::protobuf::Message*)

Undefined symbol: google::protobuf::internal::VerifyVersion(int, int, char const*)

Undefined symbol: google::protobuf::internal::AddDescriptors(google::protobuf::internal::DescriptorTable const*)

Undefined symbol: google::protobuf::internal::DestroyMessage(void const*)

Undefined symbol: google::protobuf::internal::WireFormatLite::UInt32Size(google::protobuf::RepeatedField const&)

Undefined symbol: google::protobuf::internal::WireFormatLite::UInt64Size(google::protobuf::RepeatedField const&)

Undefined symbol: google::protobuf::internal::WireFormatLite::WriteBytes(int, std::1::basic_string<char, std::__1::char_traits, std::1::allocator> const&, google::protobuf::io::CodedOutputStream*)

Undefined symbol: google::protobuf::internal::WireFormatLite::WriteFloat(int, float, google::protobuf::io::CodedOutputStream*)

Undefined symbol: google::protobuf::internal::WireFormatLite::WriteInt32(int, int, google::protobuf::io::CodedOutputStream*)

Undefined symbol: google::protobuf::internal::WireFormatLite::WriteInt64(int, long long, google::protobuf::io::CodedOutputStream*)

Undefined symbol: google::protobuf::internal::WireFormatLite::WriteDouble(int, double, google::protobuf::io::CodedOutputStream*)

Linker command failed with exit code 1 (use -v to see invocation)

tanpengshi commented 5 months ago

I would appreciate any working solution to this, as long i can run my model on Select Ops based on a TensorFlowLiteSelectTFOps library selectively built based on the tflite model itself. Thanks a lot in advance! :)

pkgoogle commented 4 months ago

Hi @tanpengshi, it looks like the system is unable to properly link the google protobuf library ... can you ensure you are following all the directions here correctly: https://www.tensorflow.org/lite/guide/build_ios#using_local_swift_or_objective-c_apis

particularly I see that your path does not point to the TF root directory

pod 'TensorFlowLiteSwift', :path => '../../local-podspecs/TensorFlowLiteSwift.podspec'

Where as the source says:

pod 'TensorFlowLiteSwift', :path => '<your_tensorflow_root_dir>'

Let me know if that works. Thanks.

tanpengshi commented 4 months ago

I have tried the above and I am still getting the same error. In addition I did :

  pod 'TensorFlowLiteSwift'
  pod 'TensorFlowLiteSelectTfOps', '~> 0.0.1-nightly'

Then replace i replaced the TensorFlowLiteSelectTfOps.framework with my selectively build TensorFlowLiteSelectTfOps.framework, and I still get the same error when building! I suspect the error comes inherently from the selectively built framework. Are you able to replicate the error I get?

Thank you!

pkgoogle commented 4 months ago

Hi @tanpengshi, are you willing to share your model(s) that are used to make your selective build? I'm wondering if any particular ops are causing the issue. Perhaps you can just make a dummy model which includes at least one of all the ops you are using and share that.

tanpengshi commented 4 months ago

tflite_CNN_LSTM_dummy.zip

Hi @pkgoogle, here is the dummy model that contains all the ops!

pkgoogle commented 4 months ago

Hi @tanpengshi, I attempted to reproduce it with your model... I'm actually failing on the build the select framework step -- considering you were able to build the Select framework, can you share the steps that you use to build it?

I did this:

git switch r2.14 # to match your TF version
./configure
<Said yes to configure for iOS>
<copied your model file over>
bash tensorflow/lite/ios/build_frameworks.sh --input_models=tflite_CNN_LSTM_dummy.tflite --target_archs=x86_64,armv7,arm64

my output:

INFO: Analyzed target //tensorflow/lite/ios/tmp:TensorFlowLiteSelectTfOps_framework (473 packages loaded, 39899 targets configured).
INFO: Found 1 target...
ERROR: /Users/xxxxxxxxx/git/tensorflow/tensorflow/BUILD:1153:21: declared output 'tensorflow/libtensorflow_framework.2.dylib' was not created by genrule. This is probably because the genrule actually didn't create this output, or because the output was a directory and the genrule was run remotely (note that only the contents of declared file outputs are copied from genrules run remotely)
ERROR: /Users/xxxxxxxxx/git/tensorflow/tensorflow/BUILD:1153:21: Executing genrule //tensorflow:libtensorflow_framework.2.dylib_sym [for tool] failed: not all outputs were created or valid
realpath: illegal option -- -
usage: realpath [-q] [path ...]
Target //tensorflow/lite/ios/tmp:TensorFlowLiteSelectTfOps_framework failed to build
Use --verbose_failures to see the command lines of failed build steps.
INFO: Elapsed time: 2868.450s, Critical Path: 470.36s
INFO: 7940 processes: 1075 internal, 6865 local.
FAILED: Build did NOT complete successfully
tanpengshi commented 4 months ago

Hi @pkgoogle I made a mistake, the tensorflow version should be r2.16 instead of r.2.14. When I used r2.16, I can successfully build the model, while r2.14 produced similar error.

pkgoogle commented 4 months ago

Hmm I'm now currently running into a different issue... which python version are you using? I'm using an M1 and Xcode = 15.0.1. Any other potential environmental difference you can possibly think of?

ERROR: /Users/xxxxxxxx/git/tensorflow/tensorflow/python/BUILD:650:24: Linking tensorflow/python/_pywrap_tensorflow_internal.so [for tool] failed: (Exit 1): cc_wrapper.sh failed: error executing command (from target //tensorflow/python:_pywrap_tensorflow_internal.so) external/local_config_cc/cc_wrapper.sh @bazel-out/darwin_arm64-opt-exec-50AE0418-ST-c5da4e9d584e/bin/tensorflow/python/_pywrap_tensorflow_internal.so-2.params
ld: building exports trie: duplicate symbol '__ZN40TableStruct_tsl_2fprotobuf_2fdnn_2eproto7offsetsE'
clang: error: linker command failed with exit code 1 (use -v to see invocation)
Error in child process '/usr/bin/xcrun'. 1
Target //tensorflow/lite/ios/tmp:TensorFlowLiteSelectTfOps_framework failed to build
Use --verbose_failures to see the command lines of failed build steps.
INFO: Elapsed time: 4574.330s, Critical Path: 373.41s
INFO: 14670 processes: 407 internal, 14263 local.
tanpengshi commented 4 months ago

My apologies again, if you use TensorFlow r2.16, you will run into the issue later on in Xcode:

The TensorFlow version should instead be r2.9, with the .frameworks bazel installed with Python 3.9 installed with Anaconda with M3 and Xcode 15.3

pkgoogle commented 4 months ago

Hey @tanpengshi, as 2.9 is quite old at this point, I think we need to fix HEAD, going to change this into a build/install issue for the current state.

I tried again in tf-nightly, reproduce steps, on M1 Mac

git switch tf-nightly
./configure
<Said yes to configure for iOS>
<copied your model file over>
bash tensorflow/lite/ios/build_frameworks.sh --input_models=tflite_CNN_LSTM_dummy.tflite --target_archs=x86_64,armv7,arm64

my output:

INFO: Build options --apple_bitcode, --apple_platform_type, --copt, and 2 more have changed, discarding analysis cache.
INFO: Analyzed target //tensorflow/lite/tools:list_flex_ops_no_kernel_main (8 packages loaded, 1387 targets configured).
INFO: Found 1 target...
Target //tensorflow/lite/tools:list_flex_ops_no_kernel_main up-to-date:
  bazel-bin/tensorflow/lite/tools/list_flex_ops_no_kernel_main
INFO: Elapsed time: 9.985s, Critical Path: 5.90s
INFO: 93 processes: 2 internal, 91 local.
INFO: Build completed successfully, 93 total actions
~/git/tensorflow/tensorflow/lite/ios/tmp ~/git/tensorflow
~/git/tensorflow
INFO: Reading 'startup' options from /Users/xxxxxxxxxx/git/tensorflow/.bazelrc: --windows_enable_symlinks
INFO: Options provided by the client:
  Inherited 'common' options: --isatty=1 --terminal_columns=228
INFO: Reading rc options for 'build' from /Users/xxxxxxx/git/tensorflow/.bazelrc:
  Inherited 'common' options: --experimental_repo_remote_exec
INFO: Reading rc options for 'build' from /Users/xxxxxxxxxxx/git/tensorflow/.bazelrc:
  'build' options: --define framework_shared_object=true --define tsl_protobuf_header_only=true --define=use_fast_cpp_protos=true --define=allow_oversize_protos=true --spawn_strategy=standalone -c opt --announce_rc --define=grpc_no_ares=true --noincompatible_remove_legacy_whole_archive --features=-force_no_whole_archive --enable_platform_specific_config --define=with_xla_support=true --config=short_logs --config=v2 --define=no_aws_support=true --define=no_hdfs_support=true --experimental_cc_shared_library --experimental_link_static_libraries_once=false --incompatible_enforce_config_setting_visibility
INFO: Reading rc options for 'build' from /Users/xxxxxxxx/git/tensorflow/.tf_configure.bazelrc:
  'build' options: --action_env PYTHON_BIN_PATH=/Users/xxxxxxxxxxx/miniforge3/envs/67790/bin/python3 --action_env PYTHON_LIB_PATH=/Users/xxxxxxxx/miniforge3/envs/67790/lib/python3.12/site-packages --python_path=/Users/xxxxxxxx/miniforge3/envs/67790/bin/python3
INFO: Found applicable config definition build:short_logs in file /Users/xxxxxxxx/git/tensorflow/.bazelrc: --output_filter=DONT_MATCH_ANYTHING
INFO: Found applicable config definition build:v2 in file /Users/xxxxxxxx/git/tensorflow/.bazelrc: --define=tf_api_version=2 --action_env=TF2_BEHAVIOR=1
INFO: Found applicable config definition build:ios in file /Users/xxxxxxxx/git/tensorflow/.bazelrc: --apple_platform_type=ios --apple_bitcode=embedded --copt=-fembed-bitcode --copt=-Wno-c++11-narrowing --noenable_platform_specific_config --copt=-w --cxxopt=-std=c++17 --host_cxxopt=-std=c++17 --define=with_xla_support=false
INFO: Build options --apple_bitcode, --apple_platform_type, --copt, and 3 more have changed, discarding analysis cache.
INFO: Analyzed target //tensorflow/lite/ios/tmp:TensorFlowLiteSelectTfOps_framework (477 packages loaded, 39860 targets configured).
INFO: Found 1 target...
ERROR: /Users/xxxxxxx/git/tensorflow/tensorflow/python/BUILD:652:24: Linking tensorflow/python/_pywrap_tensorflow_internal.so [for tool] failed: (Exit 1): cc_wrapper.sh failed: error executing command (from target //tensorflow/python:_pywrap_tensorflow_internal.so) external/local_config_cc/cc_wrapper.sh @bazel-out/darwin_arm64-opt-exec-50AE0418-ST-c5da4e9d584e/bin/tensorflow/python/_pywrap_tensorflow_internal.so-2.params
ld: building exports trie: duplicate symbol '__ZN39TableStruct_xla_2fservice_2fhlo_2eproto7offsetsE'
clang: error: linker command failed with exit code 1 (use -v to see invocation)
Error in child process '/usr/bin/xcrun'. 1
Target //tensorflow/lite/ios/tmp:TensorFlowLiteSelectTfOps_framework failed to build
Use --verbose_failures to see the command lines of failed build steps.
INFO: Elapsed time: 4470.638s, Critical Path: 614.03s
INFO: 6872 processes: 111 internal, 6761 local.
FAILED: Build did NOT complete successfully

Hi @yishuangP, can you please take a look? Thanks.