Duplicate symbols (filename case sensitivity issue with libsdl )

[brentpicasso]

Versions

• Kivy: 2.3.0
• Python : 3.11
• MacOS version :
• XCode Version :
• Cython version :

I'm seeing duplicate symbols for libsdl libraries. This seems to be due to differences in case for libsdl files.

• See attached output of xcode build.

• Example from build output:

  /Users/macmini/Library/Developer/Xcode/DerivedData/racecaptureapp-gvoxmqndeqrfaxcpilbfwwppcdqu/Build/Products/Debug-iphoneos/libsdl2_mixer.a(music_drmp3.o)
  /Users/macmini/Library/Developer/Xcode/DerivedData/racecaptureapp-gvoxmqndeqrfaxcpilbfwwppcdqu/Build/Products/Debug-iphoneos/libSDL2_mixer.a(music_drmp3.o)

  My initial questions

• Which variant of these is correct?

• Suggestions on how to fix? I'm happy to submit a PR. :+1:

libsdl_duplicate_symbols.txt

Steps I did to observe this:

I did a complete rebuild of the toolchain (deleted build and dist) and re-installed with this command:

python3 toolchain.py build python3 kivy==2.3.0 pillow ffmpeg ffpyplayer

• Additional context: I'm migrating a project from kivy 2.2.1 to kivy 2.3.0
• I'm on the latest of kivy-ios
• Output of toolchain status:

Brents-Mac-mini:kivy-ios macmini$ python3 toolchain.py status
audiostream  - Not built
click        - Not built
curly        - Not built
cymunk       - Not built
ffmpeg       - Build OK (built at 2024-02-07 16:18:21.205165)
ffpyplayer   - Build OK (built at 2024-02-07 16:19:05.286216)
flask        - Not built
freetype     - Build OK (built at 2024-02-07 16:13:27.340200)
hostopenssl  - Build OK (built at 2024-02-07 15:57:33.440418)
hostpython3  - Build OK (built at 2024-02-07 16:03:27.748946)
ios          - Build OK (built at 2024-02-07 16:09:54.205482)
itsdangerous - Not built
jinja2       - Not built
kivent_core  - Not built
kivy         - Build OK (built at 2024-02-07 16:12:21.744860)
kiwisolver   - Not built
libcurl      - Not built
libffi       - Build OK (built at 2024-02-07 15:59:42.695370)
libjpeg      - Build OK (built at 2024-02-07 16:13:45.282976)
libpng       - Build OK (built at 2024-02-07 16:08:31.277707)
libzbar      - Not built
markupsafe   - Not built
matplotlib   - Not built
netifaces    - Not built
numpy        - Not built
openssl      - Build OK (built at 2024-02-07 16:00:49.600563)
photolibrary - Not built
pillow       - Build OK (built at 2024-02-07 16:14:23.302231)
plyer        - Not built
py3dns       - Not built
pycrypto     - Not built
pykka        - Not built
pyobjus      - Build OK (built at 2024-02-07 16:10:04.599033)
python3      - Build OK (built at 2024-02-07 16:07:31.367719)
pyyaml       - Not built
sdl2         - Build OK (built at 2024-02-07 16:08:45.134009)
sdl2_image   - Build OK (built at 2024-02-07 16:08:54.650620)
sdl2_mixer   - Build OK (built at 2024-02-07 16:09:04.534253)
sdl2_ttf     - Build OK (built at 2024-02-07 16:09:44.320951)
werkzeug     - Not built
zbarlight    - Not built
Brents-Mac-mini:kivy-ios macmini$ python3 toolchain.py status
audiostream  - Not built
click        - Not built
curly        - Not built
cymunk       - Not built
ffmpeg       - Build OK (built at 2024-02-07 16:18:21.205165)
ffpyplayer   - Build OK (built at 2024-02-07 16:19:05.286216)
flask        - Not built
freetype     - Build OK (built at 2024-02-07 16:13:27.340200)
hostopenssl  - Build OK (built at 2024-02-07 15:57:33.440418)
hostpython3  - Build OK (built at 2024-02-07 16:03:27.748946)
ios          - Build OK (built at 2024-02-07 16:09:54.205482)
itsdangerous - Not built
jinja2       - Not built
kivent_core  - Not built
kivy         - Build OK (built at 2024-02-07 16:12:21.744860)
kiwisolver   - Not built
libcurl      - Not built
libffi       - Build OK (built at 2024-02-07 15:59:42.695370)
libjpeg      - Build OK (built at 2024-02-07 16:13:45.282976)
libpng       - Build OK (built at 2024-02-07 16:08:31.277707)
libzbar      - Not built
markupsafe   - Not built
matplotlib   - Not built
netifaces    - Not built
numpy        - Not built
openssl      - Build OK (built at 2024-02-07 16:00:49.600563)
photolibrary - Not built
pillow       - Build OK (built at 2024-02-07 16:14:23.302231)
plyer        - Not built
py3dns       - Not built
pycrypto     - Not built
pykka        - Not built
pyobjus      - Build OK (built at 2024-02-07 16:10:04.599033)
python3      - Build OK (built at 2024-02-07 16:07:31.367719)
pyyaml       - Not built
sdl2         - Build OK (built at 2024-02-07 16:08:45.134009)
sdl2_image   - Build OK (built at 2024-02-07 16:08:54.650620)
sdl2_mixer   - Build OK (built at 2024-02-07 16:09:04.534253)
sdl2_ttf     - Build OK (built at 2024-02-07 16:09:44.320951)
werkzeug     - Not built
zbarlight    - Not built

[brentpicasso]

I checked my android buildozer build and it appears the uppercase variant is used there. Does that mean the uppercase variant is the "correct" version?

libSDL2*.so

[tcaduser]

It looks like this issue has come up before: https://github.com/kivy/buildozer/issues/566

If case insensitivity still the default for macOS, it might be easiest to move to a case insensitive disk.

[tcaduser]

In the recipe, it is mixed case: https://github.com/kivy/kivy-ios/blob/master/kivy_ios/recipes/sdl2/__init__.py#L8

so there is probably a string conversion going on somewhere in either kivy-ios or the rest of the build system (xcode)

[brentpicasso]

I checked my mac, and it is current set to case insensitive

Brents-Mac-mini:kivy-ios macmini$ nano blah.txt
Brents-Mac-mini:kivy-ios macmini$ cat blah.txt 
asdf
Brents-Mac-mini:kivy-ios macmini$ cat BLAH.txt
asdf

@tcaduser thanks. I see in that link the SDL2 is upper case, which I think is the correct form. So we need to find out what thing is causing the lower case form?

[tcaduser]

If your filesystem is case insensitive, it means that the same exact file is being referenced twice.

Looking on your linker line, I see the same libraries being listed multiple times:

-lsdl2
-lSDL2
-lsdl2_image
-lSDL2_image
-lsdl2_mixer
-lSDL2_mixer
-lsdl2_ttf
-lSDL2_ttf

ordinarily a linker shouldn't care if the same filename is listed multiple times, so it may mean that the case difference is confusing clang:

    /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang 

so you would need to find the recipe referencing the libraries with different case sensitivity.

[brentpicasso]

So you mean grepping this repository's files for a lower case form of lsdl2? We can do that.

[tcaduser]

what I am seeing is that the recipes are referencing themselves with the mixed case name:

jsanchez@jslaptop:~/git/kivy-ios/kivy_ios$ grep -s -i   SDL2 */*/*    | grep libr
recipes/sdl2_image/__init__.py:    library = "Xcode/build/Release-{plat.sdk}/libSDL2_image.a"
recipes/sdl2/__init__.py:    library = "Xcode/SDL/build/Release-{plat.sdk}/libSDL2.a"
recipes/sdl2_mixer/__init__.py:    library = "Xcode/build/Release-{plat.sdk}/libSDL2_mixer.a"
recipes/sdl2_ttf/__init__.py:    library = "Xcode/build/Release-{plat.sdk}/libSDL2_ttf.a"

and the dependent packages are using the case insensitive name:

recipes/audiostream/__init__.py:    depends = ["python", "sdl2", "sdl2_mixer"]
recipes/curly/__init__.py:    depends = ["python", "libcurl", "sdl2", "sdl2_image"]
recipes/ffpyplayer/__init__.py:    depends = ["python", "sdl2", "ffmpeg"]
recipes/kivy/__init__.py:    depends = ["sdl2", "sdl2_image", "sdl2_mixer", "sdl2_ttf", "ios",
recipes/sdl2_image/__init__.py:    depends = ["sdl2"]
recipes/sdl2_mixer/__init__.py:    depends = ["sdl2"]
recipes/sdl2_ttf/__init__.py:    depends = ["libpng", "sdl2"]

[tcaduser]

Since I've never seen this problem before, but I haven't build a package in a while, I am guessing that either

Your version of XCode is too new

or

Your version of XCode is too old

[brentpicasso]

Thanks, I'll check the xcode version.

When you say "mixed cased name" - do you mean the uppercase form (SDL2) and the case insensitive name is the lower case form (sdl2) ?

[tcaduser]

also note that the linker is being passed: -Xlinker -no_deduplicate

perhaps from XCode itself.

     -no_deduplicate
                 Don't run deduplication pass in linker

my hypothesis, based on: https://forums.developer.apple.com/forums/thread/44905

is that XCode may have a bug in setting this flag.

Are you doing the final compile? It looks like you are doing a Debug build. Can you choose a Release build in XCode, and see if that flag magically goes away.

Please note that a flag like this is to prevent symbol clashes and resulting in bad behavior.

[brentpicasso]

Checked two things:

• I'm running Xcode 14.3. Latest is 15 (released Sept 2023)
• I tried running the archive command (which I presumably does a release build) - and got the same error.

I can try upgrading to 15.

[brentpicasso]

In order to go to Xcode 15 I first have to upgrade to MacOS 13.5.

Perspective: my builds were previously running correctly on kivy 2.2.1 - the problems only started when I upgraded to kivy 2.3.0 and this latest kivy-ios,

XCode was the same (14.3) under the previous setup.

Given this, could it be possible something in kivy-ios was regressed?

[tcaduser]

I don't know. The owners of this project are doing a lot of changes. I would look into your XCode build configuration and see if you can remove the no_deduplicate flag manually. Release and Debug flags are things that you can control in gui

[tcaduser]

or you can see if the owners of this project would be willing to filter out duplicated library names when generating the xcode project

[brentpicasso]

Thanks for all of the help! I will check those and get back. Hopefully the project maintainers chime in.

[misl6]

@brentpicasso can you please share a screenshot of "Frameworks, Libraries, and Embedded content" from XCode?

Since we moved from libname.a to libname.xcframework something may gone wrong during your Xcode project migration.

[brentpicasso]

@misl6 Here you go:

[misl6]

That's why.

The libname.a files should not be there (even if still available under dist/lib/**/*)

Can you try to remove them?

That section should look like that after the changes:

[brentpicasso]

@misl6 thank you for the help>

I'm not following - in my screenshots I don't see libname.a listed there. Where are you seeing it?

[brentpicasso]

I searched my project.pbxproj and there are no references to the string libname in there...

[tcaduser]

You can do this in the gui for XCode. Remove all of the names with the ".a" extension from the assets list, and then try to rebuild. This should then edit the pbxproj for you

On Thu, Feb 8, 2024 at 12:40 PM Brent Picasso @.***> wrote:

I searched my project.pbxproj and there are no references to the string libname in there...

— Reply to this email directly, view it on GitHub https://github.com/kivy/kivy-ios/issues/893#issuecomment-1934724791, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAS6BJMZPMPFQXDUH5D4V53YSUL2HAVCNFSM6AAAAABC6NKPMGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSMZUG4ZDINZZGE . You are receiving this because you were mentioned.Message ID: @.***>

[brentpicasso]

Oh, so not specifically "libname.a" - but remove anything with a ".a" extension from the gui.

I can try that. Thank you.

[tcaduser]

I think so. The .xcframework directories are the new way. Having bare .a files is no longer preferred method.

[brentpicasso]

I think I got it, thanks for the help!

[brentpicasso]

@misl6 is there a migration guide this info could be added to?