Header-only targets support

richardtop commented 2 years ago

Description

Swift Package Manager doesn't support header-only targets. A header-only target is the target without any sources (.m, .cpp, .c, .mm etc) but having just the headers .hh, .h, etc) While this issue is of limited importance for Swift / Objective-C ecosystem, with the Swift Interop support it's becoming more important.

Expected behavior

Header-only targets are commonly used in C++ world, having a native support for them would allow for better C++ package ecosystem available to be used with Swift.

Example: https://github.com/nlohmann/json - the most popular package for handling JSON in C++

Actual behavior

It's impossible to build a header-only target, an error occurs. To work around this issue, at least a single empty file should be present and declared as a target's source.

Example:

https://github.com/nlohmann/json/pull/2807/files _SwiftPackageManagerFile.cpp is added together with the package manifest in order to make the target build

https://github.com/apple/swift-numerics/blob/main/Sources/_NumericsShims/include/_NumericsShims.h The same approach is used by the swift-numerics package in order to work around this issue.

target 'json' referenced in product 'json' is empty

Other references:

https://forums.swift.org/t/header-only-library-using-swift-package-manager/42700 Swift forums post discussing the same issue

Steps to reproduce

Create a target with only headers. Or use one of the examples listed above
Try to build the target, observe the error
Add a single .m file to the Sources with an import statement, importing the headers
Try to build again and now the build succeeds.

Swift Package Manager version/commit hash

No response

Swift & OS version (output of `swift --version && uname -a`)

swift-driver version: 1.62.1 Apple Swift version 5.7 (swiftlang-5.7.0.123.7 clang-1400.0.29.50) Target: arm64-apple-macosx12.0 Darwin **** 21.5.0 Darwin Kernel Version 21.5.0: Tue Apr 26 21:08:37 PDT 2022; root:xnu-8020.121.3~4/RELEASE_ARM64_T6000 arm64

json_not_working.zip json_working.zip

tomerd commented 2 years ago

cc @abertelrud @neonichu

shengyang998 commented 1 year ago

Any news?

richardtop commented 1 year ago

Partially fixed here: https://github.com/apple/swift-package-manager/pull/6006 Similar issue: https://github.com/apple/swift-package-manager/issues/4806

The problem is still reproduced, but at the linker stage, it's still not feasible to work with the header-only targets, example with the following json package manifest:

// swift-tools-version:5.9
// The swift-tools-version declares the minimum version of Swift required to build this package.

import PackageDescription

let package = Package(
    name: "json",
    platforms: [
        .iOS(.v9), .macOS(.v10_10), .tvOS(.v9), .watchOS(.v2)
    ],
    products: [
        .library(
            name: "json",
            targets: ["json"]),
    ],
    targets: [
        .target(
            name: "json",
            dependencies: [],
            path: ".",
            exclude:
                [
                    "appveyor.yml",
                    "benchmarks",
                    "cmake",
                    "ChangeLog.md",
                    "doc",
                    "include",
                    "test",
                    "third_party",
                    "CMakeLists.txt",
                    "CODE_OF_CONDUCT.md",
                    "LICENSE.MIT",
                    "Makefile",
                    "meson.build",
                    "nlohmann_json.natvis",
                    "README.md",
                    "wsjcpp.yml",
                ],
            sources:
                [
//                    "_SwiftPackageManagerFile.cpp"
                ],
            publicHeadersPath: "./single_include/")
    ],

    cxxLanguageStandard: .cxx11
)

Error:

Build input file cannot be found: '/Users/user/Library/Developer/Xcode/DerivedData/Pragma-byueqyrrcrksgzcqwsdlfcjpwjhr/Build/Products/Debug-iphonesimulator/json.o'. Did you forget to declare this file as an output of a script phase or custom build rule which produces it?

If I remove the sources component completely, I'm getting the following issue:

x-xcode-log://80F109DD-40DD-4B22-9B16-877DE1E67D92 library product 'json' should not contain executable targets (it has 'json')

So the issue is technically "resolved" at the SPM side, but it's still not possible to work with header-only targets in Xcode at all.

aleksproger commented 1 year ago

Another, problem occurs when I try to link with test binary using swift test Seems it cannot link to C++ standard library even when I pass .linkedLibrary("c++")

richardtop commented 1 year ago

C++ Interop Office Hours:

Is it possible to make it work (build) with the SPM only (command line)? If yes, then some workaround can be applied to the SPM and Xcode later
Looks like C++ Interop mode is not enabled, how does flipping that mode on/off affects the result (same result, i.e. getting a failure)
Need to localize the problem. Whether the current problem is related to the Xcode or to the SPM. If we can make the package build somehow with the SPM, then it's an Xcode problem

Notes after testing:

Since the header-only library really needs to be imported to other C++/C targets and not exposed to Swift, I've tried removing the _SwiftPackageManagerFile.mm and swapping the header-only library under the hood of another library. Still getting the same error:

Build input file cannot be found: '/Users/User/Library/Developer/Xcode/DerivedData/AppName-bqorhjgeoaezwzgiazmezxxylzqb/Build/Products/Debug-iphonesimulator/json.o'. Did you forget to declare this file as an output of a script phase or custom build rule which produces it?

So it's definitely a problem in the Xcode linker now.

kkebo commented 10 months ago

swift-mmio has a header-only target. They resolved the issue by .systemLibrary.

https://github.com/apple/swift-mmio/pull/54

richardtop commented 10 months ago

swift-mmio has a header-only target. They resolved the issue by .systemLibrary.

apple/swift-mmio#54

Could you please clarify how does this work?

valentary commented 5 months ago

swift-mmio has a header-only target. They resolved the issue by .systemLibrary. apple/swift-mmio#54

Could you please clarify how does this work?

I'm trying to get Eigen to work https://eigen.tuxfamily.org/index.php?title=Main_Page as we use it a lot, it's also header only, but I'm having no luck.

I tried the systemLibrary approach, it just needs the manual creation of the module.modulemap you see in the PR.

Unfortunately it still fails to build when it get to:

// Does the compiler support C99?
// Need to include <cmath> to make sure _GLIBCXX_USE_C99 gets defined
#include <cmath>

and it can't find cmath

valentary commented 5 months ago

Actually just creating the module.modulemap works as an alternative to creating a .m or .mm file for a target

richardtop commented 5 months ago

Could you please provide an example?

valentary commented 5 months ago

Could you please provide an example?

Sure, but it's not working, I just have the same problem but without having to provide the .m or .mm file.

So for my Eigen example

Directory looks like so (I chopped some stuff out for brevity)

spm-eigen
├── Package.swift
├── Sources
│   └── Eigen
│       ├── include
│       │   └── eigen3
│       │       ├── Eigen
│       │       │   ├── Cholesky.hpp
│       │       │   ├── CholmodSupport.hpp
│       │       │   ├── Core
│       │       │   ├── Dense
│       │       │   ├── Eigen
│       │       │   ├── Eigenvalues
│       │       │   ├── Geometry
│       │       │   ├── Householder
│       │       │   ├── IterativeLinearSolvers
│       │       │   ├── Jacobi
│       │       │   ├── KLUSupport
│       │       │   ├── LU
│       │       │   ├── MetisSupport
│       │       │   ├── OrderingMethods
│       │       │   ├── PaStiXSupport
│       │       │   ├── PardisoSupport
│       │       │   ├── QR
│       │       │   ├── QtAlignedMalloc
│       │       │   ├── SPQRSupport
│       │       │   ├── SVD
│       │       │   ├── Sparse
│       │       │   ├── SparseCholesky
│       │       │   ├── SparseCore
│       │       │   ├── SparseLU
│       │       │   ├── SparseQR
│       │       │   ├── StdDeque
│       │       │   ├── StdList
│       │       │   ├── StdVector
│       │       │   ├── SuperLUSupport
│       │       │   ├── UmfPackSupport
│       │       │   └── src
│       │       │       ├── ....
│       ├── module.modulemap
└── Tests
    └── spm-eigenTests
        └── spm_eigenTests.swift

module.modulemap is simply:

module Eigen {
    header "include/eigen3/Eigen/Eigen"
    export *
}

And package.swift:

// swift-tools-version: 5.10
// The swift-tools-version declares the minimum version of Swift required to build this package.

import PackageDescription

let package = Package(
    name: "spm-eigen",
    products: [
        // Products define the executables and libraries a package produces, making them visible to other packages.
        .library(
            name: "Eigen",
            targets: ["Eigen"]),
    ],
    targets: [
        // Targets are the basic building blocks of a package, defining a module or a test suite.
        // Targets can depend on other targets in this package and products from dependencies.
        .target(
            name: "Eigen",
            cxxSettings: [
                .headerSearchPath("Sources/Eigen/include")
            ],
            swiftSettings: [.interoperabilityMode(.Cxx)],
            linkerSettings: [.linkedLibrary("std")]
        ),
        .testTarget(
            name: "spm-eigenTests",
            dependencies: ["Eigen"]),
    ],
    //cLanguageStandard: .c11
    cxxLanguageStandard: .cxx11
)

richardtop commented 5 months ago

So with your solution you still need to add m/mm file to make it compile? Also what kind of error are you getting, cause it has changed, there have been some tweaks made under the hood in the Xcode/SPM and the error has changed since I've reported the issue.

valentary commented 5 months ago

So with your solution you still need to add m/mm file to make it compile? Also what kind of error are you getting, cause it has changed, there have been some tweaks made under the hood in the Xcode/SPM and the error has changed since I've reported the issue.

Not quite no, I get the same result if I use the module.modulemap or if I create an empty .m or .mm file. I get further than if I don't use either.

If I remove the module.modulemap I get an error during package resolution: target 'Eigen' referenced in product 'Eigen' is empty

But now, with module.modulemap, When I try to run a simple test, I get an error during build when a header file tries to include <cmath> , So it at least detects the header files now, just doesn't seem to find stdlib I think?

valentary commented 5 months ago

<module-includes>:1:9: note: in file included from <module-includes>:1:
#import "spm-eigen/Sources/Eigen/include/eigen3/Eigen/Cholesky.hpp"
        ^
spm-eigen/Sources/Eigen/include/eigen3/Eigen/Cholesky.hpp:11:10: note: in file included from spm-eigen/Sources/Eigen/include/eigen3/Eigen/Cholesky.hpp:11:
#include "Core"
         ^
spm-eigen/Sources/Eigen/include/eigen3/Eigen/Core:19:10: note: in file included from spm-eigen/Sources/Eigen/include/eigen3/Eigen/Core:19:
#include "src/Core/util/Macros.h"
         ^
spm-eigen/Sources/Eigen/include/eigen3/Eigen/src/Core/util/Macros.h:679:10: error: 'cmath' file not found
#include <cmath>
         ^
spm-eigen/Tests/spm-eigenTests/spm_eigenTests.swift:2:18: error: could not build Objective-C module 'Eigen'
@testable import Eigen

valentary commented 4 months ago

I finally got Eigen working. Though I've been through so many iterations, I'm not quite sure what the precise issue was.

I have conan package up all my cpp libraries for ios, and now the Eigen headers it generated, I think I managed to grab a wrong version from somewhere during this process.

I get errors unless I add my own Eigen.cpp file that includes the umbrella header and I add a module.modulemap

I still have not got it to work with a swift test target, which threw me off track for some time, but it now works for my other packaged swift libraries as a depnedency

swiftlang / swift-package-manager