GN build parity: packaging/install

mspang commented 4 years ago

GN build needs support for installation.

1) Libraries should be able to declare their public headers as metadata 2) This metadata must propagate through dependencies 3) Implement an action that packages up static libs & their headers

mspang commented 4 years ago

I am unsure if this is blocking. The integrations I'm involved build from source and don't make extra copies of artifacts or sources along the way.

caipiblack commented 2 years ago

Hi,

I'm also looking for this feature, to deploy the library and headers automatically with a GN rule instead of doing it "by hand" (or external script)

Could someone treat this issue please ?

Is there any blocking point to do this with GN ?

Thanks

stale[bot] commented 1 year ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs.

caipiblack commented 1 year ago

This is just to make sure that the issue will not be closed. This feature is really required for the integration of matter library into external projects.

angelybo commented 1 year ago

Hi, I am also looking for this - integration with external projects requires a lot of manual hand holding. I"ll open a PR for proof of concept when it's working.

stale[bot] commented 1 year ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs.

woody-apple commented 1 year ago

I don't believe this is being targeted any time soon. We already have many external projects using Matter as is. @angelybo or @caipiblack can you please elaborate what is blocking integration?

caipiblack commented 1 year ago

Hello,

I don't know how "external projects" use the Matter stack and I will be curious to know.

--> Did they integrate their code in the Matter build system to build their app ?

Our idea is to be able to use the Matter library like any library. The only missing thing is "install" part that permits to install:

headers in: {prefix}/include
libraries in: {prefix}/lib
pkgconfig files (.pc) in: {prefix}/lib/pkgconfig/

Basically most of the libraries do it, for exemple JsonCPP:

cd /tmp
git clone https://github.com/open-source-parsers/jsoncpp.git
cd jsoncpp/
mkdir build && cd build
cmake -DCMAKE_INSTALL_PREFIX:PATH=/tmp/install-dir ..
make
make install

Let's take a look at the install folder (/tmp/install-dir):

mathieu@mathieu-PC-Portable:/tmp/install-dir$ ls
include  lib
mathieu@mathieu-PC-Portable:/tmp/install-dir$ ls include/
json
mathieu@mathieu-PC-Portable:/tmp/install-dir$ ls include/json/
allocator.h   config.h    json_features.h  reader.h  version.h
assertions.h  forwards.h  json.h           value.h   writer.h
mathieu@mathieu-PC-Portable:/tmp/install-dir$ ls lib/
cmake         libjsoncpp.so        libjsoncpp.so.25  pkgconfig
libjsoncpp.a  libjsoncpp.so.1.9.5  objects-Release
mathieu@mathieu-PC-Portable:/tmp/install-dir$ ls lib/pkgconfig/
jsoncpp.pc

When we have something like this, then we are able to link a software A with a library B "easily" (without having to configure "include directories" that points to another repository and such things)

The .pc is not mandatory but usefull, it helps to specify include dirs, ldflags, cflags (...) required to build a software with the library.

We are integrating our project (and Matter) in OpenWRT and Yocto, in the two cases this is how it works:

We have a "recipe" for our software that describe how to build it and what are the runtime or build dependencies. Matter is defined as a "build dependancy" (dependancy required to build our software)
There is a "recipe" for each dependency that describe the same thing

When we build a software A with yocto or openwrt, the mechanism is able to know that the software require the libraries X,Y or Z, so first it build the libraries and then it install their headers/libs/pc in a folder and then it build the software A with correct cflags/ldflags to find/link headers/libs from this folder

When the "install" thing is included in the build mechanism (Cmake, GN..) of the project, it's perfect.

In our case as there is no "install" mechanism in the Matter library, we have to do it manually.

Attached (matter-recipe.zip) you will find our recipe to build the "Matter" stack in Yocto it is based on the meta-matter from NXP but has been modified because in their example they just build the examples and their recipe cannot be used to provide the matter library and headers.

You will also find the tool that we made to do the "install" part "externally". Our tool is not really good because, to reduce maintenance we add includes from most of the directories, so we probably have useless includes exported.

To resume:

It should be good to implement something in the matter stack that permits to:

Copy all required headers to link with libCHIP.a in {prefix}/include
Copy all required libs in {prefix}/lib
Generate a .pc file with required parameters in {prefix}/lib/pkgconfig

This thing should be integrated in the GN stuff idealy or on external script if it's not possible to do it with GN.

Doing it with GN instead of external script is better because: in the GN stuff, you are supposed to know the include files part of libCHIP.a, so you don't have to set manual folders like my script (avoid maintenance when folders are created/deleted/modified..)

NOTE: The "prefix" (path where to install include and lib folder) should be a parameter, like in the Cmake system "CMAKE_INSTALL_PREFIX"

ericriff commented 8 months ago

I'd also like to see this, it would make this SDK (or at least the core lib) package-able.

gerickson commented 3 months ago

+1 on this. I also mentioned this in my comment in #33521. Recapping that here and echoing @caipiblack's well-expressed comments above:

I am just getting started with Matter myself and have noted some challenges with the build system as well. Questions you (@gaganpydi) might have encountered that I have encountered:

I cannot seem to find a way to specify: "use this C compiler, this C++ compiler, this linker, this archiver, these C flags, these C++ flags, and these linker flags". This is beyond trivial in autotools and straightforward enough in Meson or CMake.

I cannot seem to find a way to stage the build such that I get the binary artifacts and the public header files that define the public APIs for those binary artifacts. For example, under , I see many executables and many *.a's under lib/, but not headers to be found (this relates to @caipiblack's comments above).

I cannot seem to find a way to specify, "I already have the satisfied dependencies for nlassert, nlio, openttread, ot-br-posix, etc. in my project. Build and link against them here, here, here, here, and here.". This is straightforward enough in CMake, autotools, or Meson.

I cannot seem to find a way to specify, "I don't want to use any of the platform definitions contained in the project. I will be supplying these externally. Either trust me and link against weakly-defined stubs or find them here." This is somewhat related to (2) in that if there were a stage-headers or install-headers target / goal, a platform / product integrator could run that first, build his/her platform against that, and then cycle back and build the rest of Matter against that. Building my platform / product integration in the Matter tree is a non-starter.

(which I didn't note in #33521), I cannot seem to find a way to do the moral equivalent of --enable-shared from autotools in which I enable shared versions of the Matter libraries in addition to the static version(s).

My expectation was that the core Matter library was going to build and function more like OpenSSL or some such or even like building an out-of-tree kernel module in Linux. So, far, that hasn't been my experience. However, I can also imagine this is new, uninitiated user error.

gerickson commented 3 months ago

Per this URL, it looks like gn has a public declaration that, at minimum, should tag which headers are public.

I'll try instrumenting a few and see if that gets them propagated in any manner into the build artifacts tree in any way.

gerickson commented 3 months ago

As an experiment, I made the following patch:

$ git diff .
diff --git a/src/messaging/BUILD.gn b/src/messaging/BUILD.gn
index a170d625111a..7a13425741e0 100644
--- a/src/messaging/BUILD.gn
+++ b/src/messaging/BUILD.gn
@@ -48,6 +48,10 @@ source_set("configurations") {
 static_library("messaging") {
   output_name = "libMessagingLayer"

+  public = [
+    "ExchangeContext.h"
+  ]
+
   sources = [
     "ApplicationExchangeDispatch.cpp",
     "ApplicationExchangeDispatch.h",

And then regenerated the project with:

$ gn --root=`pwd`/repo gen ${BuildRoot}/third_party/matter/.build/Makefile/${BuildProduct}/${BuildConfig} --args='is_debug=false'

and the built with:

$ ninja -j8 -C ${BuildRoot}/third_party/matter/.build/Makefile/${BuildProduct}/${BuildConfig}

but did not end up with ExchangeContext.h in _${BuildRoot}/thirdparty/matter/.build/Makefile/${BuildProduct}/${BuildConfig}.

Since they show up in the build artifacts, I thought that maybe CHIPVersion.h or ASN1OID.h might be good templates to follow; however, those are both generated by Python scripts with an explicit action.

So, I remain stumped at how to "stage" public headers for an interface or library.

gerickson commented 3 months ago

I also tried this diff on the assumption that listing the header in sources was implicitly making it private and overriding the public declaration:

diff --git a/src/messaging/BUILD.gn b/src/messaging/BUILD.gn
index a170d625111a..feb72410c250 100644
--- a/src/messaging/BUILD.gn
+++ b/src/messaging/BUILD.gn
@@ -48,6 +48,10 @@ source_set("configurations") {
 static_library("messaging") {
   output_name = "libMessagingLayer"

+  public = [
+    "ExchangeContext.h"
+  ]
+
   sources = [
     "ApplicationExchangeDispatch.cpp",
     "ApplicationExchangeDispatch.h",
@@ -55,7 +59,6 @@ static_library("messaging") {
     "ErrorCategory.cpp",
     "ErrorCategory.h",
     "ExchangeContext.cpp",
-    "ExchangeContext.h",
     "ExchangeDelegate.h",
     "ExchangeHolder.h",
     "ExchangeMessageDispatch.cpp",

Unfortunately, that did not work either.

mspang commented 3 months ago

Strongly recommend including headers in situ from the main source tree since that's all that we're testing.

Although it's possible to distribute a library with a well specified interface and compiled code, the design of the Matter SDK isn't suited to that. Particularly, even the lowest level interfaces depend on build time configuration that is ABI affecting and that that may need tuning not only on a platform basis but even on an application basis. Trying to install as if it were a reusable artifact would be very misleading.

gerickson commented 3 months ago

Acknowledged that there are configuration "dials" for Matter (as there were for OpenWeave before it) that alter the ABI. I think that can be well-documented and -published such that there isn't any ambiguity about the stability of the ABI. That fact certainly makes a strong argument against having a shared library option for the SDK.

That caveated, assuming that I wanted to add the public headers and interfaces (which in the source tree or out of the source tree are still the same) to the set of things that end up in the build artifacts folder, what is the right gn syntax to make that happen given that my experiments above with what seems to be the well-documented public approach didn't work?

In the interest of Matter growth and adoption, I think there exists an opportunity here to make Matter much more easily approachable and much more easily integrated into new products and platforms. I'd be delighted to do the work and make the pull request(s); I just need some guidance on the gn magic to do so.

mspang commented 3 months ago

You can list headers with a script like

gn gen out/host && gn desc out/host '*' --format=json >| matter.json python3 headers.py

import functools
import json

gn_json = json.loads(open("matter.json").read())

def is_header(filename):
    return filename.endswith(".h") or filename.endswith(".hpp")

def public_headers(target):
    if target["public"] != "*":
        return target["public"]
    return filter(is_header, target.get("sources", []))

@functools.cache
def transitive_headers(target):
    target_json = gn_json[target]
    return set(public_headers(target_json)).union(
        *[transitive_headers(public_dep) for public_dep in target_json.get("deps", [])]
    )

for header in sorted(transitive_headers("//src/lib:lib")):
    print(header)

which produces

//out/host/gen/include/CHIPVersion.h
//out/host/gen/include/FirmwareBuildTime.h
//out/host/gen/include/app/AppBuildConfig.h
//out/host/gen/include/app/icd/server/ICDServerBuildConfig.h
//out/host/gen/include/asn1/ASN1OID.h
//out/host/gen/include/ble/BleBuildConfig.h
//out/host/gen/include/core/CHIPBuildConfig.h
//out/host/gen/include/crypto/CryptoBuildConfig.h
//out/host/gen/include/inet/InetBuildConfig.h
//out/host/gen/include/matter/tracing/build_config.h
//out/host/gen/include/platform/CHIPDeviceBuildConfig.h
//out/host/gen/include/platform/Linux/dbus/bluez/DbusBluez.h
//out/host/gen/include/platform/Linux/dbus/openthread/introspect.h
//out/host/gen/include/platform/Linux/dbus/wpa/DBusWpa.h
//out/host/gen/include/platform/Linux/dbus/wpa/DBusWpaBss.h
//out/host/gen/include/platform/Linux/dbus/wpa/DBusWpaInterface.h
//out/host/gen/include/platform/Linux/dbus/wpa/DBusWpaNetwork.h
//out/host/gen/include/setup_payload/CHIPAdditionalDataPayloadBuildConfig.h
//out/host/gen/include/system/SystemBuildConfig.h
//src/access/AccessControl.h
//src/access/AuthMode.h
//src/access/Privilege.h
//src/access/RequestPath.h
//src/access/SubjectDescriptor.h
//src/access/examples/ExampleAccessControlDelegate.h
//src/access/examples/PermissiveAccessControlDelegate.h
//src/app/AppConfig.h
//src/app/AttributeAccessInterface.h
//src/app/AttributeAccessInterfaceCache.h
//src/app/AttributeAccessInterfaceRegistry.h
//src/app/AttributeEncodeState.h
//src/app/AttributePathExpandIterator.h
//src/app/AttributePathParams.h
//src/app/AttributePersistenceProvider.h
//src/app/AttributeReportBuilder.h
//src/app/AttributeValueDecoder.h
//src/app/AttributeValueEncoder.h
//src/app/BufferedReadCallback.h
//src/app/CASEClient.h
//src/app/CASEClientPool.h
//src/app/CASESessionManager.h
//src/app/ChunkedWriteCallback.h
//src/app/ClusterStateCache.h
//src/app/CommandHandler.h
//src/app/CommandHandlerExchangeInterface.h
//src/app/CommandPathParams.h
//src/app/CommandPathRegistry.h
//src/app/CommandResponseHelper.h
//src/app/CommandSender.h
//src/app/ConcreteAttributePath.h
//src/app/ConcreteClusterPath.h
//src/app/ConcreteCommandPath.h
//src/app/ConcreteEventPath.h
//src/app/DataVersionFilter.h
//src/app/DefaultAttributePersistenceProvider.h
//src/app/DeferredAttributePersistenceProvider.h
//src/app/DeviceProxy.h
//src/app/EventHeader.h
//src/app/EventLogging.h
//src/app/EventLoggingDelegate.h
//src/app/EventLoggingTypes.h
//src/app/EventManagement.h
//src/app/EventPathParams.h
//src/app/FailSafeContext.h
//src/app/GenericEventManagementTestEventTriggerHandler.h
//src/app/GlobalAttributes.h
//src/app/InteractionModelDelegatePointers.h
//src/app/InteractionModelEngine.h
//src/app/InteractionModelHelper.h
//src/app/InteractionModelTimeout.h
//src/app/MessageDef/ArrayBuilder.h
//src/app/MessageDef/ArrayParser.h
//src/app/MessageDef/AttributeDataIB.h
//src/app/MessageDef/AttributeDataIBs.h
//src/app/MessageDef/AttributePathIB.h
//src/app/MessageDef/AttributePathIBs.h
//src/app/MessageDef/AttributeReportIB.h
//src/app/MessageDef/AttributeReportIBs.h
//src/app/MessageDef/AttributeStatusIB.h
//src/app/MessageDef/AttributeStatusIBs.h
//src/app/MessageDef/Builder.h
//src/app/MessageDef/ClusterPathIB.h
//src/app/MessageDef/CommandDataIB.h
//src/app/MessageDef/CommandPathIB.h
//src/app/MessageDef/CommandStatusIB.h
//src/app/MessageDef/DataVersionFilterIB.h
//src/app/MessageDef/DataVersionFilterIBs.h
//src/app/MessageDef/EventDataIB.h
//src/app/MessageDef/EventFilterIB.h
//src/app/MessageDef/EventFilterIBs.h
//src/app/MessageDef/EventPathIB.h
//src/app/MessageDef/EventPathIBs.h
//src/app/MessageDef/EventReportIB.h
//src/app/MessageDef/EventReportIBs.h
//src/app/MessageDef/EventStatusIB.h
//src/app/MessageDef/InvokeRequestMessage.h
//src/app/MessageDef/InvokeRequests.h
//src/app/MessageDef/InvokeResponseIB.h
//src/app/MessageDef/InvokeResponseIBs.h
//src/app/MessageDef/InvokeResponseMessage.h
//src/app/MessageDef/ListBuilder.h
//src/app/MessageDef/ListParser.h
//src/app/MessageDef/MessageBuilder.h
//src/app/MessageDef/MessageDefHelper.h
//src/app/MessageDef/MessageParser.h
//src/app/MessageDef/Parser.h
//src/app/MessageDef/ReadRequestMessage.h
//src/app/MessageDef/ReportDataMessage.h
//src/app/MessageDef/StatusIB.h
//src/app/MessageDef/StatusResponseMessage.h
//src/app/MessageDef/StructBuilder.h
//src/app/MessageDef/StructParser.h
//src/app/MessageDef/SubscribeRequestMessage.h
//src/app/MessageDef/SubscribeResponseMessage.h
//src/app/MessageDef/TimedRequestMessage.h
//src/app/MessageDef/WriteRequestMessage.h
//src/app/MessageDef/WriteResponseMessage.h
//src/app/OTAUserConsentCommon.h
//src/app/OperationalSessionSetup.h
//src/app/OperationalSessionSetupPool.h
//src/app/PendingResponseTracker.h
//src/app/PendingResponseTrackerImpl.h
//src/app/ReadClient.h
//src/app/ReadPrepareParams.h
//src/app/RequiredPrivilege.h
//src/app/SafeAttributePersistenceProvider.h
//src/app/SimpleSubscriptionResumptionStorage.h
//src/app/SpecificationDefinedRevisions.h
//src/app/StatusResponse.h
//src/app/SubscriptionResumptionSessionEstablisher.h
//src/app/SubscriptionResumptionStorage.h
//src/app/SubscriptionsInfoProvider.h
//src/app/TestEventTriggerDelegate.h
//src/app/TimedHandler.h
//src/app/TimedRequest.h
//src/app/TimerDelegates.h
//src/app/WriteClient.h
//src/app/common/CompatEnumNames.h
//src/app/data-model/BasicTypes.h
//src/app/data-model/DecodableList.h
//src/app/data-model/Decode.h
//src/app/data-model/EncodableToTLV.h
//src/app/data-model/Encode.h
//src/app/data-model/FabricScoped.h
//src/app/data-model/FabricScopedPreEncodedValue.h
//src/app/data-model/List.h
//src/app/data-model/NullObject.h
//src/app/data-model/Nullable.h
//src/app/data-model/PreEncodedValue.h
//src/app/data-model/WrappedStructEncoder.h
//src/app/icd/server/ICDConfigurationData.h
//src/app/icd/server/ICDManager.h
//src/app/icd/server/ICDNotifier.h
//src/app/icd/server/ICDServerConfig.h
//src/app/icd/server/ICDStateObserver.h
//src/app/reporting/Engine.h
//src/app/reporting/ReportScheduler.h
//src/app/reporting/ReportSchedulerImpl.h
//src/app/reporting/SynchronizedReportSchedulerImpl.h
//src/app/reporting/reporting.h
//src/app/server/AclStorage.h
//src/app/server/AppDelegate.h
//src/app/server/CommissioningModeProvider.h
//src/app/server/CommissioningWindowManager.h
//src/app/server/DefaultAclStorage.h
//src/app/server/Dnssd.h
//src/app/server/EchoHandler.h
//src/app/server/OnboardingCodesUtil.h
//src/app/server/Server.h
//src/app/util/MatterCallbacks.h
//src/app/util/af-types.h
//src/app/util/att-storage.h
//src/app/util/attribute-metadata.h
//src/app/util/attribute-storage-null-handling.h
//src/app/util/basic-types.h
//src/app/util/ember-strings.h
//src/app/util/endpoint-config-defines.h
//src/app/util/odd-sized-integers.h
//src/app/util/types_stub.h
//src/ble/BLEEndPoint.h
//src/ble/Ble.h
//src/ble/BleApplicationDelegate.h
//src/ble/BleConfig.h
//src/ble/BleConnectionDelegate.h
//src/ble/BleError.h
//src/ble/BleLayer.h
//src/ble/BleLayerDelegate.h
//src/ble/BlePlatformDelegate.h
//src/ble/BleRole.h
//src/ble/BleUUID.h
//src/ble/BtpEngine.h
//src/ble/CHIPBleServiceData.h
//src/controller/AbstractDnssdDiscoveryController.h
//src/controller/AutoCommissioner.h
//src/controller/CHIPCluster.h
//src/controller/CHIPCommissionableNodeController.h
//src/controller/CHIPDeviceController.h
//src/controller/CHIPDeviceControllerFactory.h
//src/controller/CHIPDeviceControllerSystemState.h
//src/controller/CommandSenderAllocator.h
//src/controller/CommissioneeDeviceProxy.h
//src/controller/CommissionerDiscoveryController.h
//src/controller/CommissioningDelegate.h
//src/controller/CommissioningWindowOpener.h
//src/controller/CurrentFabricRemover.h
//src/controller/DeviceDiscoveryDelegate.h
//src/controller/DevicePairingDelegate.h
//src/controller/ExampleOperationalCredentialsIssuer.h
//src/controller/InvokeInteraction.h
//src/controller/OperationalCredentialsDelegate.h
//src/controller/ReadInteraction.h
//src/controller/SetUpCodePairer.h
//src/controller/TypedCommandCallback.h
//src/controller/TypedReadCallback.h
//src/controller/WriteInteraction.h
//src/credentials/CHIPCert.h
//src/credentials/CHIPCert_Internal.h
//src/credentials/CHIPCertificateSet.h
//src/credentials/CertificateValidityPolicy.h
//src/credentials/CertificationDeclaration.h
//src/credentials/DeviceAttestationConstructor.h
//src/credentials/DeviceAttestationCredsProvider.h
//src/credentials/DeviceAttestationVendorReserved.h
//src/credentials/FabricTable.h
//src/credentials/GroupDataProvider.h
//src/credentials/GroupDataProviderImpl.h
//src/credentials/LastKnownGoodTime.h
//src/credentials/OperationalCertificateStore.h
//src/credentials/PersistentStorageOpCertStore.h
//src/credentials/TestOnlyLocalCertificateAuthority.h
//src/credentials/attestation_verifier/DacOnlyPartialAttestationVerifier.h
//src/credentials/attestation_verifier/DefaultDeviceAttestationVerifier.h
//src/credentials/attestation_verifier/DeviceAttestationDelegate.h
//src/credentials/attestation_verifier/DeviceAttestationVerifier.h
//src/credentials/attestation_verifier/TestPAAStore.h
//src/credentials/examples/DeviceAttestationCredsExample.h
//src/credentials/examples/ExampleDACs.h
//src/credentials/examples/ExamplePAI.h
//src/credentials/examples/LastKnownGoodTimeCertificateValidityPolicyExample.h
//src/credentials/examples/StrictCertificateValidityPolicyExample.h
//src/credentials/tests/CHIPAttCert_test_vectors.h
//src/crypto/CHIPCryptoPAL.h
//src/crypto/DefaultSessionKeystore.h
//src/crypto/OperationalKeystore.h
//src/crypto/PersistentStorageOperationalKeystore.h
//src/crypto/RandUtils.h
//src/crypto/RawKeySessionKeystore.h
//src/crypto/SessionKeystore.h
//src/include/platform/BuildTime.h
//src/include/platform/CHIPDeviceConfig.h
//src/include/platform/CHIPDeviceError.h
//src/include/platform/CHIPDeviceEvent.h
//src/include/platform/CHIPDeviceLayer.h
//src/include/platform/CommissionableDataProvider.h
//src/include/platform/ConfigurationManager.h
//src/include/platform/ConnectivityManager.h
//src/include/platform/DeviceControlServer.h
//src/include/platform/DeviceInstanceInfoProvider.h
//src/include/platform/GeneralUtils.h
//src/include/platform/KeyValueStoreManager.h
//src/include/platform/KvsPersistentStorageDelegate.h
//src/include/platform/PersistedStorage.h
//src/include/platform/PlatformManager.h
//src/include/platform/RuntimeOptionsProvider.h
//src/include/platform/TestOnlyCommissionableDataProvider.h
//src/include/platform/ThreadStackManager.h
//src/include/platform/internal/BLEManager.h
//src/include/platform/internal/CHIPDeviceLayerInternal.h
//src/include/platform/internal/DeviceNetworkInfo.h
//src/include/platform/internal/EventLogging.h
//src/include/platform/internal/GenericConfigurationManagerImpl.h
//src/include/platform/internal/GenericConnectivityManagerImpl.h
//src/include/platform/internal/GenericConnectivityManagerImpl_BLE.h
//src/include/platform/internal/GenericConnectivityManagerImpl_NoBLE.h
//src/include/platform/internal/GenericConnectivityManagerImpl_NoThread.h
//src/include/platform/internal/GenericConnectivityManagerImpl_NoWiFi.h
//src/include/platform/internal/GenericConnectivityManagerImpl_TCP.h
//src/include/platform/internal/GenericConnectivityManagerImpl_Thread.h
//src/include/platform/internal/GenericConnectivityManagerImpl_UDP.h
//src/include/platform/internal/GenericConnectivityManagerImpl_WiFi.h
//src/include/platform/internal/GenericDeviceInstanceInfoProvider.h
//src/include/platform/internal/GenericPlatformManagerImpl.h
//src/include/platform/internal/GenericPlatformManagerImpl_FreeRTOS.h
//src/include/platform/internal/GenericPlatformManagerImpl_POSIX.h
//src/include/platform/internal/testing/ConfigUnitTest.h
//src/inet/BasicPacketFilters.h
//src/inet/EndPointBasis.h
//src/inet/EndPointStateSockets.h
//src/inet/EndpointQueueFilter.h
//src/inet/IANAConstants.h
//src/inet/IPAddress.h
//src/inet/IPPacketInfo.h
//src/inet/IPPrefix.h
//src/inet/Inet.h
//src/inet/InetArgParser.h
//src/inet/InetConfig.h
//src/inet/InetError.h
//src/inet/InetFaultInjection.h
//src/inet/InetInterface.h
//src/inet/InetInterfaceImpl.h
//src/inet/InetLayer.h
//src/inet/TCPEndPoint.h
//src/inet/TCPEndPointImpl.h
//src/inet/TCPEndPointImplSockets.h
//src/inet/UDPEndPoint.h
//src/inet/UDPEndPointImpl.h
//src/inet/UDPEndPointImplSockets.h
//src/inet/ZephyrSocket.h
//src/inet/arpa-inet-compatibility.h
//src/lib/address_resolve/AddressResolve.h
//src/lib/address_resolve/AddressResolve_DefaultImpl.h
//src/lib/address_resolve/TracingStructs.h
//src/lib/asn1/ASN1.h
//src/lib/asn1/ASN1Config.h
//src/lib/asn1/ASN1Error.h
//src/lib/asn1/ASN1Macros.h
//src/lib/core/CASEAuthTag.h
//src/lib/core/CHIPCallback.h
//src/lib/core/CHIPConfig.h
//src/lib/core/CHIPCore.h
//src/lib/core/CHIPEncoding.h
//src/lib/core/CHIPError.h
//src/lib/core/CHIPKeyIds.h
//src/lib/core/CHIPPersistentStorageDelegate.h
//src/lib/core/CHIPSafeCasts.h
//src/lib/core/CHIPVendorIdentifiers.hpp
//src/lib/core/ClusterEnums.h
//src/lib/core/DataModelTypes.h
//src/lib/core/ErrorStr.h
//src/lib/core/Global.h
//src/lib/core/GroupId.h
//src/lib/core/InPlace.h
//src/lib/core/NodeId.h
//src/lib/core/OTAImageHeader.h
//src/lib/core/Optional.h
//src/lib/core/PasscodeId.h
//src/lib/core/PeerId.h
//src/lib/core/ReferenceCounted.h
//src/lib/core/ScopedNodeId.h
//src/lib/core/TLV.h
//src/lib/core/TLVBackingStore.h
//src/lib/core/TLVCircularBuffer.h
//src/lib/core/TLVCommon.h
//src/lib/core/TLVData.h
//src/lib/core/TLVDebug.h
//src/lib/core/TLVReader.h
//src/lib/core/TLVTags.h
//src/lib/core/TLVTypes.h
//src/lib/core/TLVUpdater.h
//src/lib/core/TLVUtilities.h
//src/lib/core/TLVWriter.h
//src/lib/core/Unchecked.h
//src/lib/dnssd/ActiveResolveAttempts.h
//src/lib/dnssd/Advertiser.h
//src/lib/dnssd/Advertiser_ImplMinimalMdnsAllocator.h
//src/lib/dnssd/Constants.h
//src/lib/dnssd/IPAddressSorter.h
//src/lib/dnssd/IncrementalResolve.h
//src/lib/dnssd/MinimalMdnsServer.h
//src/lib/dnssd/Resolver.h
//src/lib/dnssd/ResolverProxy.h
//src/lib/dnssd/ServiceNaming.h
//src/lib/dnssd/TxtFields.h
//src/lib/dnssd/Types.h
//src/lib/dnssd/minimal_mdns/AddressPolicy.h
//src/lib/dnssd/minimal_mdns/AddressPolicy_DefaultImpl.h
//src/lib/dnssd/minimal_mdns/ListenIterator.h
//src/lib/dnssd/minimal_mdns/Logging.h
//src/lib/dnssd/minimal_mdns/Parser.h
//src/lib/dnssd/minimal_mdns/Query.h
//src/lib/dnssd/minimal_mdns/QueryBuilder.h
//src/lib/dnssd/minimal_mdns/QueryReplyFilter.h
//src/lib/dnssd/minimal_mdns/RecordData.h
//src/lib/dnssd/minimal_mdns/ResponseBuilder.h
//src/lib/dnssd/minimal_mdns/ResponseSender.h
//src/lib/dnssd/minimal_mdns/Server.h
//src/lib/dnssd/minimal_mdns/ServerIPAddresses.h
//src/lib/dnssd/minimal_mdns/core/BytesRange.h
//src/lib/dnssd/minimal_mdns/core/Constants.h
//src/lib/dnssd/minimal_mdns/core/DnsHeader.h
//src/lib/dnssd/minimal_mdns/core/FlatAllocatedQName.h
//src/lib/dnssd/minimal_mdns/core/HeapQName.h
//src/lib/dnssd/minimal_mdns/core/QName.h
//src/lib/dnssd/minimal_mdns/core/QNameString.h
//src/lib/dnssd/minimal_mdns/core/RecordWriter.h
//src/lib/dnssd/minimal_mdns/records/IP.h
//src/lib/dnssd/minimal_mdns/records/Ptr.h
//src/lib/dnssd/minimal_mdns/records/ResourceRecord.h
//src/lib/dnssd/minimal_mdns/records/Srv.h
//src/lib/dnssd/minimal_mdns/records/Txt.h
//src/lib/dnssd/minimal_mdns/responders/IP.h
//src/lib/dnssd/minimal_mdns/responders/Ptr.h
//src/lib/dnssd/minimal_mdns/responders/QueryResponder.h
//src/lib/dnssd/minimal_mdns/responders/RecordResponder.h
//src/lib/dnssd/minimal_mdns/responders/ReplyFilter.h
//src/lib/dnssd/minimal_mdns/responders/Responder.h
//src/lib/dnssd/minimal_mdns/responders/Srv.h
//src/lib/dnssd/minimal_mdns/responders/Txt.h
//src/lib/dnssd/platform/Dnssd.h
//src/lib/dnssd/platform/DnssdBrowseDelegate.h
//src/lib/support/Base64.h
//src/lib/support/BitFlags.h
//src/lib/support/BitMask.h
//src/lib/support/BufferReader.h
//src/lib/support/BufferWriter.h
//src/lib/support/BytesCircularBuffer.h
//src/lib/support/BytesToHex.h
//src/lib/support/CHIPArgParser.hpp
//src/lib/support/CHIPCounter.h
//src/lib/support/CHIPFaultInjection.h
//src/lib/support/CHIPMem.h
//src/lib/support/CHIPMemString.h
//src/lib/support/CHIPPlatformMemory.h
//src/lib/support/CodeUtils.h
//src/lib/support/CommonIterator.h
//src/lib/support/CommonPersistentData.h
//src/lib/support/DLLUtil.h
//src/lib/support/DefaultStorageKeyAllocator.h
//src/lib/support/Defer.h
//src/lib/support/EnforceFormat.h
//src/lib/support/FibonacciUtils.h
//src/lib/support/FixedBufferAllocator.h
//src/lib/support/Fold.h
//src/lib/support/FunctionTraits.h
//src/lib/support/IniEscaping.h
//src/lib/support/IntrusiveList.h
//src/lib/support/Iterators.h
//src/lib/support/LambdaBridge.h
//src/lib/support/LifetimePersistedCounter.h
//src/lib/support/LinkedList.h
//src/lib/support/ObjectLifeCycle.h
//src/lib/support/PersistedCounter.h
//src/lib/support/PersistentData.h
//src/lib/support/PersistentStorageAudit.h
//src/lib/support/PersistentStorageMacros.h
//src/lib/support/Pool.h
//src/lib/support/PoolWrapper.h
//src/lib/support/PrivateHeap.h
//src/lib/support/ReferenceCountedHandle.h
//src/lib/support/SafeInt.h
//src/lib/support/SafePointerCast.h
//src/lib/support/SafeString.h
//src/lib/support/Scoped.h
//src/lib/support/ScopedBuffer.h
//src/lib/support/SetupDiscriminator.h
//src/lib/support/SortUtils.h
//src/lib/support/Span.h
//src/lib/support/StateMachine.h
//src/lib/support/StringBuilder.h
//src/lib/support/StringSplitter.h
//src/lib/support/TestGroupData.h
//src/lib/support/TestPersistentStorageDelegate.h
//src/lib/support/ThreadOperationalDataset.h
//src/lib/support/TimeUtils.h
//src/lib/support/TypeTraits.h
//src/lib/support/Variant.h
//src/lib/support/VerificationMacrosNoLogging.h
//src/lib/support/ZclString.h
//src/lib/support/logging/BinaryLogging.h
//src/lib/support/logging/CHIPLogging.h
//src/lib/support/logging/Constants.h
//src/lib/support/logging/TextOnlyLogging.h
//src/lib/support/static_support_smart_ptr.h
//src/lib/support/utf8.h
//src/lib/support/verhoeff/Verhoeff.h
//src/messaging/ApplicationExchangeDispatch.h
//src/messaging/EphemeralExchangeDispatch.h
//src/messaging/ErrorCategory.h
//src/messaging/ExchangeContext.h
//src/messaging/ExchangeDelegate.h
//src/messaging/ExchangeHolder.h
//src/messaging/ExchangeMessageDispatch.h
//src/messaging/ExchangeMgr.h
//src/messaging/Flags.h
//src/messaging/ReliableMessageContext.h
//src/messaging/ReliableMessageMgr.h
//src/messaging/ReliableMessageProtocolConfig.h
//src/messaging/SessionParameters.h
//src/platform/DeviceSafeQueue.h
//src/platform/GLibTypeDeleter.h
//src/platform/Linux/BLEManagerImpl.h
//src/platform/Linux/BlePlatformConfig.h
//src/platform/Linux/CHIPDevicePlatformConfig.h
//src/platform/Linux/CHIPDevicePlatformEvent.h
//src/platform/Linux/CHIPLinuxStorage.h
//src/platform/Linux/CHIPLinuxStorageIni.h
//src/platform/Linux/CHIPPlatformConfig.h
//src/platform/Linux/ConfigurationManagerImpl.h
//src/platform/Linux/ConnectivityManagerImpl.h
//src/platform/Linux/ConnectivityUtils.h
//src/platform/Linux/DeviceInstanceInfoProviderImpl.h
//src/platform/Linux/DiagnosticDataProviderImpl.h
//src/platform/Linux/InetPlatformConfig.h
//src/platform/Linux/KeyValueStoreManagerImpl.h
//src/platform/Linux/NetworkCommissioningDriver.h
//src/platform/Linux/PlatformManagerImpl.h
//src/platform/Linux/PosixConfig.h
//src/platform/Linux/SystemPlatformConfig.h
//src/platform/Linux/ThreadStackManagerImpl.h
//src/platform/Linux/bluez/AdapterIterator.h
//src/platform/Linux/bluez/BluezAdvertisement.h
//src/platform/Linux/bluez/BluezConnection.h
//src/platform/Linux/bluez/BluezEndpoint.h
//src/platform/Linux/bluez/BluezObjectIterator.h
//src/platform/Linux/bluez/BluezObjectList.h
//src/platform/Linux/bluez/BluezObjectManager.h
//src/platform/Linux/bluez/ChipDeviceScanner.h
//src/platform/Linux/bluez/Types.h
//src/platform/logging/LogV.h
//src/protocols/Protocols.h
//src/protocols/bdx/BdxMessages.h
//src/protocols/bdx/BdxTransferDiagnosticLog.h
//src/protocols/bdx/BdxTransferDiagnosticLogPool.h
//src/protocols/bdx/BdxTransferProxy.h
//src/protocols/bdx/BdxTransferProxyDiagnosticLog.h
//src/protocols/bdx/BdxTransferServer.h
//src/protocols/bdx/BdxTransferServerDelegate.h
//src/protocols/bdx/BdxTransferSession.h
//src/protocols/bdx/BdxUri.h
//src/protocols/bdx/DiagnosticLogs.h
//src/protocols/bdx/StatusCode.h
//src/protocols/bdx/TransferFacilitator.h
//src/protocols/echo/Echo.h
//src/protocols/interaction_model/Constants.h
//src/protocols/interaction_model/StatusCode.h
//src/protocols/interaction_model/StatusCodeList.h
//src/protocols/secure_channel/CASEDestinationId.h
//src/protocols/secure_channel/CASEServer.h
//src/protocols/secure_channel/CASESession.h
//src/protocols/secure_channel/CheckInCounter.h
//src/protocols/secure_channel/CheckinMessage.h
//src/protocols/secure_channel/Constants.h
//src/protocols/secure_channel/DefaultSessionResumptionStorage.h
//src/protocols/secure_channel/MessageCounterManager.h
//src/protocols/secure_channel/PASESession.h
//src/protocols/secure_channel/PairingSession.h
//src/protocols/secure_channel/RendezvousParameters.h
//src/protocols/secure_channel/SessionEstablishmentDelegate.h
//src/protocols/secure_channel/SessionEstablishmentExchangeDispatch.h
//src/protocols/secure_channel/SessionResumptionStorage.h
//src/protocols/secure_channel/SimpleSessionResumptionStorage.h
//src/protocols/secure_channel/StatusReport.h
//src/protocols/secure_channel/UnsolicitedStatusHandler.h
//src/protocols/user_directed_commissioning/UDCClientState.h
//src/protocols/user_directed_commissioning/UDCClients.h
//src/protocols/user_directed_commissioning/UserDirectedCommissioning.h
//src/setup_payload/AdditionalDataPayload.h
//src/setup_payload/AdditionalDataPayloadGenerator.h
//src/setup_payload/AdditionalDataPayloadParser.h
//src/setup_payload/Base38.h
//src/setup_payload/Base38Decode.h
//src/setup_payload/Base38Encode.h
//src/setup_payload/ManualSetupPayloadGenerator.h
//src/setup_payload/ManualSetupPayloadParser.h
//src/setup_payload/QRCodeSetupPayloadGenerator.h
//src/setup_payload/QRCodeSetupPayloadParser.h
//src/setup_payload/SetupPayload.h
//src/setup_payload/SetupPayloadHelper.h
//src/system/PlatformEventSupport.h
//src/system/SocketEvents.h
//src/system/SystemAlignSize.h
//src/system/SystemClock.h
//src/system/SystemConfig.h
//src/system/SystemError.h
//src/system/SystemEvent.h
//src/system/SystemFaultInjection.h
//src/system/SystemLayer.h
//src/system/SystemLayerImpl.h
//src/system/SystemLayerImplSelect.h
//src/system/SystemMutex.h
//src/system/SystemPacketBuffer.h
//src/system/SystemPacketBufferInternal.h
//src/system/SystemStats.h
//src/system/SystemTimer.h
//src/system/TLVPacketBufferBackingStore.h
//src/system/TimeSource.h
//src/system/WakeEvent.h
//src/tracing/backend.h
//src/tracing/log_declares.h
//src/tracing/macros.h
//src/tracing/metric_event.h
//src/tracing/metric_keys.h
//src/tracing/metric_macros.h
//src/tracing/registry.h
//src/transport/CryptoContext.h
//src/transport/GroupPeerMessageCounter.h
//src/transport/GroupSession.h
//src/transport/MessageCounter.h
//src/transport/MessageCounterManagerInterface.h
//src/transport/PeerMessageCounter.h
//src/transport/SecureMessageCodec.h
//src/transport/SecureSession.h
//src/transport/SecureSessionTable.h
//src/transport/Session.h
//src/transport/SessionConnectionDelegate.h
//src/transport/SessionDelegate.h
//src/transport/SessionHolder.h
//src/transport/SessionManager.h
//src/transport/SessionMessageCounter.h
//src/transport/SessionMessageDelegate.h
//src/transport/SessionUpdateDelegate.h
//src/transport/TraceMessage.h
//src/transport/TracingStructs.h
//src/transport/TransportMgr.h
//src/transport/TransportMgrBase.h
//src/transport/UnauthenticatedSessionTable.h
//src/transport/raw/ActiveTCPConnectionState.h
//src/transport/raw/BLE.h
//src/transport/raw/Base.h
//src/transport/raw/MessageHeader.h
//src/transport/raw/PeerAddress.h
//src/transport/raw/TCP.h
//src/transport/raw/TCPConfig.h
//src/transport/raw/Tuple.h
//src/transport/raw/UDP.h
//third_party/inipp/repo/inipp/inipp/inipp.h
//third_party/nlassert/repo/include/nlassert.h
//third_party/nlfaultinjection/include/nlfaultinjection.hpp
//third_party/nlio/repo/include/nlbyteorder-big.h
//third_party/nlio/repo/include/nlbyteorder-little.h
//third_party/nlio/repo/include/nlbyteorder.h
//third_party/nlio/repo/include/nlbyteorder.hpp
//third_party/nlio/repo/include/nlio-base.h
//third_party/nlio/repo/include/nlio-base.hpp
//third_party/nlio/repo/include/nlio-byteorder-big.h
//third_party/nlio/repo/include/nlio-byteorder-big.hpp
//third_party/nlio/repo/include/nlio-byteorder-little.h
//third_party/nlio/repo/include/nlio-byteorder-little.hpp
//third_party/nlio/repo/include/nlio-byteorder.h
//third_party/nlio/repo/include/nlio-byteorder.hpp
//third_party/nlio/repo/include/nlio-private.h
//third_party/nlio/repo/include/nlio.h
//third_party/nlio/repo/include/nlio.hpp
//zzz_generated/app-common/app-common/zap-generated/attribute-type.h
//zzz_generated/app-common/app-common/zap-generated/cluster-enums-check.h
//zzz_generated/app-common/app-common/zap-generated/cluster-enums.h
//zzz_generated/app-common/app-common/zap-generated/cluster-objects.h
//zzz_generated/app-common/app-common/zap-generated/ids/Attributes.h
//zzz_generated/app-common/app-common/zap-generated/ids/Clusters.h
//zzz_generated/app-common/app-common/zap-generated/ids/Commands.h
//zzz_generated/app-common/app-common/zap-generated/ids/Events.h

This list will vary depending on how the build was configured. There will probably be something missing as listing headers in the build file is only weakly enforced in CI, but they can be added at the appropriate place.

Other settings relevant to library interface are also in the JSON such as defines and include paths and can be gathered a similar way.

The ZAP generated code is also going to require special care.

I still wouldn't encourage separate distribution of headers and sources. There isn't enough decoupling or layering or ABI attention to make precompilation worthwhile.

mspang commented 3 months ago

In summary - making a tarball of what's compiled by GN isn't the hard part. The hard part is that to port Matter, you need to make significant changes to the SDK and recompile it.

project-chip / connectedhomeip

GN build parity: packaging/install #1513