Importing created BOM into Dependency-Track multiple times removes components

[malice00]

We have just started using dependency-track and are therefor using this tool to generate BOMs for our iPhone apps. As with other tooling, we run this tool every night and import the result to DT. Since our depdencies don't change every night, we expected the result to be the same every night. However, the initial import of the BOM shows more components in the project than the following imports.

In the current project, we have 65 components (according to the BOM -- haven't validated manually though), which is the exact amount being shown after the first import. On the newer imports, DT only shows 46 components... Seeing this only happens when importing my CocaPods-BOM, I assume it is not a problem in DT, but rather in the data generated by this tool.

I've looked through some of the dependencies to maybe make some sense of it and I noticed that the components that are disappearing have similar/the same purl (difference is in the query-part, which I believe is unimportant) -- maybe/probably because they are defined as a subspec?

Attached are the Podfile, Podfile.lock and the generated BOM (local paths/servers removed). And in case it is important/helps to find the problem: this is a react native iPhone app...

Podfile.txt Podfile.lock.txt bom.xml.txt

[macblazer]

Thanks for the files. I'll see if I can reproduce the problem.

My guess is that some string comparison we're doing is looking for the first pod matching part of the name instead of the first pod matching the whole name. Combine that with the order of iteration through the pods being indeterminate and maybe the code finds varying things between runs.

[macblazer]

After reviewing the Podfile.txt, there's not a lot I can do with that specifically since it calls out to a bunch of other files not included like '../node_modules/react-native/scripts/react_native_pods' and '../node_modules/@react-native-community/cli-platform-ios/native_modules', and '../node_modules/react-native-permissions/iOS'. It also calls the directives/functions use_native_modules and use_react_native. It's possible those are all just standard React Native things, but there might be some specific things in there that you've set up that I don't see.

I am continuing the analysis mostly based on the lockfile at this point which does indeed have 65 pods listed.

I may make a synthetic Podfile that will use the same top level pods indicated in the Podfile.lock and see if I can find the error in the complete processing.

[malice00]

Ah, yes, sorry about that. In RN projects, several dependencies come from the NodeJS-dependencies... I tried to create a simple package.json with lock-file, since apparently our dependencies actually don't match (anymore?)... Maybe this might help you? Run 'npm i' with a current version of NodeJS and you should get those referenced files.

package.json.txt package-lock.json.txt

[macblazer]

I tried that and got a couple errors about conflicting react versions while resolving: @react-native-community/masked-view@0.1.10. Something wants react@16.14.0 and the package.json specifies react@17.0.1 so it just bails out.

I'll try a few other things, but I haven't been able to get a simpler reproducible case yet.

[malice00]

I can run the above project just fine -- just make sure the lock-file is there (both files renamed to lose the '.txt'), and NPM should install everything just fine...

If not, I can send some PodSpecs if you tell me which ones you'd like to look at...

[malice00]

Sorry for double posting, I wanted to make sure a notification was sent when I posted this.

It seems my sample project only works when using the same old version of NodeJS and NPM that we are using... So I looked into creating a new project.

The easiest way to get a reproducible project is to simply create a new react-native project:

npx react-native@latest init test

If you have all the necessary tools installed, this should run for a bit and you should have a project where you can then directly run cyclonedx-cocoapods:

cd ios cyclonedx-cocoapods

Unfortunately one of the components generates a tag 'author' and 'publisher' that was too long to be imported by DependencyTrack -- this needs to be reduced before importing.

The BOM can now be imported into DependencyTrack and there should be 80 components in the project. Re-uploading the same BOM a little later, should reduce the number of components to 49!

I hope this helps in locating the issue...

[macblazer]

Finally found some more time for this. The good news is I have replicated the problem and understand it.

It stems from the use of CocoaPods subspecs. The bom.xml file generated by cyclonedx-cocoapods on a basic ReactNative app as above has 80 dependencies listed. Of those dependencies, 32 of them have CocoaPods subspecs being used (seen with a # in the <purl> tag).

There are five unique Pods that are represented in those 32 subspecs, and four of the five are also listed as the complete dependency without any subspec. One Pod, pkg:cocoapods/ReactCommon@0.71.8, is listed only with a subspec (two different entries with two different subspecs just to make it a little more confusing).

So doing the math, 80 total dependencies in the bom.xml minus the 32 subspec listings and adding one because that ReactCommon pod is not listed as the full code dependency at all, we end up with 49. From 80 to 49 just as you pointed out in your April post.

So it's a problem of this tool adding all of the subspecs that are indeed referenced separately in the complex Podfile as well as adding the complete dependency. Dependency Track (DT) seems to initially keep those subspecs separately, and then later it realizes that those are actually duplicated so it reduces to the unique 49.

It seems to me there is one mostly clear fix that can be done in this code. Add the subspec dependencies only if the full pod is not also listed.. This would simplify the BOM to 50 dependencies (leaving the sort-of-duplicated ReactCommon in twice); and DT would still see 49. This change is complicated by this info from the CocoaPods documentation:

On one side, a specification automatically inherits as a dependency all it children ‘sub-specifications’ (unless a default subspec is specified). A Pod should make available the full library by default. Users can fine tune their dependencies, and exclude unneeded subspecs, once their requirements are known. Therefore, this attribute is rarely needed. It is intended to be used to select a default if there are ‘sub-specifications’ which provide alternative incompatible implementations, or to exclude modules rarely needed (especially if they trigger dependencies on other libraries).

By dropping the explicit subspecs from the output of this tool, we may be losing a bit of information if the Pod is using a default subspec and the app is explicitly opting-in to some other subspec(s). This is probably acceptable to the community at large, since you mainly want to know overall what unique Pods you are including and can dig into more details in the code if really necessary.

The slightly harder problem to solve is the ReactCommon case where the app is including two different subspecs only and not the full dependency. Dependency Track would see these as one and the same Pod being used, but the reality is the code is using two different pieces of the Pod.

A different solution could be Merge all whole pods and subspecs into a single dependency in the BOM, and drop all subspec output.. This tool could do that, and never output a purl with a # in it at all. This would mean that the output would drop to 49 dependencies just like DT sees.

Seems like the ecosystem is all about knowing "does X software use Y dependency?" so maybe this last solution of not worrying about subspecs and just collapsing it down to answering the basic "is any part of this Pod used at all?" will be good enough for everybody. When a CVE comes along in a Pod you still have to manually look at your usage and see if you are using the affected part of the code.

[malice00]

First of all, thanks for this in depth research and explanation!

I think to make a decision on which would be the best solution, is to check how/where CVEs are registered in the CocoaPods-world. I've been browsing through OSS Index a bit and I do feel that CVEs in CocoaPods are registered on pod-level, not the subspec. If this is indeed the case, then I think it would indeed be sufficient to generate an SBOM containing only the pods.

Although, it might also be nice to maybe add a switch in the tool to be able to generate either a 'pod'-SBOM for DT and a more detailed one to support during clearing... Then again, in my example it would only help in the case of ReactCommon, since for all other dependencies both the full pod and subspecs where listed...

[macblazer]

There is one other solution to this problem: update the documentation. Nothing changes in the cyclonedx-cocoapods code; it identifies all of the dependencies from the Podfile, including subspecs if those are included. Dependency Track can continue to do its thing of merging them down to unique repositories to track (for CVEs, licenses, etc).

The README file here can be updated to show an example of what the output looks like with CocoaPods subspecs, and how that will appear in DT on the first import and then on subsequent imports.

[malice00]

Thinking about that, I actually feel this is the easiest way to 'solve' this issue! Currently I already have all information I need both in DT or more detailed in the SBOM itself. And when this behavior is documented, nobody needs to be confused about what's happening like I was!