Application Version History

[kchason]

Background

Some forensic tools expose the history of applications on mobile devices, to include version installation history and there is not a current method within the ontology to represent this data.

Requirements

Requirement 1

Allow representation of the installed date, removed date, and installed version of applications.

Risk / Benefit analysis

Benefits

Similar to uco-observable:numberOfLaunches this allows for consumers to be able to glean additional contextual information about the usage of the application.

Risks

The submitter is unaware of risks associated with this change.

Competencies demonstrated

Competency 1

A device is discovered to have had a mobile device management application installed that has been upgraded several times through the application store. There was a vulnerability identified in an earlier version of the application that allowed arbitrary remote code execution that has since been patched.

Competency Question 1.1

During which date(s) could the device potentially have been compromised and the user unaware of the activities occuring on the device?

Result 1.1

Query is untested as the drafting namespace has not been developed for this proposal.

SELECT DISTINCT ?lApplicationIdentifier ?lVersion ?lInstalledDate ?lUninstalledDate
WHERE
{
    ?nApplication
    a uco-observable:Application ;
    uco-core:hasFacet ?nApplicationFacet ;
    .

    ?nApplicationFacet
    a uco-observable:ApplicationFacet ;
    drafting:installedVersions ?nApplicationVersion ;
    uco-observable:applicationIdentifier ?lApplicationIdentifier ;
    .

    ?nApplicationVersion
    a uco-observable:ApplicationVersion ;
    uco-core:hasFacet ?nApplicationVersionFacet ;
    .

    ?nApplicationVersionFacet
    a uco-observable:ApplicationVersionFacet ;
    drafting:installedDate ?lInstalledDate ;
    drafting:version ?lVersion ;
    .
    OPTIONAL {
        ?nApplicationVersionFacet drafting:uninstalledDate ?lUninstalledDate .
    }
}

?lApplicationIdentifier	?lVersion	?lInstalledDate	?lUninstalledDate
Chat App	1.1.0	2020-01-01	2021-03-01
Chat App	1.4.0	2021-03-01	2022-03-01

Solution suggestion

Create an object uco-observable:ApplicationVersion

Create a similarly named facet uco-observable:ApplicationVersionFacet that contains:

• installedDate
• uninstalledDate
• version

Add a new property to uco-observable:ApplicationFacet called installedVersions that contains 0 -> n references to ApplicationVersions.

Coordination

• Tracking in Jira ticket OC-234
• [x] Administrative review to be completed
• [x] Requirements to be discussed in Ontology Committee (OC) meeting, 2022-05-05
• [x] Requirements Review vote occurred, passing, on 2022-05-05
• [x] Requirements development phase completed.
• [x] Solutions Approval to be discussed in OC meeting, 2022-08-16
• [x] Solutions Approval to be discussed again in OC meeting, 2022-08-25
• [x] Solutions Approval vote occurred, passing, on 2022-08-25
• [x] Solutions development phase completed.
• [x] Implementation merged into develop
• [x] Milestone linked
• [x] Documentation logged in pending release page

[ajnelson-nist]

@kchason , thanks for this.

One nit: installedVersions should be spelled in the singular. This is more a note for future properties.

Structurally, I think this will take a significantly different form. I've been working on a similar representation issue with software asset management, with a package manager ecosystem model "SWIDREG" posted here. There ends up being a significant difference in abstraction levels between (using the SWIDREG term style) an application and a versioned application. This is all separate from an application deployment (not a SWIDREG term, but if I recall correctly another NIST project uses a term close to this), which is what you'd find as a usable/runnable piece of a system.

I don't think it's appropriate to use a Facet-housed property to link a observable:Application to one or more drafting:ApplicationVersions. For one thing, there could be significant disagreement on whether versions known to exist but not found on the phone should be part of a concept representing, say, a long-historied mobile application. Further, the temporality you suggested with the properties drafting:installedDate and drafting:uninstalledDate imply there is a time-bound relationship between the device and the versioned application being installed on the device. Also, your representation might have some difficulty with explaining that a single versioned application may have been uninstalled and reinstalled multiple times. You'd be tying the time-bounding of the installation to the inherent identity of the versioned application That time bounding is more appropriate as either a "Deployment" ObservableObject subclass, or as an ObservableRelationship. I suspect ObservableRelationship is most appropriate.

The relevant UCO design document section is Section 6 - Relationships, drafted by @sbarnum . If something isn't quite gelling between what I'm writing here and what Sean posted, we should look to refine the document.

Meanwhile, UCO does have observable:version, used in observable:ApplicationFacet. So, UCO's idea of an application inherently ties a version in already. What UCO doesn't have is an abstraction that goes the other way - a "versioned application" to an "application as a series of releases". (I may have found another ontology that represents these abstraction levels, but it's through what may or may not be a significant semantic bend. I'll probably have more on this after UCO's 1.0.0 release, unless the Ontology Committee gets particularly eager to see this explored.)

Summarizing, and this is just my opinion: You have identified a need and corresponding coverage gap with the shape of the results of your query. We might not need to build an ApplicationVersion class. We should demonstrate how to represent the time interval in which an application is installed on a device, and this might need concept development. If this should use an ObservableRelationship object, and there isn't an ObservableRelationship vocabulary member that looks like it fits, a string-value should probably be added.

In light of observable:ApplicationFacet having observable:version, do you have any re-thinking to do on this proposal?

[plbt5]

@kchason Thanks for using the template for specifying the CP. My comments, as follows:

3. Benefits: Why is to be able to glean additional contextual information considered a benefit? What can the user do or achieve with that information that s/he couldn't before?
4. CQ 1.1:
   1. I'd suggest to rephrase "have been compromised and the user unaware" to "have been compromised , leaving the user unaware"; the word "and" might imply two distinct CQs, one about dates and one about users.
   2. The Result implies the availability of an lUninstalledDate that is not mentioned. Moreover, users can install new versions without uninstalling the previous version of an app. This implies to elaborate on the concept of lUninstalledDate in relation to installations of apps and what is signified with this concept in reality.
   3. Are there additional constraints or conflicts expected with different operating systems?
5. Result:
   1. Is a date sufficient or should a time be inserted as well?
   2. Since users are allowed to install the same app more than once, how should such be reflected in the results?

This CP includes one CQ only, implying that no more competencies will be included in the implementation following this CP. Although I assume this to be correct, just for assurance: is this CP meant to cover this as the only CQ that is to be implemented, or is the CQ just an example of what is intended to be covered? In case of the latter, please add the CQs that are required to completely cover the intention of this CP.

[kchason]

@ajnelson-nist the version naming can definitely be updated to be singular.

UCO does have observable:version, but it doesn't have the concept of multiple versions, or generally version history so I believe there is still an outstanding need for this proposal. However, if there's an ontological form that makes more sense, I'm open to it.

@plbt5 I believe the CQs cover the use cases, but generally the idea was that it allows answers to questions (in conjunction with other information) such as

• Which installed applications had access to the internet at X date?
• During which date(s) could the device potentially have been compromised and the user unaware of the activities occuring on the device?

The lUninstalledDate is intentionally not required for that reason, but there are tools that do record the uninstall date (if applicable) so we should be able to represent that to further satisfy the aforementioned use cases.

With regards to the date format, a date/time does make sense, but I thought our date objects were actually date/time formats? This does allow multiple installations of the same version since it doesn't have a uniqueness constraint.

[plbt5]

@kchason Some remarks to your comment:

@plbt5 I believe the CQs cover the use cases, but generally the idea was that it allows answers to questions (in conjunction with other information) such as

• Which installed applications had access to the internet at X date?
• During which date(s) could the device potentially have been compromised and the user unaware of the activities occuring on the device?

If answers to these questions can be given without additional concepts, then indeed the use case is covered. However, it might be useful to specify these questions as additional CQ's, because these are new questions that could not be answered before; hence, new competencies. This feeds the test cases as well.

The lUninstalledDate is intentionally not required for that reason, but there are tools that do record the uninstall date (if applicable) so we should be able to represent that to further satisfy the aforementioned use cases.

Ah, I was unclear in my question, apologies. What I meant was: what is our position regarding parallel installations of an app? If one installs a (new or identical) version and not uninstall the existing one, does that mean that the previous one is by definition always uninstalled? I think that there are systems that allow installing an app more than once, but in different contexts. Do we need to consider situations where copy A was uninstalled while copy B is still there?

With regards to the date format, a date/time does make sense, but I thought our date objects were actually date/time formats? OK.

This does allow multiple installations of the same version since it doesn't have a uniqueness constraint.

Indeed, serialised installations are supported, however, due to the absence of a uniqueness constraint, it doesn't seem to support the parallel installations.

[sbarnum]

I am not sure if the 5/5/22 meeting was one of the few I missed but I do not recall seeing, reviewing or voting on this CP.

I can definitely see the need for expressing version history on applications but I do not believe the proposed solution is appropriate. I think the appropriate approach can be implemented with just a couple changes to the proposed solution yielding not only an appropriate approach but a much simpler one as well.

The primary issue with the proposed solution is that ApplicationVersion is not really an appropriate ObservableObject in the domain graph as it does not stand alone as a domain concept. It is rather a characteristic property of an Application.

I strongly propose that the following adjustments be made to the proposed solution:

• Remove the new observable:ApplicationVersion class definition
• Modify observable:ApplicationVersionFacet to be observable:ApplicationVersion and be a subclass of core:ClassBase (see CP-430) rather than `core:Facet'
• Modify observable:installedVersion to be observable:installedVersionHistory

These simple changes should enable all of the desired expressivity in a much simpler and more appropriate manner.

It could look something like:

{
  "@id": "kb:Application-62a8d5d4-245a-4c1a-944c-04a988e7953a",
  "@type": "uco-observable:Application",
  "uco-core:name": "Magic Crayon",
  "core:hasFacet": [
    {
      "@id": "kb:ApplicationFacet-e3f48e54-83ab-4800-b71d-1a072c89f4f1",
      "@type": "uco-observable:ApplicationFacet",
      "uco-observable:applicationIdentifier": "cpe:/a:DrawFan:MagicCrayon:4.5",
      "uco-observable:operatingSystem": "ios 12",
      "uco-observable:version": "4.5",
      "uco-observable:installedVersionHistory": [
        {
          "@id": "kb:ApplicationVersion-6aba579b-affe-4d4f-b2cc-14b2dbc72a7a",
          "@type": "uco-observable:ApplicationVersion",
          "uco-observable:installedDate": "2021-06-25T22:10:19.00Z",
          "uco-observable:uninstalledDate": "2022-01-12T09:22:20.00Z",
          "uco-observable:version": "3.5"
        },
        {
          "@id": "kb:ApplicationVersion-10336642-77ba-4c34-b059-ea0379ed768e",
          "@type": "uco-observable:ApplicationVersion",
          "uco-observable:installedDate": "2022-01-12T09:22:21.00Z",
          "uco-observable:uninstalledDate": "2022-06-15T17:59:43.25Z",
          "uco-observable:version": "4.2"
        },
        {
          "@id": "kb:ApplicationVersion-a930546c-944a-4902-a260-81c6d21d16ba",
          "@type": "uco-observable:ApplicationVersion",
          "uco-observable:installedDate": "2022-07-28T12:00:00.00Z",
          "uco-observable:version": "4.5"
        }
      ]
    }
  ]
}

[ajnelson-nist]

@kchason and @sbarnum - please can the two of you settle offline on the solution for this Issue. @sbarnum 's late remarks in yesterday's meeting confused the implementation status of this proposal.

At 6PM EDT Thursday, whatever is in PR 372 will be the solution proposed to the OCs.

[kchason]

@sbarnum your suggestions look like they'd meet the use case I was aiming for. I'll try to get this solution updated shortly.

[kchason]

The PR should be updated with your suggestions above

[sbarnum]

Thank you Keith.

I see two small issues remaining in the PR.

1) It looks like the property shape for observable:version got removed from ApplicationFacet. I believe it needs to still remain there as the simple and direct indicator of the version of an application. The applicationVersionHistory is an additional level of historical info that may or may not be provided. I think that shape needs added back in. 2) The sh:targetClass for ApplicationVersion is currently observable:ApplicationVersionFacet and needs the "Facet" part removed.

I made and committed these changes into the branch. Let me know if you have any major issues with either fix.

[kchason]

No conceptual issue with the changes, though it appears it's failing CI and has a merge conflict after the changes.

[ajnelson-nist]

CI is fixed, and the review workflow's done enough for us to consider @sbarnum 's updated implementation for a Solutions Approval vote Thursday.

I had a piece of feedback while we're still in the discussion window.

Something about the installedVersionHistory property made me uncomfortable, and made me wonder about whether the Application concept was overly broad to accept this property (via a Facet).

The definition of Application is currently:

An application is a particular software program designed for end users.

I've been concerned with installedVersionHistory pushing this definition, which is independent of whether the application is deployed or a deployable thing from, say, an app store, away from the "pre-deployment" software consumption lifecycle phase and firmly into the "deployment" lifecycle phase.

However, ApplicationFacet already has operatingSystem as a property. So, I am less concerned with installedVersionHistory broaching that design issue. It's already broached.

Is it appropriate---I don't mean in terms of committee procedures, I mean conceptually---for Application's definition to be adjusted to pertain to software that has been deployed within some operating system or other environment?

[sbarnum]

I would object to changing applications defn to software that has been deployed. Applications exist that are not deployed.

I agree with your uneasiness on installedVersionHistory as a property of Application though possibly not for the same reasons as you. I mentioned this at the last meeting but did not go into depth as I did not want to derail getting the needed ability to express application version history info. The root of my uneasiness is that "installation" of an application is a relationship between an Application and some platform such as a Device. What versions of an Application were installed or uninstalled on a Device and when are inherently properties of relationships (Application "installed-on" Device; Application "uninstalled-on" Device) between the Application and the Device. They are not properties simply of an Application. I viewed installedApplicationHistory as kind of a cheat for now that should likely be adjusted in the future. With the current approach a separate Application object instance must be used for every time an application is deployed on different Devices. It would be far more appopriate to define an Application object once for a given application then express thousands of different times it was installed on different Devices with relationships between the Application and the Devices.