Web Profile: Align on scoping guidance for developers and labs

[mikewhiteman]

Overview: We need to develop scoping guidance for developers and labs to understand what applications are in-scope for testing. Our guidance should likely cover:

• How can a developer/lab determine which applications are in-scope for testing?
• Within those applications, what integrations (e.g., 3p SaaS platforms) are in-scope for testing?
• In what scenarios are APIs invoked by mobile/Quest apps in-scope for testing?

Recommendation: Slack channel has an active discussion on this which I won't try to capture fully here. Let's continue the conversation in Slack until we align on verbiage.

[johntidwell1]

Scoping Guidance / How to Apply This Specification

Scoping a web application test can be a complex and challenging task, as it requires defining the boundaries of the application and determining what needs to be tested. Here is some guidance to help define a reasonable scope while evaluating a web application within the context of this specification.

This specification is designed to be applied to one or more target web application(s) (first party components) and their integrated third-party components, subject to the following considerations:

FIRST-PARTY SCOPING CONSIDERATIONS

• Target Web Application(s): Before testing, define the target web application(s) as a group of components that operate together to provide a logical set of services. For example, if a large business platform offers multiple services such as online dating, instant messaging, and investment banking; each can be scoped and evaluated separately using this specification, with all grouped subcomponents considered in-scope for testing & evaluation.
• Shared Backend Components: First party shared backend components or APIs are considered in the scope if they are utilized by the defined target application(s).

THIRD-PARTY API SCOPING CONSIDERATIONS

• Sensitive Operations: Any third-party (3P) product API that supports sensitive operations, such as Authentication, accessing or mutating user data, & account recovery are within the scope of an ADA web assessment. These 3P APIs will only be subject to web assessment requirements defined within the Authentication, Session Management, & Access Control sections.
• Limited Testing: Testing of these 3P APIs will be limited to components and configurations utilized by the tested application. Other 3P API components and ADA web requirements will be out of scope and will not be tested in relation to 3P Product APIs.

ADDITIONAL CLARIFICATIONS

• Out-of-Scope Components: The following components are explicitly out of scope for this testing:
  • Third-party APIs not utilized by the target application
  • Non-sensitive operations performed by third-party APIs
• Examples and Scenarios: To illustrate the scoping decisions, consider the following examples:
  • A web application uses a third-party authentication API to handle user login. In this case, the authentication API is within scope for testing.
  • A web application uses a third-party analytics gateway to process application performance metrics. As the gateway does not handle sensitive user data, it is out of scope for testing.

[johntidwell1]

@mikewhiteman I think the above guidance can help with "How can a developer/lab determine which applications are in-scope for testing? Within those applications, what integrations (e.g., 3p SaaS platforms) are in-scope for testing?"

I'll let the slack group know to come here / post comments / review the pull request.

With In what scenarios are APIs invoked by mobile/Quest apps in-scope for testing? I think we are already covered here as each of our requirements in the specification guide mention if they are subject to web application, web api, and/or mobile api coverage.

[mikewhiteman]

Target Web Application(s): Before testing, define the target web application(s) as a group of components that operate together to provide a logical set of services. For example, if a large business platform offers multiple services such as online dating, instant messaging, and investment banking; each can be scoped and evaluated separately using this specification, with all grouped subcomponents considered in-scope for testing & evaluation.

This makes sense at a high-level - the remaining consideration would be, if a large application has multiple discrete "products" each with their own logical set of services, which of those top-level products/applications is ultimately in-scope for testing?

If we use the previous example (dating, instant messaging, investment banking) - I imagine that some of the apps may have direct platform integrations (e.g., dating app provides Facebook Login) but other apps may not directly integrate with platforms or touch platform data. Could "Does the application directly interact with platform data or APIs?" be used as a meaningful litmus test for determining if an application in-scope? I could imagine this decision is very clear cut for some apps but perhaps murkier for others (e.g., App A stores platform data in a manner which is accessible to App B).

Lab feedback will definitely be valuable for this one. 😄

[johntidwell1]

@mikewhiteman do we think we want to narrowing scope the standard here to only platform data integrations (products that utilize tech provided information)? Or should it be more broad based in where this ADA standard can be applied? Or we could have a specific set of instructions when utilization of the standard is for a tech provider use case / test?

Applicability for example with the mobile standard is pretty broad, and doesn't narrowly scope to a platform use context. "This document is intended for system and application administrators, security specialists, auditors, help desk, platform deployment, and/or DevOps personnel who plan to develop, deploy, assess, or secure mobile applications."

[mikewhiteman]

To your point, we may not need to scope directly within the technical standard itself, since this standard could be applied for many different use cases internal/external to the ADA.

But I imagine we will need to provide guidance to labs (perhaps independent of the standard) on how to approach complex apps if we want to drive consistency in testing. The mobile profile benefits from having fairly discrete and testable products (e.g., a mobile app) but the web side tends to be a bit more complex, as you highlighted in the guidance above.

On Thu, Aug 1, 2024 at 5:16 PM johntidwell1 @.***> wrote:

@mikewhiteman https://github.com/mikewhiteman do we think we want to narrowing scope the standard here to only platform data integrations (products that utilize tech provided information)? Or should it be more broad base in where this ADA standard can be applied? Or we could have a specific set of instructions when utilization is for a tech provider use case / test?

Applicability for example with the mobile standard is pretty broad, and doesn't narrowly scope to a platform use context. "This document is intended for system and application administrators, security specialists, auditors, help desk, platform deployment, and/or DevOps personnel who plan to develop, deploy, assess, or secure mobile applications."

— Reply to this email directly, view it on GitHub https://github.com/appdefensealliance/ASA-WG/issues/85#issuecomment-2264014049, or unsubscribe https://github.com/notifications/unsubscribe-auth/ALCZ3I6SKVYJRY4FR4O4MVDZPKQRXAVCNFSM6AAAAABLRDUOKWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDENRUGAYTIMBUHE . You are receiving this because you were mentioned.Message ID: @.***>

[mikewhiteman]

@alex941115 @8radree are we comfortable aligning on scoping guidance post 1.0? It’s important we provide the labs with actionable guidance but it’s likely an independent discussion from the standard itself.