R96 and R97 need to be more concrete

[chaals]

It is not really clear in R96, and completely unclear in R97 what the required parameters are. Presumably, the BPI is not required to scale to an infinite number of transactions in a given time period. But 10x? 1000x?

Likewise for availability. Is it necessary to respond in milliseconds? Is it reasonable to timeout and say the thing failed after 1/2 a second, or after 20 seconds?

There will be use cases where higher scalability/availability is required, and BPIs can always meet those. There will also be use cases where scalability and availability necessary are in the order of single-digit number of transactions processed per hour, and 20 minute responses would not cause any problems.

The spec needs to set some basic requirements in order to test interoperability - and these should probably be at the lower end of "usable for real things". Too low, and we leave most systems waiting long after they should have given up and declared that something is wrong. Too high, and people will not bother since it's not necessary - or even actively exclude use cases like running on real world networks of limited capacity from conformance.

[Therecanbeonlyone1969]

@chaals ... I hear you. So let's look at the wording as a whole.

[R96] BPI service orchestration utilized in a BPI MUST have low latency.

Low latency in this context refers to a latency that does not impact the overall system latency of the BPI.

the keywords are "does not impact the overall system latency of the BPI". That means Service orchestration latency << overall system latency. Overall system latency that is acceptable is use case dependent and will be determined by the business users.

Don't you think that is specific enough within the context set by the business?

[R97] BPI service orchestration utilized in a BPI MUST be scalable and highly available.

This requirement ensures that overall system latency is not impacted when volume meaningfully and rapidly changes (scalable) at any given point in time (highly available).

This is similar to the above argument. Scalability and availability determine overall system latency when processing volumes meaningfully -- keyword -- and rapidly -- another keyword -- change. Meaningful and rapidly are use case dependent and their meaning are determined by the business requirements. And "at any given point in time" refers to the high availability of resources that can be spun up at a moment's notice. However long a moment's notice is, is dependent on the business requirements.

This text seeks to strike the delicate balance between specificity in a requirement and avoidance of implementation specifics e.g. numbers or any other pre-judicial statements.

A test harness will provide performance tests of the stack across different performance scenarios. As long as your stack is performant in your target use cases, you are fine. The tests will highlight under which conditions a stack is less performant but still fulfilling the requirements 96 and 97.

[Therecanbeonlyone1969]

@chaals ... per decision in the WG meeting, please, respond to the above response. Otherwise, the issue will be closed at the next WG meeting.

[chaals]

Proposed #127 to address R96. I am not sure that R97 is as easy to fix.

I understand the goal of not providing specific numbers, but instead we provide no concrete guidance.

Providing minimum numbers would at least enable developers and users to share expectations of scalability, and so understand where problems are not going to occur, and under what conditions they need to think some more. This would come at the expense of requiring unnecessarily good performance for some applications, or of not meeting most of the useful goals of implementors.

We could also phrase something a bit like I proposed for R96. Or we could refer to the agreement required by R1 and friends, state that they should (or may or must) contain specific targets, and apply those. In that scenario we could also provide minimum numbers as a default.

[Therecanbeonlyone1969]

Closing. PR Merged