update time and submit time sequencing

[boguwei]

I wanted to open a discussion around forcing update_time to be equivalent to submit_time for first PUT. (ref here)

There could be a variety of reasons where a receiving USS misses the first announcement of an operation from another USS. The effect of this requirement would result in the receiving USS rejecting all subsequent updates to the operation from the other USS? How does the receiving USS knows when an operation is first announced, versus an update?

[issmith1]

hi Bogu, thanks for bringing this up. Indeed if a first announcement is missed we don't want to miss subsequent emergency messages about that operation.

For PUT uss/operation/1234 the validator logic can be as follows. Would this make sense?

if (gufi.1234 exists) { // this is an Op Modification validate submit_time < update_time do Op Mod (update db) } else { // this is a New Op validate submit_time = update_time do Add Op (insert to db) }

For PUT Message similar logic too.

[boguwei]

for the logic you described, if we missed the first PUT operation, then gufi.1234 would not exist in our database. when we receive the second PUT operation, we would assume that since we did not have the gufi, it is the first PUT for the op. validation would fail at that point because the PUT in fact is not.

am I interpreting the logic you described right?

[issmith1]

correct, if the intended OpMod was correct WRT update and submit times, then the validation would fail. We would get no Op models. This is a tough trade off.

[boguwei]

to ask a more underlying question then: why do we specifically prescribe a greater than comparison for updates? or put another way, if I were to get an op update which had the update and submit times be exactly the same, what's the intended concern on my part as the receiving party?

i get that it could be problematic if the update time was before the submit time, but i'm up until now I haven't been too worried about the distinction that update > submit times.

[esparano]

I guess it does seem odd that the receiver of the operation is the one to determine whether it's a submit or update.

And while this type of scenario shouldn't arise because the receiver should have already gotten the first PUT, is there any way to correct this type of miscommunication once it occurs, or are both participants stuck?

[issmith1]

Client-determined intent to create or modify would support, in a straightforward way, both validation robustness and lost data handling.

@nasajoey can speak better on this, but In the earlier parts of this UTM project (which had a significantly different requirements) we had interfaces with both POST- and PUT-model. Because the early prototype of the USS-API interface had no use cases for POST, the minimal set of APIs was chosen and the POST was dropped. As we were rapidly trying various approaches and data models, some of which were abandoned, the prototype had less data validation, and fewer validation tests.

Now that we have strict validation we are bumping into this issue. The strict validation keeps all USSs talking to each other -- albeit without lost data handling -- which can allow us to better test and evaluate the concepts.

[boguwei]

@esparano and I actually ran into this specific corner case during the shakedown, which is how it came up. it would be good if we could come to some consensus for Sprint 4 next week, since this is artificially bottlenecking our ability to interoperate.

the way I see it, there are 3 options we could move forward on (please correct if any of the assumptions in this are wrong):

1. keep the spec as is, but loosen validations
2. change the spec to be >= in all circumstances, and change the validation to maintain strict conformance
3. change the spec to allow clients to determine whether the PUT is a create or an update, and change validations to maintain strict conformance.

my preference would be for no. 2, because (a) I'm still not sure there is a functional impact to having a >= check in all cases, and (b) client-determined intent would involve putting in additional state to the operation which would require further validation in and of itself?

[issmith1]

Since we have pinned the API at the beginning of SIM 4 Sprint, one #1 is the only short term possibility.

[issmith1]

@nasajoey Please determine if we can relax the current validations as per #1, or if we will be considering a longer term API change which will be managed with the next API unpinning.

[nasajoey]

I think the historical reasons for this requirement may be moot due to use of grid for discovery. I think @boguwei 's second option above is good, with the requirement that a USS MUST use submit == update for their first announcement. That may prove useful someday and it doesn't hurt this issue. It can still be a trigger if the first time a USS gets a PUT for an op and submit_time <update to do something. Or the receiver may decide that it does the same thing no matter what as long as submit <= update.

Thoughts? go with second option with the added requirement that we ask senders to do the right thing with these times?

[nasajoey]

@issmith1 , @esparano , @boguwei please check my logic and if we can converge, we'll close this off.

[nasajoey]

To more formalize, I think we make the == a SHOULD rather than a MUST. So the two requirements would be:

1. For OperationPlan data, submit_time MUST be less than or equal to update_time
2. For OperationPlan data, on the first announcement of an Operation, submit_time SHOULD be equal to update_time.

[boguwei]

i think that makes sense. to clarify, would that mean the validation suites will only fail if no. 1 is violated?

[nasajoey]

@boguwei , that is correct... for validation and tests, USSs will be checked against 1 only.

[utmnasa]

The USS Spec (document, not swaggerspec) says The key words 'MUST,' 'MUST NOT,' 'REQUIRED,' 'SHALL,' 'SHALL NOT,' 'SHOULD,' 'SHOULD NOT,' 'RECOMMENDED,' 'NOT RECOMMENDED,' 'MAY,' and 'OPTIONAL' in this document are to be interpreted as described in [RFC2119].

[nasajoey]

@utmnasa that is correct. I believe my 2 requirements are interpreted correctly using RFC2119. Please correct if not.

[issmith1]

@nasajoey PR updated as per #1 and #2 from 5h ago.

[esparano]

I think #2 makes sense... I'd rather have USSs maintain strict validation so that everyone is in sync. I think that would fix the issue where we got stuck rejecting the operation.

[issmith1]

thank you @boguwei for contributing the solution