Extensibility

[martinthomson]

I'm in the process of writing up a few DAP (upload) extensions and realized that if they are going to be generically useful, then it would make sense to have corresponding changes to be included in task configurations. For this draft, that means having some means to include that configuration in the creation of the task, but there are no extensibility points in this document that would allow that to happen. Can an extension mechanism be defined?

Most likely, this would have to be like upload extensions, where ignorance of the extension would invalidate the entire request.

I would also recommend that the DP configuration be moved to an extension. Then you can move the definition of the specifics to the DP draft and remove one potentially-awkward interdependency.

[cjpatton]

I'm in the process of writing up a few DAP (upload) extensions and realized that if they are going to be generically useful, then it would make sense to have corresponding changes to be included in task configurations. For this draft, that means having some means to include that configuration in the creation of the task, but there are no extensibility points in this document that would allow that to happen. Can an extension mechanism be defined?

I had this exact thought recently. Could you take a look at https://github.com/ietf-wg-ppm/draft-ietf-ppm-dap-taskprov/pull/84, specifically the new "Extending this Document" section? I think this is a step in the right direction, although I think there may still be some open questions, e.g., how precisely does one "extend" the DpConfig structure.

Most likely, this would have to be like upload extensions, where ignorance of the extension would invalidate the entire request.

I believe this is covered by the opt-in mechanism: https://ietf-wg-ppm.github.io/draft-ietf-ppm-dap-taskprov/draft-ietf-ppm-dap-taskprov.html#section-4.4-4

I would also recommend that the DP configuration be moved to an extension. Then you can move the definition of the specifics to the DP draft and remove one potentially-awkward interdependency.

Are you thinking of an extension to DAP or to this document?

[martinthomson]

I don't think that #84 really works for me. As far as I can see, you aren't adding explicit extension points (in TLV, say), but saying "add stuff to the various configs as needed", which creates a coordination problem. If you have multiple definitions for extension, how do they ensure that both can be used? That is, which addition comes first?

That's why we end up with TLVs (though with a requirement to understand the extension, you might reasonably drop the "L" part).

(I was thinking that it might be nice to have a taskprov extension in the DP draft.)

[cjpatton]

I don't think that #84 really works for me. As far as I can see, you aren't adding explicit extension points (in TLV, say), but saying "add stuff to the various configs as needed", which creates a coordination problem. If you have multiple definitions for extension, how do they ensure that both can be used? That is, which addition comes first?

Do you mean multiple definitions of "an extension"? How might this happen? Isn't the IANA registry supposed to ensure that a codepoint points to one and only one document?

Or do you mean multiple definitions of "extension"?

That's why we end up with TLVs (though with a requirement to understand the extension, you might reasonably drop the "L" part).

I'm pretty sure TLV stands for Tag-Length-Value; let me know if you mean something else.

In fact, the DAP document does define TLV-style extension points: for DAP batch modes, VDAFs, and DP mechanisms. (The batch mode registry is in the DAP document; the VDAF registry is in the VDAF document; and the DP mechanism registry is here.) Note that the L (i.e., the length) actually comes before the T (i.e., the tag) in our case; see TaskConfig in #84. We could easily change this if we wanted.

So how do you add a DP mechanism? Here is the extension point for DP mechanisms (excerpted from the #84):

enum {
    reserved(0), /* Reserved for testing purposes */
    none(1),
    (255)
} DpMechanism;

struct {
    DpMechanism dp_mechanism;
    select (DpConfig.dp_mechanism) {
        case none: Empty;
    };
} DpConfig;

Let's say RFC 1234 defines a new DP mechanism. The document is supposed to describe how it modifies DpMechanism. In particular, it would define its codepoint like so:

enum {
    cool_guy(0x42), (255)
} DpMechanism;

It also needs to describe how it modifies DpConfig. That is, it would specify the payload for the case cool_guy arm in the select block. I'm not sure what the best way to spell this would be. Perhaps something like this:

The payload of the cool_guy variant of the DpConfig following is structured as follows:

struct {
    Cool stuff<0..2^16>;
} CoolGuyConfig;

What do you think?

(I was thinking that it might be nice to have a taskprov extension in the DP draft.)

A bit of context as to why we define DpMechanism here rather than in the DP draft: There is no active work on the DP draft at the moment. The last time we talked about it, the WG was split between doing nothing at all and taking an entirely different approach. In the meantime, we already knew we need the extension point in the task config, so we went ahead and added it to this draft. It has existed well before adoption.

Now, to your point: Now that we have consensus to work on taskprov, we could do something like this:

• Add a section to DAP that defines BatchModeConfig (i.e., taskprov encoding)
• Add a section to VDAF that defines VdafConfig (i.e., taskprov encoding)
• Refresh the DP draft, moving the DpMechanism there as well as the DpConfig (i.e., taskprov encoding) (or https://github.com/martinthomson/dap-dp-ext could do this?)

I think the problem we'd have is one of chickens and eggs. My understanding is that RFCs can't have normative circular dependencies. For example, taskprov normatively uses DAP, so DAP can't normatively use taskprov. Perhaps that's not the case? It would definitely make things more complicated when we have to update drafts, although hopefully we're pretty close to done with DAP.

PR #84 solves this differently: documents that extend taskprov have a special section that describes how they extend it.

[cjpatton]

@martinthomson FYI I'm merging #84, as I think there are some improvments that are helpful in any case. We can fix stuff as needed as part of this issue.

[cjpatton]

2024/10/24 interim: I misunderstood the issue that @martinthomson is raising here. What's actually needed is an extension point for TaskConfig itself. For example, the parties may want to agree on the identity (e.g., a URI) of whichever party gets the aggregate result. This could be expressed as an extension for taskprov.

So we'll create a PR that adds an extension point to TaskConfig (similar to the report extensions in DAP) and ask IANA to create a registry for it.

[martinthomson]

I recommend that the DP mechanisms be removed from the taskprov draft in favor of being specified in the document that first defines a real DP mechanism. The extension mechanism should be enough for DP.

I can see two ways of dealing with new DP mechanisms:

1. Define a new extension for each and sketch out some rules for how to handle cases where there are multiple mechanisms (apply all of them or don't allow multiple both seem reasonable, though I'd probably choose the latter).
2. Define a new extension for all DP mechanisms and have that contain content that is based on the choice of mechanism. That sort of nesting would match the current design, but would also run afoul of recommendations in RFC 9170.

[cjpatton]

Thanks @martinthomson! I've filed #88 which is more narrowly scoped on this issue ... I'll reference your comment there.