[rfc 112] Should resolver or environment assert requirement.matches

[bjhargrave]

Original bug ID: BZ#2059 From: david.savage@paremus.com Reported version: R5

[bjhargrave]

Comment author: david.savage@paremus.com

Discussion on bug BZ#2039 has uncovered yet another subtly of the current OBR API design.

When I've been working on this API I've been assuming that the resolver is the authority on whether a requirement matches a capability - with all the messy namespace rules, mandatory attribute checks and filter parsing that that entails.

However I think Tom (and possibly Richard) are assuming that the resolver is a simple constraint solving service that take a list of capabilities and try to merge them based on the uses constraints. The messy problems of understanding namespaces are then the environments problem to sort out during a call to findProviders.

Either option will work, but we need to decide on one. The reason for my initial assumption came from my interpretation of the requirements in the OBR RFC namely:

4.1 -> Handle dependency resolution so that bundles can be deployed without generating errors 4.3 -> Must handle all the requirements/capabilities and their directives as defined in the OSGi R4 specifications

In the environment does the matching interpretation the API makes it possible to "Handle dependency resolution" but the OBR implementation doesn't have handle it.

I do not think that a resolver must be the final authority on namespaces - it can delegate to other services (unspecced for now) but at least everyone can go to one service to figure out if a requirement matches a capability.

This issue primarily effects subsystems or other management agent implementors.

[bjhargrave]

Comment author: @tjwatson

I am surprised by this bug, but it does explain some of the different directions that we each seem to be taking the API in.

I will point out the javadoc of findProviders

/**

• Find any capabilities that can potentially provide a match to the supplied
• requirements.
• A resolver should use the iteration order or the returned capability
• collection to infer preference in the case where multiple capabilities
• match a requirement. Capabilities at the start of the iteration are implied
• to be preferred over capabilities at the end.
• @ param requirements
• the requirements that a resolver is attempting to satisfy
• @ return an immutable collection of capabilities that match the supplied
• requirements */

The return statement clearly states that this is an immutable collection of capabilities that MATCH the supplied requirements. The findProviders on Repository states the same thing. I think it would be pretty disappointing if a Repository handed back "providers" for a requirement that could never match the requirement. Same goes for Environment in my opinion.

[bjhargrave]

Comment author: david.savage@paremus.com

I am surprised by this bug, but it does explain some of the different directions that we each seem to be taking the API in.

Yep, I think I'd glossed over it when we had the discussion on Requirement.matches but it does explain the differences.

I think either could work, we just need to decide.

I will point out the javadoc of findProviders

/**

• Find any capabilities that can potentially provide a match to the supplied
• requirements.
• A resolver should use the iteration order or the returned capability
• collection to infer preference in the case where multiple capabilities
• match a requirement. Capabilities at the start of the iteration are implied
• to be preferred over capabilities at the end.
• @ param requirements
• the requirements that a resolver is attempting to satisfy
• @ return an immutable collection of capabilities that match the supplied
• requirements */

The return statement clearly states that this is an immutable collection of capabilities that MATCH the supplied requirements. The findProviders on Repository states the same thing. I think it would be pretty disappointing if a Repository handed back "providers" for a requirement that could never match the requirement. Same goes for Environment in my opinion.

Right but I think I posted a comment on the matches requirement email thread where I showed how you could implement matches by delegating to the resolver.

For the record here:

boolean matches(Requirement req, Capability cap) { Environment env = buildSingleCapabilityEnvironment(cap); return !resolver.resolve(env, makeOptional(req)).isEmpty(); }

This creates an environment with one capability and puts an optional requirement in to be resolved. If the matches check succeeds in the resolver then a non empty wiring is returned.

But this obviously won't work if the resolver delegates back to the environment.

[bjhargrave]

Comment author: david.savage@paremus.com

Right but I think I posted a comment on the matches requirement email thread where I showed how you could implement matches by delegating to the resolver.

The Requirement, Environment or Repository implementations could then all delegate to the Resolver using single entry environments to check if capabilities matched. They don't have to, they could implement checks themselves but it would provide one authority.

For the record here:

boolean matches(Requirement req, Capability cap) { Environment env = buildSingleCapabilityEnvironment(cap); return !resolver.resolve(env, makeOptional(req)).isEmpty(); }

This creates an environment with one capability and puts an optional requirement in to be resolved. If the matches check succeeds in the resolver then a non empty wiring is returned.

But this obviously won't work if the resolver delegates back to the environment.

[bjhargrave]

Comment author: Richard Hall <heavy@ungoverned.org>

To me it seems that you missed the point of the original design. The Environment + Resolver == "complete resolver". The reason to separate out the Environment was to provide a hook so that other clients could wrap their own state into something that is usable by the resolver. For example, Equinox stores it state differently than Felix and both are different than OBR. It also provides them hooks to change the priority, etc.

[bjhargrave]

Comment author: david.savage@paremus.com

To me it seems that you missed the point of the original design. The Environment + Resolver == "complete resolver". The reason to separate out the Environment was to provide a hook so that other clients could wrap their own state into something that is usable by the resolver. For example, Equinox stores it state differently than Felix and both are different than OBR. It also provides them hooks to change the priority, etc.

Right, I was looking at the requirements.

I think the Environment + Resolver design can work but I'm not sure if it meets others expectations. If we can close this this week and the subsystems implementors are happy then we have a design.

I agree with the comments on state, but for me the issue here is about asserting a match statement.

[bjhargrave]

Comment author: @tjwatson

Environment has not only the state information but access to a "database" of capabilities. It is in a much better position to optimize the ldap searching over the available capabilities than the resolver. Otherwise the resolver has to build up internal data structures that contain a "database" of possible capabilities for each resolver operation. This would be expensive since the resolver has no state and must throw out the "database" after each resolve operation.

[bjhargrave]

Comment author: Richard Hall <heavy@ungoverned.org>

To me it seems that you missed the point of the original design. The Environment + Resolver == "complete resolver". The reason to separate out the Environment was to provide a hook so that other clients could wrap their own state into something that is usable by the resolver. For example, Equinox stores it state differently than Felix and both are different than OBR. It also provides them hooks to change the priority, etc.

Right, I was looking at the requirements.

I think the Environment + Resolver design can work but I'm not sure if it meets others expectations. If we can close this this week and the subsystems implementors are happy then we have a design.

I agree with the comments on state, but for me the issue here is about asserting a match statement.

Just to be clear, it is already implemented this way in the Felix framework. The OBR implementation will have a different Environment implementation than the Felix framework since it needs to account for repositories, etc., but I expect it will use the same filter implementation/capability indexing. So, from an implementation point of view, it should be fairly straightforward. From a user's point of view, they'll never know.

[bjhargrave]

Comment author: david.savage@paremus.com

Environment has not only the state information but access to a "database" of capabilities. It is in a much better position to optimize the ldap searching over the available capabilities than the resolver. Otherwise the resolver has to build up internal data structures that contain a "database" of possible capabilities for each resolver operation. This would be expensive since the resolver has no state and must throw out the "database" after each resolve operation.

Right, it's a nieve implementation but again I'm not trying to say how to optimize it.

To do a repository slightly better I'd suggest off the top of my head, something like:

String sql = "select * from capabilities where " + ldapToSQL(req);

ResultSet resultset = executeQuery(sql);

for (result : resultset) { Capability cap = fromResult(result); if (matches(req, cap)) { // add cap to list
} }

I'm sure there are other options. This has the benefit of using fast lookups via dedicated kit, but saves encoding namespace matching rules into sql which seems like a really bad idea...

[bjhargrave]

Comment author: david.savage@paremus.com

Just to be clear, it is already implemented this way in the Felix framework. The OBR implementation will have a different Environment implementation than the Felix framework since it needs to account for repositories, etc., but I expect it will use the same filter implementation/capability indexing. So, from an implementation point of view, it should be fairly straightforward. From a user's point of view, they'll never know.

Understood, I think this is purely an issue from the point of view of who ever has to build an environment. Which is subsystems etal I think. End users won't see the difference.

From the requirements I interpreted them to mean an implementation would "handle dependency resolution" but we could also read this as the design must "handle dependency resolution" - the requirement is ambiguous in this respect. As we need to vote on the design though I think we need to agree on this point.

We probably also need some words in the RFC document to cover it once we come to a conclusion.

[bjhargrave]

Comment author: Richard Hall <heavy@ungoverned.org>

Understood, I think this is purely an issue from the point of view of who ever has to build an environment. Which is subsystems etal I think. End users won't see the difference.

From the requirements I interpreted them to mean an implementation would "handle dependency resolution" but we could also read this as the design must "handle dependency resolution" - the requirement is ambiguous in this respect.

You seem to equate "dependency resolution" with "evaluating a filter". This is not the case in my mind. "Dependency resolution" means "selecting a provider for a given requirement" to me. Thus, the current design is fine.

As we need to vote on the design though I think we need to agree on this point.

We probably also need some words in the RFC document to cover it once we come to a conclusion.

Well, let's see who else shares the same confusion. If it's just you, then maybe it is not so problematic. :-)

[bjhargrave]

Comment author: david.savage@paremus.com

You seem to equate "dependency resolution" with "evaluating a filter". This is not the case in my mind. "Dependency resolution" means "selecting a provider for a given requirement" to me. Thus, the current design is fine.

For the record for me dependency resolution implies "selecting a valid set of providers to satisfy the requirements", but as the spec requirement is ambiguous we probably need to decide on whether we're happy with the design?

Well, let's see who else shares the same confusion. If it's just you, then maybe it is not so problematic. :-)

Yep that obviously simplifies things :)

Obviously anyone is free to do something custom and I probably will, but as long as the spec is good enough for all then it should be useful? Of course the risk is it doesn't quite satisfy anyones requirements which makes it pointless.

As I say the other interpretation will work but it put's more effort on environment/repository builders to process the matches. For me resolver implementers are likely the only people who care about some of the more esoteric aspects of osgi requirements matching.

[bjhargrave]

Comment author: Richard Hall <heavy@ungoverned.org>

Obviously anyone is free to do something custom and I probably will, but as long as the spec is good enough for all then it should be useful? Of course the risk is it doesn't quite satisfy anyones requirements which makes it pointless.

Well, I've already proven it is useful for me, since I've been using this design for quite a while now. So, at least we know it isn't totally pointless. :-)

As I say the other interpretation will work but it put's more effort on environment/repository builders to process the matches.

Agreed, but it also puts more power in their hands since it gives them a much bigger hook. For simple filter evaluation, more than likely they will just be able to grab an implementation of that functionality from anywhere, they won't have to implement that themselves unless they specifically need to customize it.

For me resolver implementers are likely the only people who care about some of the more esoteric aspects of osgi requirements matching.

Yeah, we already have a use case to the contrary when it comes to ordering for priority, which the environment can do however it wants. Again, more power, but admittedly at potentially more cost for the Environment implementer, but not the Resolver implementer.

Still, a simple Environment will not be too difficult to implement.

[bjhargrave]

Comment author: david.savage@paremus.com

Well, I've already proven it is useful for me, since I've been using this design for quite a while now. So, at least we know it isn't totally pointless. :-)

Understood, sorry didn't mean the functionality was pointless but the spec could be if it doesn't do what others want.

As I say the other interpretation will work but it put's more effort on environment/repository builders to process the matches.

Agreed, but it also puts more power in their hands since it gives them a much bigger hook. For simple filter evaluation, more than likely they will just be able to grab an implementation of that functionality from anywhere, they won't have to implement that themselves unless they specifically need to customize it.

True, I just thought that place would be the resolver but it could be somewhere else, though I think the immediate issue will be is that other service specced - doesn't have to be this spec though.

For me resolver implementers are likely the only people who care about some of the more esoteric aspects of osgi requirements matching.

Yeah, we already have a use case to the contrary when it comes to ordering for priority, which the environment can do however it wants. Again, more power, but admittedly at potentially more cost for the Environment implementer, but not the Resolver implementer.

Still, a simple Environment will not be too difficult to implement.

Hopefully we can bottom this out this week, my intention is to get the rest of the rfc document updated this week bar these two or three issues we're discussing now. There's an eeg call next week so if there's no other updates to these issues I guess we can close them then and finish this off.

[bjhargrave]

Comment author: John Ross <jwross@us.ibm.com>

As an Environment and Repository implementor, it has always been my understanding that the returned capabilities must match the provided requirement, so no surprise here for me. In fact, given a requirement, I don't see any sense in returning anything that doesn't match it and, in any case, see very little value in having an environment or repository that does not provide this filtering but would (apparently) always simply return everything it had.

I don't see any value in requiring Repository, Environment, and Resolver to do the matching. I think putting the responsibility on the resolver instead would require a not insignificant API change. I'm okay with the way things stand.

In terms of the RFC requirements, I always understood them as something the API as a whole must solve, not any particular part of the API.

[bjhargrave]

Comment author: david.savage@paremus.com

As an Environment and Repository implementor, it has always been my understanding that the returned capabilities must match the provided requirement, so no surprise here for me. In fact, given a requirement, I don't see any sense in returning anything that doesn't match it and, in any case, see very little value in having an environment or repository that does not provide this filtering but would (apparently) always simply return everything it had.

I certainly wouldn't expect it to return everything, but simply a best attempt.

Namespace rules have the potential to get quite complex and my concern is the potential for chaos if different requirements, environments and repositories impls from different vendors can't agree on matches because they've each got slightly different bugs/interpretations in the matches check.

At the moment (I think) we're in quite a good position in that only the "requirement -> filter" and "capability -> mandatory" directives have any effect on the matches check. Until of course someone points out some really obscure matching rule with fragments or bundles - but I had a quick check and I think the only one I could see as an issue is "fragment-attachment" but this get's modelled as an optional osgi.wiring.host capability so is ok.

If we could agree to fix this and say for ever more this is what "matches" means then I have less of a problem with delegating the problem to the edge but if we ever change it then I think all bets are off.

I don't see any value in requiring Repository, Environment, and Resolver to do the matching. I think putting the responsibility on the resolver instead would require a not insignificant API change. I'm okay with the way things stand.

In terms of the RFC requirements, I always understood them as something the API as a whole must solve, not any particular part of the API.

Great if everyone else is ok and if we can nail down matches in terms of definition above then I'm happy with this approach too.

[bjhargrave]

Comment author: david.savage@paremus.com

I just noticed whilst writing up the changes for varargs/isEffective where the source of the ambiguity is in this issue

/**

• Find any capabilities that can potentially provide a match to the supplied
• requirements.

I've been focussing on the word POTENTIALLY.

The return statement clearly states that this is an immutable collection of capabilities that MATCH the supplied requirements. The findProviders on Repository states the same thing. I think it would be pretty disappointing if a Repository handed back "providers" for a requirement that could never match the requirement. Same goes for Environment in my opinion.

Have we moved any closer to a resolution on this? I see the options as:

1) Resolver defines matches 2) Specification defines matches as requirement:filter + capability:mandatory 3) Environments, repositories and requirements all define matches however they like.

My preferences is for 1 as it leaves the option open to extend matches later though (initially) unspecced services that plug into the resolver. Though some thinking is also needed about what it means to redefine matches.

[bjhargrave]

Comment author: @tjwatson

Have we moved any closer to a resolution on this?

I thought we had agreed that the findProviders methods would be responsible for returning Capabilities that match the Requirement. I think it would be terribly confusing if a Repository would return random Capabilities from Repository.findProviders (same for Environment.findProviders). What is the point if the caller cannot depend on valid/matching Capabilities being returned for the requirement. To me this is like asking the service registry for a service with a filter but then still requiring the caller to check the service properties to make sure that they match and also forcing the caller to do an instanceof check just to be certain the service is of the proper type.

I see the options as:

1) Resolver defines matches

I don't like as stated before.

2) Specification defines matches as requirement:filter + capability:mandatory

Matches has three parts as stated in the javadoc of Requirement.matches:

• The specified capability has the same name space as this requirement.
• The filter specified by the filter directive of this requirement matches the attributes of the specified capability.
• The requirement directives and the capability directives that apply to the name space are satisfied.

It seems we need to lock down the directives to be only the mandatory directive on Capabilities for the osgi.wiring.package, osgi.wiring.bundle and osgi.wiring.host namespaces.

3) Environments, repositories and requirements all define matches however they like.

My preferences is for 1 as it leaves the option open to extend matches later though (initially) unspecced services that plug into the resolver. Though some thinking is also needed about what it means to redefine matches.

My preference is to lock down the definition of matches for Repositories and leave it open ended for Environments. The Environment has the power to do what ever it wants in findProviders. Perhaps it has some other definition of matches it wants to use for the results it gets back from the Resolver.resolve method. The point is the Environment is an implementation detail of the Resolver client. We should not restrict what an Environment can or wants to do.

[bjhargrave]

Comment author: david.savage@paremus.com

Matches has three parts as stated in the javadoc of Requirement.matches:

• The specified capability has the same name space as this requirement.
• The filter specified by the filter directive of this requirement matches the attributes of the specified capability.
• The requirement directives and the capability directives that apply to the name space are satisfied.

It seems we need to lock down the directives to be only the mandatory directive on Capabilities for the osgi.wiring.package, osgi.wiring.bundle and osgi.wiring.host namespaces.

That makes sense to me, if we can lock it down then I'm less worried about delegating this problem to the edge.

3) Environments, repositories and requirements all define matches however they like.

My preferences is for 1 as it leaves the option open to extend matches later though (initially) unspecced services that plug into the resolver. Though some thinking is also needed about what it means to redefine matches.

My preference is to lock down the definition of matches for Repositories and leave it open ended for Environments.

Just to be clear it's Repositories and Requirements as they both define a matches check. If we've locked down matches for the osgi wiring as stated above then I could live with this.

But for extension namespaces we're basically saying this is going to lead to non portable code as only the vendor that defines the namespace is likely to define matches "correctly" and worse what if two vendors define a similarly named namespace that behaves differently. All at the expense of a matches check in the resolver which we've already been discussing in a separate issue is not the expensive part of the resolve operation.

To "fix" one of these problems we could suggest that vendors use reverse dns naming to avoid potential problems later?

The Environment has the power to do what ever it wants in findProviders. Perhaps it has some other definition of matches it wants to use for the results it gets back from the Resolver.resolve method. The point is the Environment is an implementation detail of the Resolver client. We should not restrict what an Environment can or wants to do.

Agreed, I think extensions of namespaces has a lot of interesting possibilites I'm just not convinced the Environment is the right place to do it, but there we go.

If we agree on matches for the wiring packages then that seems like a way to start and if we accept that environments, repositories and requirements are not portable for namespaces outside of the spec then it seems like a missed opportunity but I can write this up and we can finish this rfc.

[bjhargrave]

Comment author: John Ross <jwross@us.ibm.com>

Have we moved any closer to a resolution on this? I thought we had agreed that the findProviders methods would be responsible for returning Capabilities that match the Requirement. I think it would be terribly confusing if a Repository would return random Capabilities from Repository.findProviders (same for Environment.findProviders). What is the point if the caller cannot depend on valid/matching Capabilities being returned for the requirement. To me this is like asking the service registry for a service with a filter but then still requiring the caller to check the service properties to make sure that they match and also forcing the caller to do an instanceof check just to be certain the service is of the proper type. I see the options as:

1) Resolver defines matches I don't like as stated before. 2) Specification defines matches as requirement:filter + capability:mandatory Matches has three parts as stated in the javadoc of Requirement.matches:

• The specified capability has the same name space as this requirement.
• The filter specified by the filter directive of this requirement matches the attributes of the specified capability.
• The requirement directives and the capability directives that apply to the name space are satisfied. It seems we need to lock down the directives to be only the mandatory directive on Capabilities for the osgi.wiring.package, osgi.wiring.bundle and osgi.wiring.host namespaces. 3) Environments, repositories and requirements all define matches however they like.

My preferences is for 1 as it leaves the option open to extend matches later though (initially) unspecced services that plug into the resolver. Though some thinking is also needed about what it means to redefine matches.

My preference is to lock down the definition of matches for Repositories and leave it open ended for Environments. The Environment has the power to do what ever it wants in findProviders. Perhaps it has some other definition of matches it wants to use for the results it gets back from the Resolver.resolve method. The point is the Environment is an implementation detail of the Resolver client. We should not restrict what an Environment can or wants to do.

I don't understand why an implementation of Environment or Repository would ever return a Capability for which requirement.matches(capability) == false.

I don't see how an Environment or Repository implementor can avoid having to also implement Requirement, whose matches method will be constrained as described by the spec (Javadoc).

To give Environment or Repository freedom to define "matches" however they like seems to imply the same freedom in requirement.matches.

I can see how an Environment, in certain cases, might want to restrict access to capabilities from certain repositories, but this does not affect requirement.matches. Again, I can't imagine a use case where Environment provides a capability for which requirement.matches == false.

Do (1) and (3) imply Requirement.matches will go away?

[bjhargrave]

Comment author: @tjwatson

I don't understand why an implementation of Environment or Repository would ever return a Capability for which requirement.matches(capability) == false.

I don't see how an Environment or Repository implementor can avoid having to also implement Requirement, whose matches method will be constrained as described by the spec (Javadoc).

To give Environment or Repository freedom to define "matches" however they like seems to imply the same freedom in requirement.matches.

Lets just define what matches means and be done with it.

I can see how an Environment, in certain cases, might want to restrict access to capabilities from certain repositories, but this does not affect requirement.matches. Again, I can't imagine a use case where Environment provides a capability for which requirement.matches == false.

My point is Environment can do what ever it wants to because it is an implementation detail of the Resolver caller. Believe me I cannot think of a reason Environment would want to return anything that does not actually match the capability. But who cares, the environment is in charge and should be able to do what ever it wants. There is no way to write a CT for an implementation of Environment since the implementation of it is not shared (in a standard way).

What I do want is for the Resolver to simply trust what the Environment is returning as capabilities and not have to do any additional matching operations.

Do (1) and (3) imply Requirement.matches will go away?

Not sure why it would. I don't want to discuss this point since I think options 1 and 3 do not make sense to begin with.

[bjhargrave]

Comment author: Richard Hall <heavy@ungoverned.org>

But for extension namespaces we're basically saying this is going to lead to non portable code as only the vendor that defines the namespace is likely to define matches "correctly" and worse what if two vendors define a similarly named namespace that behaves differently. All at the expense of a matches check in the resolver which we've already been discussing in a separate issue is not the expensive part of the resolve operation.

Well, if people created extension namespaces that had some sort of unique matching rules, then it doesn't really matter who does the matching, because there would be no way to do this in a portable way, unless I'm missing something. For example, if the resolver did the matching, there would be no way for a standard resolver to know how to do custom matching for custom namespaces.

[bjhargrave]

Comment author: John Ross <jwross@us.ibm.com>

Agreed, I think extensions of namespaces has a lot of interesting possibilites I'm just not convinced the Environment is the right place to do it, but there we go. If we agree on matches for the wiring packages then that seems like a way to start and if we accept that environments, repositories and requirements are not portable for namespaces outside of the spec then it seems like a missed opportunity but I can write this up and we can finish this rfc.

Not sure what we're agreeing to here, but it sounds like the following.

(1) Repositories.findProviders must return only Capabilities for which Requirement.matches(Capability) == true.

(2) Environments.findProviders may return Capabilities for which Requirement.matches(Capability) == false. In which case, the caller of Resolver.resolve needs to be prepared for an undeployable result. Since we expect the Environment and Resolver client to "know" each other, this is not a concern.

(3) Resolver is not required to perform any further matching of the results of Environment.findProviders in order to compute a resolution.

Is that right?

[bjhargrave]

Comment author: Richard Hall <heavy@ungoverned.org>

Agreed, I think extensions of namespaces has a lot of interesting possibilites I'm just not convinced the Environment is the right place to do it, but there we go. If we agree on matches for the wiring packages then that seems like a way to start and if we accept that environments, repositories and requirements are not portable for namespaces outside of the spec then it seems like a missed opportunity but I can write this up and we can finish this rfc.

Not sure what we're agreeing to here, but it sounds like the following.

(1) Repositories.findProviders must return only Capabilities for which Requirement.matches(Capability) == true.

(2) Environments.findProviders may return Capabilities for which Requirement.matches(Capability) == false. In which case, the caller of Resolver.resolve needs to be prepared for an undeployable result. Since we expect the Environment and Resolver client to "know" each other, this is not a concern.

(3) Resolver is not required to perform any further matching of the results of Environment.findProviders in order to compute a resolution.

Is that right?

I think so (although I see no good reason for 2), but let me add...

My resolver implementation makes assumptions that the returned candidates from Environment do match in some capacity. For example, if the namespace of the requirement is osgi.wiring.package then it might assume that any matching capabilities are from the same namespace and have a package name in their attributes.

Does this really matter, though? I don't think so, since we only need to worry about standard behavior here. If people want to do custom stuff, then it is they who should worry about whether or not it will work with standard implementations of the other pieces. Regardless, I thought I would point it out.

[bjhargrave]

Comment author: david.savage@paremus.com

Just to brain dump some (hopefully) clarifying thoughts quickly as this issue thread is getting kind of messy.

• Namespaces are a grouping mechanism
• Attributes define an identity
• Directives are instructions for a resolver
• Capabilities have a namespace + attributes + directives
• Requirements have a namespace + directives
• Resources have capabilities + requirements
• Repositories are collections of resources
• Environments are state + resources + preferences
• Resolvers select a valid set of capabilities to satisfy the requirements from an environment

Now the thorny problem we have here is we need to state who defines matches for a requirement and a capability. In our (Paremus') usage of this model, there are potentially hundreds of namespaces, most of them defined by end users not repository, environment or resolver implementors. To give you a quick brain dump of some options:

osgi.service -> ds, cm, http, meta, blueprint, etc gogo.command -> any gogo command name javax.sql.driver -> company name/product name user.authentication -> oath, ldap, nt, etc business.function -> accounts, sales, advertising, etc

These are all abstract capabilities that it is possible for a bundle or group of bundles to provide/require.

The generic requirements model seems to be entirely about extension, but if extension is only possible if repository, environment and resolver builders understand the namespace then it seems basically useless. I don't want to have to implement a sql repository in order to create a new namespace. This seems like having to create a new Map class to hold my business objects.

The comment has been made that it is nonsensical for a resolver to have to match a requirement if an environment could have filtered it in findProviders, but I would turn that argument around and say how can an environment or a repository hope to understand the almost infinite number of types of requirements.

Though, I should also say that in all of our usages of these namespaces I've been happy with the requirement:filter+capability:mandatory mapping of matches, so whilst it seems restrictive I'd be ok with defining that's what matches means - but then we can't extend it.

Some thoughts anyway...

[bjhargrave]

Comment author: david.savage@paremus.com

Well, if people created extension namespaces that had some sort of unique matching rules, then it doesn't really matter who does the matching, because there would be no way to do this in a portable way, unless I'm missing something. For example, if the resolver did the matching, there would be no way for a standard resolver to know how to do custom matching for custom namespaces.

Right, a resolver would need some plugin mechanism, but at least environments, repositories and requirements can all define matches in relation to some resolver. I think if a resolver is not extensible and doesn't understand a namespace then it has to basically throw ResolveException

Though as I mention in comment 24 I'm ok with defining matches as requirement:filter + capability:mandatory for all namespaces though it seems a little restrictive. The point being I think we absolutely /have/ to have extension namespaces and I'd /like/ to be able to define more interesting matching rules.

[bjhargrave]

Comment author: david.savage@paremus.com

Not sure what we're agreeing to here, but it sounds like the following.

(1) Repositories.findProviders must return only Capabilities for which Requirement.matches(Capability) == true.

(2) Environments.findProviders may return Capabilities for which Requirement.matches(Capability) == false. In which case, the caller of Resolver.resolve needs to be prepared for an undeployable result. Since we expect the Environment and Resolver client to "know" each other, this is not a concern.

(3) Resolver is not required to perform any further matching of the results of Environment.findProviders in order to compute a resolution.

Is that right?

Not quite, I see it as the following:

(1) Repositories.findProviders and Environment.findProviders should return only Capabilities that they know to POTENTIALLY allow for Requirement.matches == true

(2) Resolver must select only requirements that match capabilities so the result is always deployable - or fail if it can't match or doesn't understand the namespaces.

The important point is about the potentially statement, a repository or environment can filter requirements based on the rules defined in the Requirement.matches javadoc:

   /** 
     * Returns whether the specified capability matches this requirement. A
     * capability matches this requirement when all of the following are true:
     * <ul>
     * <li>The specified capability has the same {@ link #getNamespace() name
     * space} as this requirement.
     * <li>The filter specified by the {@ link Constants#FILTER_DIRECTIVE filter}
     * directive of this requirement matches the
     * {@ link Capability#getAttributes() attributes of the specified capability}.
     * <li>The {@ link #getDirectives() requirement directives} and the
     * {@ link Capability#getDirectives() capability directives} that apply to
     * the name space are satisfied.
     * </ul>

1) Discard any capabilities not in the same namespace 2) Discard any capabilities who's attributes do not match the filter 3) Discard any capabilities that do not match based on the directives it understands

The repository or environment filters a large number of candidates down to a small subset and the resolver just picks off the ones that really match based on some external definition of a namespace (not yet specced).

[bjhargrave]

Comment author: @tjwatson

I updated the Requirement.matches javadoc to state that the standard capability directives must be used to determine if a capability matches a requirement. So far the only standard directive that influences matching is the mandatory directive for capabilities.

[bjhargrave]

Comment author: david.savage@paremus.com

I've committed updates to the Environment and Repository findProviders method and will update the RFC document shortly.

For the record to confirm we're all on the same page.

Requirement matches is defined as:

"Returns whether the specified capability matches this requirement. A capability matches this requirement when all of the following are true:

• The specified capability has the same name space as this requirement.
• The filter specified by the filter directive of this requirement matches the attributes of the specified capability.
• The standard capability directives that influence matching and that apply to the name space are satisfied. See the capability mandatory directive."

Environment.findProviders and Repository.findProviders method now states:

"Find any capabilities that {@ link Requirement#matches(Capability) match} the supplied requirement."

I think this gives us room to extend the standard set of directives if we want to later and states how matches works for now so it will be portable - so I'm happy.

[bjhargrave]

Comment author: @tjwatson

I've committed updates to the Environment and Repository findProviders method and will update the RFC document shortly.

For the record to confirm we're all on the same page.

Requirement matches is defined as:

"Returns whether the specified capability matches this requirement. A capability matches this requirement when all of the following are true:

• The specified capability has the same name space as this requirement.
• The filter specified by the filter directive of this requirement matches the attributes of the specified capability.
• The standard capability directives that influence matching and that apply to the name space are satisfied. See the capability mandatory directive."

Environment.findProviders and Repository.findProviders method now states:

"Find any capabilities that {@ link Requirement#matches(Capability) match} the supplied requirement."

I think this gives us room to extend the standard set of directives if we want to later and states how matches works for now so it will be portable - so I'm happy.

Agreed.

[bjhargrave]

Comment author: david.savage@paremus.com

I've pushed another revision of the RFC document to svn with these changes included

[bjhargrave]

Comment author: @bosschaert

London F2F:

Remove Requirement.matches() so that Requirement becomes a pure data object and move the description of its behaviour to the Repository.findProviders() method or somewhere else around there.

Investigate the need for a utility method RepositoryUtil.matches()

[bjhargrave]

Comment author: @bjhargrave

Remove Requirement.matches() so that Requirement becomes a pure data object

Fixed in https://www.osgi.org/members/gitweb/build.git/commit/0f545465e1cf9511523f664b3a30cbe1b07fc8f8

and move the description of its behaviour to the Repository.findProviders() method or somewhere else around there.

I did not do this.

[bjhargrave]

Comment author: @bosschaert

Investigate the need for a utility method RepositoryUtil.matches()

This is now the subject of bug BZ#2207.

[bjhargrave]

Comment author: david.savage@paremus.com

I've also moved the javadoc from requirement.matches to Environment.findProviders in commit:

https://www.osgi.org/members/gitweb/build.git/commit/2758c137910604f28444af5095bb96756c152390

Should this bug be closed now?

[bjhargrave]

Comment author: @bosschaert

Closing bug.