Closed njr-11 closed 1 year ago
Also, may I suggest to consider the use of Metadata/Metamodel like the one from JPA ? Instead of using the field name we could use SingularAttribute
I always thought Metamodel API had to be be a separate API : An API to be able to describe classes in a meta level. I can't see why it should be sticked/used only by JPA, it seems a more general purpose functionality to me,
Also, recently, I worked on a Criteria API for NoSQL and I had to reinvent the wheel ... sigh.
What do you think ?
So for example (please ignore the DefaultStringAttribute and DefaultNumberAttribute, those are from NoSQL implementation) :
public class Person_ {
public static volatile StringAttribute<Person> name = new DefaultStringAttribute(Person.class, "name");
public static volatile NumberAttribute<Person, Integer> age = new DefaultNumberAttribute(Person.class, Integer.class, "age");
}
@Filter(by = Person_.name)
List<Person> findPersons(String personName);
Strings are evil ! Please let's fight the strings ...
So for example (please ignore the DefaultStringAttribute and DefaultNumberAttribute, those are from NoSQL implementation) :
public class Person_ { public static volatile StringAttribute<Person> name = new DefaultStringAttribute(Person.class, "name"); public static volatile NumberAttribute<Person, Integer> age = new DefaultNumberAttribute(Person.class, Integer.class, "age"); }
@Filter(by = Person_.name) List<Person> findPersons(String personName);
It's a good idea, but Java currently only permits annotation attributes to be primitives, enums, String, Class, other annotations, or arrays. This is per https://docs.oracle.com/javase/specs/jls/se17/html/jls-9.html#jls-9.6.1
So the following would be possible, but not with StringAttribute
and NumberAttribute
,
public class Person_ {
public static final String name = "name";
public static final String age = "age";
}
@Filter(by = Person_.name)
List<Person> findPersons(String personName);
So for example (please ignore the DefaultStringAttribute and DefaultNumberAttribute, those are from NoSQL implementation) :
public class Person_ { public static volatile StringAttribute<Person> name = new DefaultStringAttribute(Person.class, "name"); public static volatile NumberAttribute<Person, Integer> age = new DefaultNumberAttribute(Person.class, Integer.class, "age"); }
@Filter(by = Person_.name) List<Person> findPersons(String personName);
It's a good idea, but Java currently only permits annotation attributes to be primitives, enums, String, Class, other annotations, or arrays. This is per https://docs.oracle.com/javase/specs/jls/se17/html/jls-9.html#jls-9.6.1 So the following would be possible, but not with
StringAttribute
andNumberAttribute
,public class Person_ { public static final String name = "name"; public static final String age = "age"; } @Filter(by = Person_.name) List<Person> findPersons(String personName);
It's like the forth time I forget this very thing. Thank you for reminding me ... :( Ok, then attribute names generated by the Metamodel Generator will be enough, I suppose.
I might be the party pooper here, but I wonder if it wouldn't rather make sense to specify functionality that has been proven successful in the wild, in other words actually standardize instead of trying to be creative and innovative to come up with all the things you could do, but nobody can really verify is useful or even wanted by a broader community?
@odrotbohm what makes you think this functionality is not standardized ?
Aren't other Jakarta EE functionalities annotation based ?
I really can't understand your point.
Anyway looking at the proved successful Spring Data, you can do this :
@Query("SELECT t FROM Todo t where t.title = :title AND t.description = :description")
public Optional<Todo> findByTitleAndDescription(@Param("title") String title,
@Param("description") String description);
In other words Spring Data lets you define the query with annotative approach.
The only difference I can see here is that we are trying to be less "string-based" or "sql-based".
Agreed with @odrotbohm this is way beyond scope
I appreciate a lot the idea proposes by @amoscatelli and i think that is the way we have to go
Beyond that an annotation based query DSL will turn into a cluster IMO. The advantage of @Query
is IDEs like IntelliJ can build code completion for SQL or whatever query language into the IDE and the query language has all the expessiveness there.
Currently you have defined @Filter
which seems to equate to =
but what about other operators? What about projections? What about functions?
All of it is very debateable, unproven in the wild (show examples of a popular implementation that is used at scale that implements this) and requires extra handling by implementors.
This is a big -1 for me.
@francescopotenziani it seems to me that committers to a particular implementation (Jakarta NoSQL) are pushing to have features that are not generalized across implementations pushed into this spec. That is not how a specification processes should work, it is a collaboration amongst different implementations to find commonality and there is no commonality in this proposal.
The hint is in the way this proposal starts "xyz had a really interesting idea..."
"An interesting idea" (ie an experiment) is not a candidate for a specification. A specification, as mentioned by @odrotbohm , is for proven ideas that are known to work in the wild that can be implemented by multiple vendors.
If Jakarta Data goes down the path of adding random, poorly thought out ideas then the net result will be that frameworks like Spring and Micronaut Data will simply not implement the specification.
The point is that there is no implementation of something similar out there that has these features. Nobody in over a decade of running popular data access frameworks has bothered to ask for that feature. Its primary achievement would be to offer yet another way to express the same thing (static sort options). The feedback we get in Spring Data is that all the possible ways to approach sorting already impose quite some cognitive load on the developers.
Successful standards, especially in their first iteration, have always rather tried to find common ground of the already existing implementations and thus define an API that has already proven to be useful and work as it is close to what existing implementations already provide, if not event an exact copy. There's just no point in immediately adding other stuff without any clear feedback from anyone that this would actually be useful, except of course the person suggesting it.
With the latter, you not only create an API that's much more complex to use than all existing implementations (because more options to achieve the same), which is likely to put off users, you also significantly decrease the likelihood that any of the existing implementations would want to implement the specification as you'd impose both the additional complexity on the API and also in their implementations on them.
I can only suggest considering the feedback you get from folks that have worked in that area for more than a decade when it comes to estimation of complexity and the effect that has on the likelihood of the success of what you're trying to build.
In short: a Jakarta specification is not a good place to try to be creative and "try out" new feature ideas. It's a place to find common ground based on the experience of implementations gained in real-world deployments. In that regard, the Data specification is extremely lucky, as it actually can study a couple of already existing implementations. An advantage not all specifications had. Ignoring the feedback you get from folks that have seen millions of production deployments, gazillions of tickets, carefully evaluating user feedback over multiple years certainly is an option. I just think it might not be the wisest one to choose.
@graemerocher Unfortunately, may you please elaborate further your cues? I mean, instead of simply saying:
what about other operators? What about projections? What about functions?
would you please go deeper about projections and functions? For example, projections are tipically based on field names. As a consequence, they look an ideal case for the suggested approach. Functions, instead, by their own nature, might be less compatible: however, I don't see any problem in letting people who need functions use the @Query
approach while providing the others with a more expressive, more constrained, less error-prone way of specifying queries.
In the end, also a type-safe, constrained usage of CriteriaBuilder
might be seen as a "cluster" if compared to the usage of strings within the @Query
annotation. On the other hand, we can safely state the following: while @Query
provides the greatest flexibility, it also poses huge maintenance issues when it comes to refactorings: consequently, @Query
should be used as a last resort and the other approach (the CriteriaBuilder
"cluster") should be preferred. The idea which was proposed here is based on the same approach: we have the super-flexible @Query
annotation but, in the meantime, we should also provide a more constrained and expressive way of specifying queries. Yes, we already have CriteriaBuilder
, but the suggested approach will lead to even greater expressivenes, synthesis and will generally make the code look more similar to a structured natural language, which is what we should aim at in general.
Spring Data already did something similar with the mechanism of representing the query in the method name. Super-synthetic, super-expressive, unfortunately super error-prone during refactoring. This proposal, IMHO, would preserve the firts two pros and would remove the last con.
show examples of a popular implementation that is used at scale that implements this
So, yes, there might not be popular implementations for this. It means that this thread can be the start of this new approach. Since the direction is the right one, IMHO (greater expressivenes, synthesis and more similarity between code and structured natural language), instead of turning this idea down because "nobody still implements it", we should involve those decision makers in the process. Instead of standardizing bottom-up, i.e. trying to tidy up the "proven-to-work, reinvented wheels" that have been implemented in the wild, we might try to standardize top-down.
I realize it is a complete direction shift. The following sentence by @odrotbohm is a superb example of the bottom-up approach:
It's a place to find common ground based on the experience of implementations gained in real-world deployments
However, if we stick to the bottom-up standardization approach, we will keep on suffering from the fact that Java specifications always come last, condemning ourselves to late adoptions. Instead, why should not we try to drive the world from the beginning towards a clean, expressive, constrained, no-reinvented-wheel approach? Why should not we try to anticipate implementors, involving the main ones in the specification from the beginning?
I spoke about the Metamodel used in JNoSQL only to describe one of the possibilities, it's not even in the proposal description.
There is no relation between annotative approach to define queries and Jakarta NoSQL. They seem very unrelated to me and I am unaware of any involvement of this in the NoSQL project. If you think otherwise, please prove your point, otherwise you are only making unfair accusations.
You say that specification process is only about find commonality between existing implementations. This seems like a very limited and cheap point of view to me.
Yea, often this is what happened. Other times, the specification process is about finding an abstraction able to represent a common scenario we want to face without reinventing the wheel everytime, even if there are no multiple existing and proven implementations for that wheel. Some times, when a specification was ready, there was only a single RI, for example Jakarta Security and Soteria. I like to think that a specification should exist even before any implementation.
A Jakarta Specification proposal is also an occasion to debate and finding the right approach to solve a problem. A Jakarta Specification is about software engineering. A framework, especially Jakarta EE imho, should guide the community, not the opposite.
Anyway, any new API/spec requires effort and handling by implementors. But I think this is like stating the obvious.
This proposal has a very big +1 for me.
One of the most common form of criticism again Jakarta EE is about the lack of speed/functionalities. They say this is what brought to Spring back then and Eclipse Microprofile later.
All of this is cheap to me and is proving them right.
We are just discussing about how Jakarta Data could represent/design a functionality that is clearly in scope.
I have no personal gain in all of this so, please, have your way.
P.S. Also, this brings me back at university. Back then, writing implementations first and interfaces later could make you stay ungraduated.
Since the direction is the right one, …
What makes you think it is? Forward is not a direction. Repositories are a pattern to abstract concrete data access technology. All means that exposed by them have to be carefully considered to not leak any, or if possible as little as possible, about the underlying persistence mechanism. A simple query method definition is the gold standard of that. An @Query
taking a store-specific query is a well-balanced compromise. Trying to pull out query details into an annotation-based query language is the exact opposite of "right". I am honestly a bit tired of having this discussion over and over again because every other year, someone comes around, misunderstands repositories as "some way to let methods trigger queries", completely missing the fact that all of this has been discussed at least a dozen times and dismissed after careful consideration of all pros and cons.
However, if we stick to the bottom-up standardization approach, we will keep on suffering from the fact that Java specifications always come last, condemning ourselves to late adoptions. Instead, why should not we try to drive the world from the beginning towards a clean, expressive, constrained, no-reinvented-wheel approach? Why should not we try to anticipate implementors, involving the main ones in the specification from the beginning.
What makes you think that a specification is the place to do that? Why would you even accept the given administrative boundaries and processes if your goal is to experiment, be agile, gather feedback quickly and change things based on that feedback? Especially the latter is something that specifications by definition – like it or not – cannot do, as breaking changes are the enemy of people adopting one.
breaking changes are the enemy of people adopting one
@odrotbohm Which breaking changes are you foreseeing, exactly? I see additions, more than changes.
Since the direction is the right one, …
What makes you think it is?
For what concerns the reasons about "right", I expressed them between parentheses and I will report them here for the sake of reading speed: greater expressivenes, synthesis and more similarity between code and structured natural language.
All means that exposed by them have to be carefully considered to not leak any, or if possible as little as possible, about the underlying persistence mechanism. A simple query method definition is the gold standard of that. An @Query taking a store-specific query is a well-balanced compromise
I would not disagree, with the sole difference that I would state that @Query
is the passepartout which a framework should never lack, in order to provide flexibility and avoid feature misses, but that such passepartout is just the very least objective of a framework, IMHO. The very first objective is constraining us to work in a standardized, structured and clean way, in order to avoid mid-term and long-term consequences of our own quirks. The typesafe usage of CriteriaBuilder
is a good example of this, as I previously stated.
Repositories are a pattern to abstract concrete data access technology
misunderstands repositories as "some way to let methods trigger queries"
Umn, you might be right in general, and I would not even be suprised by the misunderstanding you are reporting. However, this is not I had in mind while reading this proposal. Indeed, I would say that the "query triggering/building" mechanism should be built on top of the "repository concept", and that Jakarta Data might take charge of both, possibly in separate modules. However, this is not a level of detail I would discuss here.
What makes you think that a specification is the place to do that? Why would you even accept the given administrative boundaries and processes if your goal is to experiment, be agile, gather feedback quickly and change things based on that feedback?
I hope I will not sound harsh, but the fact that my "goal is to experiment, be agile, gather feedback quickly and change things based on that feedback" is an assumption you made :) Instead, I would state that I am supporting a discussion about a feature to be designed, a discussion which might even take some months. The last part of my previous message and the whole message from @amoscatelli describes thoroughly the underlying mindset. No experiments, no quick solutions. Just software engineers designing solutions while bearing in mind software engineering concepts, trying to anticipate developer needs and aiming at avoiding that too many people start proposing their own working but non-standard, custom-flavoured solutions
The place to experiment is in the implementations not in specifications. A specification is something that is put in place and then can never be changed. Whatever error you make with your experiments (and there will be) can never be reverted and has to be maintained forever even if it is broken or suboptimal.
The existing implementations out there like Spring and Micronaut Data have been used at scale by thousands of developers. The concepts are proven. The ideas are proven. If a user wants to use experimental functionality they can easily drop down to the implementation's API and use that, without the specification suffering and the rest of implementors paying the price.
A successful specification is also one that allows multiple implementations to be built and maintained over time, a specification with a single implementation is a failure in my opinion. Examples of the best specifications (such as the Servlet API) have multiple successful and maintained implementations.
You mention CriteriaBuilder
which is in fact an example of a specification that over reached and introduced an API that has several opinions (like type safety requiring the additional of compiler plugins) and was introduced at a time when Java had yet to evolve features (records, lambdas etc.) that could have resulted in a much better DSL. Now we are stuck with CriteriaBuilder
which is one of the uglier query builders out there when you compare it to QueryDSL for example.
As for personal gain I have none either. I offered my help to define this specification with the hope that we could implement it in Micronaut Data, but if that turns out not to be the case because of overreach and people going wild in the specification process then I am fine not implementing the specification and it will be less work for us.
Ultimately all of this is net new and unproven and if Spring and Micronaut Data don't implement this specification all that will happen is that it will have vastly fewer people adopting the specification, which is a shame for the specification but for nobody else.
Now we are stuck with CriteriaBuilder which is one of the uglier query builders out there when you compare it to QueryDSL for example.
I agree with that. However, that was not the overall point about CriteriaBuilder
vs @Query
. Actually, I could have mentioned QueryDSL
, with no difference. The focus was not on CriteriaBuilder
itself and it would have not been on QueryDSL
if I had mentioned it.
The focus, instead, was about the problem they were solving and the underlying approach both solutions used: avoiding the string-based approach of the @Query
annotation, when possible, by adding a mechanism which provides greater expressivenes, greater synthesis, greater resilience to code refactorings and which constrains people to type-safety. By the mere fact that you are constraining developers to develop in a certain way, you are creating a "cluster", because not all of us will be either willing or able to follow the constrained approach. However, I can safely state that none of us would throw away QueryDSL due to the fact that it is used only by a portion of developers. Therefore, the risk of creating a "cluster" is not enough as a counterargument, IMHO.
The place to experiment is in the implementations not in specifications. A specification is something that is put in place and then can never be changed. Whatever error you make with your experiments (and there will be) can never be reverted and has to be maintained forever even if it is broken or suboptimal.
@graemerocher What do you mean by "experiments", exactly? Do you also refer to designing an interface method which may turn out to be almost unused in the future? I am asking this because nobody of us (neither me, nor @njr-11, nor @amoscatelli) is talking about implementing anything. On the contrary, we are willing to go specification-first. To be precise, we want to talk ONLY about specifications and interface design. The implementation is an aspect I'm not even willing to think about right now.
I am not a believer in specification-first design if that is what you are advocating for. You cannot know all the requirements and pitfalls of a design before it has been user tested at scale or there are existing implementations that have been user tested at scale and so far you have failed to demonstrate that these exist.
You are also conflating specification and interface design which are 2 different things. A specification is something that was published and released cannot be changed, ever. The cost of getting it wrong is very high. And you cannot know it is right without end user input into the process and developers battle testing your code.
This is not the same as a framework or implementation which can reasonable change an interface and evolve an interface over major releases.
You are also conflating specification and interface design which are 2 different things
Actually, I am not, and I think I even stated it explicitly. If you find it useful, please elaborate about why you think I am doing it.
A specification is something that was published and released cannot be changed, ever. The cost of getting it wrong is very high This is not the same as a framework or implementation which can reasonable change an interface and evolve an interface over major releases.
I do agree with both
I am not a believer in specification-first design if that is what you are advocating for. You cannot know all the requirements and pitfalls of a design before it has been user tested at scale or there are existing implementations that have been user tested at scale
Alas! This is the main, actual, underlying point :) I am happy it was put down in writing, in the end. I may even agree with it, generally speaking, especially if we are talking about designing something new from scratch. Better wait for the world, instead of imagining the world, in that case.
However, here we are not designing a new API from scratch: we are talking of extending the existing ones, by adding an alternative, more structured, less error prone way to request something that can be already requested by other means through the already existing interface. We are on a safe path, in the end. Some weeks of discussion, considerations, reviews are likely to be enough. Refusing to go specification-first in ANY case is way too conservative, IMHO.
and so far you have failed to demonstrate that these exist.
I did not even try to, actually. In the context I described before, the existence of implementations, whether popular or not, it is neither a point nor a counterargument. Moreover, from my experience, it is a very variable aspect: for example, @amoscatelli already mentioned Jakarta Security and Soteria. The presence of the sole Soteria did not block Jakarta Security from becoming part of the spec
Some weeks of discussion, considerations, reviews are likely to be enough.
On this I fundamentally disagree yes, it will not be enough, without end user input and validation of the design it is not possible to come up with a design that works IMO and is not an approach I can support
if that is how this specification is going to move forward I would like to withdraw my participation and committer status please @otaviojava let me know how to do that.
I don't have anything more to say on this matter.
Everyone please keep in mind this is only a git issue (not even a day old) proposing an idea to explore for possibly making the specification better. It is not a pull request that we are trying to decide today whether to merge into the specification. The specification benefits from having lots of different ideas proposed. Some ideas are developed further by the specification participants and make it into the specification, and others will fail to do so, each on its own merits. Let's allow the time for the participants who are interested in this to further refine the idea, try things out, collect input and make it better. And then decide if it's wanted or not. Jakarta EE 11 is still a long ways off.
I will say that I'm surprised to hear this proposal would be a burden on implementations like Spring and Micronaut. That was never the intention. It was supposed to match up with what repository query by method name already does and is just a different way to get the same information rather than by parsing method names. If that isn't the case, we've done something wrong.
Please let me clarify that, by "some weeks" I do not mean 3 or 4, but 10, or 20, i.e. between 2 and (I hope not) 5 months. Furthermore, I said "are likely to be enough". If they won't be, the discussion shall continue, obviously
For what concerns your last message, @graemerocher, I am failing to see the very reason of your request to @otaviojava .
You see, we are exchanging opinions, in a context where I am the one providing opinions and you are the one who is actively contributing by writing code, if I get it correctly. Firstly, the difference of roles and relevance looks self-evident, to me. Secondly, in terms of cardinality, it is a confrontation between my opinion and yours, i.e. two opinions amongst the whole community, i.e. a debate of limited relevance (at least currently) on a very specific feature, not on broad aspects of the specification.
After that, you might gain consensus, i.e. "an user input and validation" in your favour, so you would design/develop/discuss consequently. Alternatively, I might gain consensus, i.e. "an user input and validation" in my favour, and you would design/develop/discuss consequently. As a last possible outcome, no consensus would be gained and the decision would be up to only those who are actively working on it (i.e. @otaviojava, you and anybody else who is involved).
Drawing boundaries now, when the aforementioned relevance is so low and no actual decision can be taken, looks very very anticipated, from my point of view
for me fundamentally a specification has to operate like a specification and not a playground for unproven ideas and experiments. If a specification wants to operate that way then I have no interest in being involved. So you guys need to decide how you want to operate this specification to operate and maybe @otaviojava as lead can decide that then let me know.
The role of the specification is to find commonality amongst existing implementations that contain proven ideas and provide something that all existing can implement without becoming a burden to those implementations.
Proposals like this have no place here whatsoever without there first being an existing implementation of what is being discussed that is being used at scale which once again like I said before has yet to materialize.
Also it is fine to have differing opinions but the folks posting on here have been openly dismissive of @odrotbohm and my viewpoints and concerns, when we collectively have years and years working experience on existing implementations that are actually used at scale
I'm sorry for not getting back to this issue sooner.
IMHO:
As a specification team, should focus more on what the industry has been doing with success. We can use and explore the most significant hits, such as Micronaut, Spring Data, Apache Delta Spike, etc.
I agree with @graemerocher @odrotbohm
We'll close it as unsolved isssue for awhile.
@amoscatelli had a really interesting idea in https://github.com/jakartaee/data/issues/94#issuecomment-1378369052 about having an annotation-based approached to defining queries that is less error prone than query by method name. I think an approach like this has the potential to be able to achieve basically everything that query by method name can do, but in a more intuitive way because the annotations will guide you as to what is possible and in how to validly form queries rather than needing to know the keywords and formulas by which query by method name is formed and interpreted. This isn't proposing that we do away with query by method name as part of the spec, just to explore an additional approach that could be valuable to users.
Here are some initial thoughts, expanding on the sorting annotation that already exists and the
@Filter(by = ...)
approach.It would be easy to add comparison operations, with Equals being the default for comparisons,
and multiple conditions (given that multiple annotations have a natural order for correspondence with parameters),
Even aspects like IgnoreCase could be included (defaulted to false),
Named parameters could be done with the existing
@Param
annotation,You could even allow fixed-value parameters,
Other type of queries like delete, count, and exists could have an annotation for that purpose,
It would even be possible, with the understanding that Filter parameters come first, to have some basic update operations if we wanted that, with Set being the default operation type,
Projections could be done with:
In all of the examples above, multiple Filters are interpreted as though they are joined with the
And
keyword. To be able to do everything that query by method name can do, we would also need an equivalent for theOr
keyword. Some different possibilities that I've thought of are: