Question: how to add custom endpoints/queries?

[Serhii-the-Dev]

Seems like current documentation lacks of manual for implementation of custom queries via Elide API. For example, a service I'm currently working on, should provide an authenticated user endpoint: /users/me and a few related collections, like subscriptions, address, and so on. For now I've found only one way to implement this: add a custom REST ednpoint/GraphQL resolver that will mimic Elide API(to provide API consistency for clients). But, of course, there will be no support for all Elide features, like filters, includes, pagination, and so on. Is it possible to add such queries to Elide-based app with support of fields filtering, security annotations, etc. via Java API? P.S. For example, Spring Data REST has a concept of resource links, that allows to extend generated API graph.

[DennisMcWherter]

The approach you have taken makes sense to me if you want an entirely different API than what is being provided by Elide. You could emulate links through a @ComputedAttribute if this was the approach you wanted to take.

Another option is to create a new model that represents your composite object. Rather than exposing the raw DB directly, you can create a new JPA object that isn't persisted as its own table, but is based off of an existing table instead. For instance,

@Entity
@Include(type="me")
@Table(name="user")
public class ExposedMeEndpoint {
  // The "user" fields expected to exposed via the /me endpoint
  private Integer id;
  private String name;
  private Set<Subscription> subscriptions;

  @Id
  public Integer getId() {
    return id;
  }

  public void setId(Integer id) {
    this.id = id;
  }

  public String getName() {
    return name;
  }

  public void setName(name) {
    this.name = name;
  }

  // If a subscription column/join table doesn't exist, use JPA constructs like @JoinColumn
  // to describe how to join this new relationship
  @OneToMany
  @JoinColumn(...)
  public Set<Subscription> getSubscriptions() {
    return subscriptions;
  }

  public void setSubscriptions(Set<Subscription> subscriptions) {
    this.subscriptions = subscriptions;
  }
}

As you can see, this is a new table representing the underlying user database table with a different set of common fields and annotations.

Would something like that help solve your issue?

[Serhii-the-Dev]

@DennisMcWherter Thank you for the advices. I am still learning Elide internals and may missing some key concepts, so sorry for dumb questions :) The thing is, users/me should render the same User entity as any other exposed via Elide, like users/123. It's a shorthand for query similar to this:

SELECT users.* FROM users
  JOIN persistent_logins ON persistent_logins.username = users.email AND persistent_logins.token = :token

In fact, in most cases there are no DB queries at all, user details along with ACL roles are stored in memory via Spring Security authentication principal data. @ComputedAttribute seems fine for things like simple aggregations and counters along with @Formula, but I don't think it's a good approach to put entire app logic inside JPA entities. Duplicated entity will create a parallel collection in Elide API, while I need a single entity provided by a custom query.

[DennisMcWherter]

Hi @Serg-de-Adelantado, these are good questions! We should probably encapsulate some of this knowledge into our docs :)

Anyway, there are two approaches depending on what you're after here. If you're after an exposed entity that is the User object (but augmented) you can also think about inheritance and exposing the inherited entity. However, I don't think this is accomplishing what you're after.

Just confirming: /users/me is actually just a shorthand for /users/myId. If so, there are a couple of things we can do here. A "trivial" way to accomplish this is to make a proxy endpoint. /myEndpoint/users/me internally proxies to /elideEndpoint/users/123. It could either do this via a redirect (i.e. extract user ID and construct valid link) or entirely internally.

Another approach is to have an extended collection with a slightly different name. Namely, something like this:

@Include(type="myUser", rootLevel=true)
@ReadPermission(expression = "is accessing self")
@CreatePermission(expression = "deny all")
@ReadPermission(expression = "deny all")
@UpdatePermission(expression = "deny all")
public class MyUserEntity extends User {
}

In this case, you would have a new /myUser endpoint that exactly mimics the User endpoint. The only difference is that we have an is accessing self permission. This could be implemented as a FilterExpressionCheck for performance or a simple in-memory check for simplicity.

Are we getting closer to what you're looking for now? Unfortunately, /users/me would be pretty out-of-spec from a JSON-API perspective so we don't have a way of doing that directly. However, I think /myUser may get at what you're looking for (though it will still be returning a collection of a single user).

[Serhii-the-Dev]

Yeah, we had some discussions about API architecture and came to less strict rules described by plain REST which allows temporal and single resources, like /{city}/weather/today, /orders/my, etc. This is a reason why I am searching for a framework with GraphQL support which seems more flexible than REST approach. For example, in this particular case with /users/me I can follow the JSON API rules, expose persistent_logins table to JPA and Elide, than make requests from client side like:

{
  authTokens(filter: "token=='TOKEN_FROM_COOKIES_OR_LOCAL_STORAGE'") {
    edges {
      node {
        user {
          edges {
            node {
              id
            }
          }
        }
      }
    }
  }
}

The overhead in this case is not so huge, but we have cases of complex queries with multiple joins, for example, there are three types of news: global, organization-wide, and regional, and HQL query for all news avialable for a user looks like:

SELECT NEW News(
                n.id,
                n.title,
                n.body
            ) FROM News n
            JOIN n.filters AS filters
            LEFT JOIN filters.company AS company 
            LEFT JOIN company.clients AS clients
            LEFT JOIN clients.user AS user
            WHERE user.id = :userId
            AND (
                (filters.company IS NULL) OR
                (filters.company = clients.company AND filters.region IS NULL) OR
                (filters.company = clients.company AND filters.region = clients.region)
            )

Right now, to expose a query like this to Elide, I should convert HQL to native query, create a view table in database based on select query, create a JPA entity for this table, add relation into User entity, and query those news feeds through the entities relationship - this is a simplest working approach I was able to found. I was looking for some declarative way to add such query as a GraphQL query into Elide, with possibility to proxy filters and pagination, but it seems like there are no programmatic APIs for this.

[DennisMcWherter]

Ah I understand better now. Today-- @aklish or @clayreimann correct me if I'm wrong-- but we don't yet have support for arbitrary fields yet. This is actually the top priority on our 4.1 roadmap. Out of curiosity-- where would you expect this field to go?

{
  authTokens() { edges { node {
    me { edges { node {
      id
    } } }
  } } }
}

or something else? While I don't expect it to necessarily be trivial, I don't anticipate the workload being very high to add such support to Elide. Off the top of my head, the three major considerations we'll have to make when adding this to Elide:

1. How to express these pseudo-fields in a JPA-compliant way
2. Propose a way that ideally minimizes application logic leaking into models
3. Ensure API is flexible enough (i.e. pseudo-root collections, pseudo-subfields, etc.)

[Serhii-the-Dev]

Out of curiosity-- where would you expect this field to go?

In my current API implementation based on Spring Data REST, it is injected into /users collection, so authenticated user's profile is accessible via /users/me endpoint. Also there are endpoints like /news/feed - I've described it in previous comment, and /news/editable - which is a shorthand for collection of news available for authenticated content manager(another complex query with roles, distributed to regions) to load those news into table in admin panel, and so on. In terms of GraphQL Relay model, IMHO, it will be something like:

{
  me {
    edges {
      node {
        id
      }
    }
  }
}

since it is a single resource and not a field/relation of the User entity. Contruary, feed, and editable could be added to user node, since they are related:

{
  user(filter: "id=='1'") {
    edges {
      node {
        feeds {
          edges {
            node {
              title
            }
          }
        }
      }
    }
  }
}

Propose a way that ideally minimizes application logic leaking into models

I can suggest to take a look onto Spring Data REST approach: it exposes repositories as API endpoint, and allows to wrap entities into Resource and provide custom relationships links, like in this example. Sorry, still don't know Elide good enough to propose something more specific.

[clayreimann]

My 2¢

Are we getting closer to what you're looking for now? Unfortunately, /users/me would be pretty out-of-spec from a JSON-API perspective so we don't have a way of doing that directly. However, I think /myUser may get at what you're looking for (though it will still be returning a collection of a single user).

@DennisMcWherter I don't think that /users/me is out of spec for JSON-API, their docs are pretty light on what constitutes a valid URL (unless I'm missing a key section); however /users/me is pretty far outside of Elide's conventions.

@Serg-de-Adelantado Elide takes an opinionated stance that good web services have hard and fast conventions in service of being very uniform. The consequence of that is we don't have, and likely won't soon have, a generic mechanism for getting outside the box we've built.* (caveat below)

---

Right now, to expose a query like this to Elide, I should convert HQL to native query, create a view table in database based on select query, create a JPA entity for this table, add relation into User entity, and query those news feeds through the entities relationship - this is a simplest working approach I was able to found.

@Serg-de-Adelantado You shouldn't need a new bean to expose this list of news articles. You can simply compose a few filter expression checks and we'll do the magic of translating them into JPQL and pushing them down to the database. No views or writing raw SQL required.

---

In my current API implementation based on Spring Data REST, it is injected into /users collection, so authenticated user profile is accessible via /users/me endpoint. Also there are endpoints like /news/feed - I've described it in previous comment and, /news/editable - which is a shorthand for collection of news available for authenticated content manager to load those news into table in admin panel, and so on.

@Serg-de-Adelantado, @DennisMcWherter I would imagine that these can be fulfilled by us allowing custom functions in GraphQL, which is on our 4.1 roadmap. We've always thought that the ability to add custom functions is one of the powerful parts of GraphQL and your use case presents a good argument for mapping static functions into our GraphQL document.

So we can't currently do what you want (directly) but your goal are likely goals that we would like to support, probably through the ability to annotate your GraphQL document with custom functions. How do you guys feel about the following pseudo-code?

class News {
  int id;
  String text
  Company company

  @GraphQLFunction
  static Set<News> editable() {
    // magic for finding this news, probably involving calls to PersistentResource :(
  }
}

{
  news {
    editable { edges { node {
      id
      text
      company { edges { node
        id
        name
        ...
      } } }
      ...
    } } }
  }
}

[Serhii-the-Dev]

@clayreimann

You shouldn't need a new bean to expose this list of news articles. You can simply compose a few filter expression checks and we'll do the magic of translating them into JPQL and pushing them down to the database. No views or writing raw SQL required.

Sorry, don't found a way to perform a query like this in filters:

SELECT * FROM items WHERE items.region IN 
(SELECT region FROM region_managers WHERE region_managers.user_id = :user)

except split it on two queries and perform regions query directly in filter, and then use Operator.IN.

How do you guys feel about the following pseudo-code?

@GraphQLFunction
static Set<News> editable() {
// magic for finding this news, probably involving calls to PersistentResource :(
}

Well, in most of my usecases such side queries are performed via EntityManager which is injected by Spring into some @Service or @Repository that contains a query logic. Also, there are situations when it is not even a database queries, but some calls to external API, for example in recent project it was a service, gathering GPS data from sensors with further cahcing of that data for API calls. Can't see a way for proper work of DI in entities, plus I prefer an external configuration, something like:

elideInstance
.addRelation(onPath = "/users", entity = News::class, name = "feed")
.resolvedBy { pageInfo, filters, etc -> feedsService.queryForNews(pageInfo, filters, etc) }

But the point is, I get used to Spring Data REST to much, and now I am trying to bring those concepts into Elide which can be wrong, since Elide states as API wrapper for JPA entities, while Spring Data REST is a tool for building of REST API on top of CRUD repositories with possibility to extend an API graph. So I don't think I can make some reasonable decisions about Elide architecture right now, considering my lack of knowledge of the framework internals. Maybe I should stick to my own GraphQL wrapper on top of Elide API, that will connect different services. Also there may be problems in my DB design, that causes such complex queries.

[clayreimann]

Sorry, don't found a way to perform a query like this in filters:

If you adopt our security model you can use the FilterExpressionCheck I linked to. These security expressions builds filter expressions to do (I believe) exactly the sort of thing you're looking to do.

//Construct a filter for the Author model for books.title == 'Harry Potter'
Path.PathElement authorPath = new Path.PathElement(Author.class, Book.class, "books");
Path.PathElement bookPath = new Path.PathElement(Book.class, String.class, "title");
Path path = new Path(Arrays.asList(authorPath, bookPath));

return new FilterPredicate(path, Operator.IN, Collections.singletonList("Harry Potter"));

Well, in most of my usecases such side queries are performed via EntityManager which is injected by Spring into some @Service or @Repository that contains a query logic… Can't see a way for proper work of DI in entities

I'm not sure how we'd handle the DI portion of it since you're moving out of a Spring world into an Elide world.

Also, there are situations when it is not even a database queries, but some calls to external API, for example in recent project it was a service, gathering GPS data from sensors with further cahcing of that data for API calls.

Querying external services can easily be handled by writing a datastore to query the service in question

plus I prefer an external configuration, something like

But the point is, I get used to Spring Data REST to much, and now I am trying to bring those concepts into Elide which can be wrong, since Elide states as API wrapper for JPA entities

I think you've identified a good point here. We're not Spring REST, and there may be good reasons to write a custom REST service. Elide believes very much in convention over configuration, we feel that all of the logic that drives your API should live in your beans so that you don't go hunting all over the place to figure out what's going on.

[aklish]

Hi Serg,

Thanks for the thoughtful feedback and inquiries.

I haven't had a chance yet to fully digest this conversation (will do that hopefully soon), but a quick comment on something you wrote:

"Can't see a way for proper work of DI in entities, plus I prefer an external configuration, something like:"

Check out the "Dependency Injection" section in: http://elide.io/pages/guide/02-data-model.html

With respect to "external configuration", to add a relation (bound to a function) in Elide, you can do that through a custom store for a particular entity class. However, I do like the ease of your suggestion. It might make sense to add some syntactic sugar around this in Elide to make this simpler to do.

Aaron

On Tue, Feb 13, 2018 at 3:18 AM, Serg de Adelantado < notifications@github.com> wrote:

@clayreimann https://github.com/clayreimann

You shouldn't need a new bean to expose this list of news articles. You can simply compose a few filter expression checks and we'll do the magic of translating them into JPQL and pushing them down to the database. No views or writing raw SQL required.

Sorry, don't found a way to perform a query like this in filters:

SELECT * FROM items WHERE items.region IN (SELECT region FROM region_managers WHERE region_managers.user_id = :user)

except split it on two queries and perform regions query directly in filter.

How do you guys feel about the following pseudo-code?

@GraphQLFunction static Set editable() { // magic for finding this news, probably involving calls to PersistentResource :( }

Well, in most of my usecases such side queries are performed via EntityManager which is injected by Spring into some @Service that contains a query logic. Also, there are situations when it is not even a database queries, but some calls to external API, for example in recent project it was a service, gathering GPS data from sensors with further cahcing of that data for API calls. Can't see a way for proper work of DI in entities, plus I prefer an external configuration, something like:

elideInstance .addRelation(onPath = "/users", entity = News::class, name = "feed") .resolvedBy { pageInfo, filters, etc -> feedsService.queryForNews(pageInfo, filters, etc) }

But the point is, I get used to Spring Data REST to much, and now I am trying to bring those concepts into Elide which can be wrong, since Elide states as API wrapper for JPA entities, while Spring Data REST is a tool for building of REST API on top of CRUD repositories with possibility to extend an API graph. So I don't think I can make some reasonable decisions about Elide architecture right now, considering my lack of knowledge of the framework internals. Maybe I should stick to my own GraphQL wrapper on top of Elide API, that will connect different services. Also there may be problems in my DB design, that causes such complex queries.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/yahoo/elide/issues/627#issuecomment-365199973, or mute the thread https://github.com/notifications/unsubscribe-auth/AGp_QZ31tSRas-pSo4G8upCktS3HwI7oks5tUVN4gaJpZM4SCTBV .

[Serhii-the-Dev]

@clayreimann

If you adopt our security model you can use the FilterExpressionCheck I linked to.

Thanks, I've read the docs on elide.io. Problem is, in my custom service solution I need a one HQL/SQL query, while filter requires to made two queries: one to get a list of roles(inside filter code), and second to fetch items by those roles(which will be made by Elide). Anyway, it will work in the way you proposed, and one extra query don't seems like a significant drawback.

@aklish I had some questions about splitting of concerns for different services relying on a single database, and I've found the described concept of "entity per task" a clever decision answering to those questions. I am still afraid that such approach may lead to leak of a business logic into entity classes, but before making any decision I should "battle-test" it in a complex project.

@DennisMcWherter @clayreimann @aklish, thank you!

[clayreimann]

Not to belabor the point but our intention is that this statement is not true.

while filter requires to made two queries

Our intent with the FilterExpressionChecks is that they generate jqpl that can get pushed down to the database to run the kind of query you're hoping to run in an efficient way.

[Serhii-the-Dev]

I've made some tests with integration of Elide into current API, and found that RSQL filters could be used for identifiers masking, so the first usecase(current user profile) can be implemented like a request:

user(filter: "id=='me'") {
    edges {
        node {
            id
        }
    }
}

with a proper filter check.

But the second usecase is still a terra incognita for me: for now I have 8-9 root entities that can be viewed and edited. So there is a need in displaying of lists of those entites according to user's roles by two criterieas: readable and editable news. The first list type is used in client part of our app, where content manager can read those entites, like a regular user(with a few restrictions, related to profile settings, for example, users can only see news related to their regions and departments), and the second list type is used in admin panel, where content manager should see only those entites, that are available for editing. Usually manager can edit only some part of readable news, according to his role in department.

Right now it's made via JPA repository and pretty simple @Query, proxied to REST API like: /rest/{entity name}/serach/readable and /rest/{entity name}/search/editable. The only thing I can imagine for now to provide same endpoints with Elide is some additional filter tweek with adding of custom expressions.

The solution, proposed earlier(with multiple entities with same table for both readable and editable lists) not seems like a suitable one, since it requires to generate 16-18 additional copies of entities.

[clayreimann]

Off the top of my head I'm guessing that you could add a @ComputedAttribute that exposes whether or not the entry is editable and then filter on that. I don't recall if we've added or only talked about adding the ability to filter on computed attributes–it's certainly on our roadmap.

@DennisMcWherter do you remember if we added the ability to filter on computed attributes?