Indexing Grain State?

[BenjaminGibbs]

I have a few scenarios where the grain state needs to be indexed, so it is possible to either lookup a grain by something else other than the primary key.

One of those examples would be trying to implement an Orleans Provider for ASP Identity. For example GetUserByName and GetUserByEmail I also have a few domain specific scenarios as well.

The way I am currently approaching this, (which most likely not the best way but it is working at the moment on a simple level) is I am saving the UserEntity to ATS along with saving the Index Properties to ATS as well.

I then have a stateless grain which does the lookup of ATS to find the index I want, which then allows me to get the UserGrain by the Id.

It seems rather cumbersome way of getting the grain (but it works), however I was reading an old research paper "Orleans: Cloud Computing for Everyone" which talks about a Queryable attribute.

Annotations on grain interface members, such as Queryable, are used by the developer to cause Orleans to generate additional methods on the factory class for searching for grains that satisfy specified conditions.

Does a feature like this exist in the current build?

I have been pondering other ways to lookup and index which could cut down the network traffic from hitting ATS, so another thought I had was to have a grain to manage the lookup for each index. So call the lookupGrain which then had a reference to the actual grain I wanted.

It would be great to get any input on what thoughts there are around best practice in regard to looking up a grain by different properties within the state.

[gabikliot]

The Queryable property described in the old SOCC paper that you cited is not there anymore. We removed, as the initial implementation was not scalable.

I don't know what ATS is, so can't comment on the design you described.

Two approaches we recommend to solve this problem are: 1) Offload the index to some external indexable/queribale storage. You can write you own grain state persistent provider and map you grain state for example to different columns in Azure table or in Azure SQL. 2) Build a distributed index from grains. For example, you can have one index grain to be the full index (not scalable). Better is have some kind of sharded index. For example, let’s say you have user grains and you want to index by name. So you will have UserIndexGrain for each first letter of the name (total 26). Or for all combinations of 2 letters. Or maybe just 1000 UserIndexGrain and you hash between them. You can take this approach further on and build a rather complex distributed index. To make it reliable you can store every index grain with storage provider. One big advantage of the 2nd approach is that you all reads from index (lookups) will be answered from the grain directly, without reading from the storage (index grain will be basically a cache of the persistent index).

[BenjaminGibbs]

@gabikliot - thanks for the response, ATS is Azure Table Storage.

I think I am going to make adjustments to my design to fall more in line with your second suggestion.

[richorama]

Another idea which answers the GetUserByEmail problem: Just create another grain type (IUserByEmail) which acts as a pointer to your original grain, and just proxies requests to it.

Is that another designed pattern?

https://github.com/OrleansContrib/DesignPatterns

[pherbel]

@BenjaminGibbs Are you planning to share your "Orleans Provider for ASP Identity" solution as open source. I'm interested in.

[BenjaminGibbs]

@pherbel In the long term yes, but currently I don't have time to detach it from some of our project dependency's, as we using our own implementation of connecting to Azure Table Storage which keeps a list of indexes as well.

When I get some time I plan to move it in to isn't own project, and use @richorama's suggestion with the pointer grain. When I do I will let you know.