Map type

[miracle2k]

Should there be a map type?

I have a structure that looks like this:

  Item {
     title {
        de_DE,
        fr,
        pa,
        .....
     }
  }

The point here is that the number of language strings might be arbitrary. I find this hard to model using GraphQLObject, because:

1. The number of locales is undefined. While the resolver could handle this dynamically, I think it's a problem in Relay that the schema cannot reflect these dynamic fields.
2. I would like to query all the existing locales, which isn't allowed, it seems ("must have a sub-selection").

If I make my GraphQL server return a JSON object for "title", the Relay client doesn't complain (although maybe shouldComponentUpdate breaks), but I think I'm skirting by here. At the very least, I think I couldn't generate a schema that confirms to the spec.

[OlegIlyenko]

I think this is a valid concern as well. I currently thinking of ways to add GraphQL endpoint for our API. We are building project-based multi-tenant service. For example every project has a list of products which have name and description. Just like in your case these are localized strings as well.

One way one can approach this problem is to parametrize the field:

query ProductNames {
  products {
    nameEn: name(locale: "en")
    nameDe: name(locale: "de_DE")
  }
}

Another approach, as you mentioned, would be to generate a schema of-the-fly. In our case it would be possible, since every project has a limited set of locales which are defined in the config of this project.

Where it becomes more tricky is an addition user-defined data types. Users of our API can define new attributes for products (visually through the merchant-center application). These attributes are also typed, so it's possible to generate GraphQL schema for this project, but it has some implications:

• Generating schema on every request comes with big performance penalty, since we need to load config from a DB in order to do so
• to compensate for this performance penalty we need to implement some schema caching and cache invalidation logic
• Schema now becomes tenant-specific. This means that it becomes much harder to integrate with generic tools like GraphiQL and do introspection in general
• You can't even do an introspection queries without the auth anymore.

I guess one can just put all custom attribute JSON in a string scalar, but I don't think that other developers will appreciate JSON inside of string inside of another JSON :) I feel that generic JSON/Map-like type can provide a very useful middle-ground for these use-cases. It can also help a lot with a migration to the GraphQL. One can quickly start with Map-like structure and after some time develop schema generation and caching mechanisms.

[Sandreu]

Hello,

I agree about this, and as @OlegIlyenko said, JSON string inside JSON string seems awkward. I think Map is one portion of what JSON can provide and I have exposed my point of view in https://github.com/graphql/graphql-js/issues/172 So :+1: to include specs for a "JSON or RawObject or however you want to call it" type.

[xpepermint]

+1

[clintwood]

Hi,

I have a use case where I have 'server based schema' and 'client based schema'. The server based schema is pretty much static and will follow normal project/application changes through time. However the client based schema is specific to client side and generated on the fly for exclusive use by the client/user. It's shape is not ideal for a generic map type as it can become deeply hierarchical.

What I need to do is store the resulting client side JSON blob against the user on the server side. I specifically do not want any validation or type checking done on this JSON blob server side except for checking for valid JSON. At the moment I'm storing it as stringified JSON against an attribute in server side schema which does not seem ideal.

So I'm very much in favor of JSONObject/RawObject/UncheckedObject or whatever as simple JSON object as proposed here: https://github.com/graphql/graphql-js/pull/172.

[jvliwanag]

It looks like the use case of @miracle2k can be solved by just using a list. So something like:

  Item {
     titles {
        language
        text
     }
  }

wherein titles is an array. One can always create an input argument if you want to select a subset.

@clintwood 's use case however looks different since there's no known schema and may be hierarchical.

IMO, as long as there's a known structure, a list type could replace a map.

[sonewman]

@jvliwanag correct me if I am wrong, but this would mean that the values in the list would have to return as an array. This is OK if someone is developing an API from scratch and has control over defining the response payload.

But if an existing API is being moved over to use GraphQL, which already has a defined contract returning a map of key value pairs (even if the values are always of a defined object type / structure), then it appears this is unavoidable.

I am interested to know how it would be possible to use a list, unfortunately the the list type does not seem to accept anything except an array: graphql/graphql-js/blob/master/src/execution/execute.js#L679-L683.

Interestingly the error:

 User Error: expected iterable, but did not find one.

Suggests it could be possible to supply a Map or Set, but of course the code says otherwise :crying_cat_face:

[D1plo1d]

+1. Maps would be useful in mutations for sending arbitrary key/value pairs.

[mincedmit]

+1 as well, imagine this can also allow for embedded documents in query responses if using a record store on the backend?

[leebyron]

Support in the JS library itself for arbitrary iterables rather than just Arrays is coming soon.

There are significant tradeoffs to a Map type vs a list of key/value pairs. One issue is paginating over the collection. Lists of values can have clear pagination rules while Maps which often have non-ordered key-value pairs are much more difficult to paginate.

Another issue is usage. Most often Map is used within APIs where one field of the value is being indexed, which is in my opinion is an API anti-pattern as indexing is an issue of storage and an issue of client caching but not an issue of transport. This anti-pattern concerns me. While there are some good uses for Maps in APIs, I fear that the common usage will be for these anti-patterns so I'm suggesting proceeding with caution.

For the original post, I think there are two good ways to solve this API puzzle:

First is what @OlegIlyenko suggests. That you explicitly ask for the languages which you care about fetching. If you need multiple you can use aliases to query for multiple. This may be the right path if you know up front which you want to query.

item {
  title: title(lang: $custom_lang)
  title_en: title(lang: "en")
  title_fr: title(lang: "fr")
}

Second is returning a list of tuples. This may be the right path if you don't know up front which you want, or if you specifically want them all.

item {
  titles {
    language
    text
  }
}

such that you might get the result:

"item": {
  "titles": [
    { "language": "en", text: "Hello" },
    { "language": "fr", text: "Bonjour" }
  ]
}

[akuckartz]

Please try to use JSON-LD language maps: http://www.w3.org/TR/json-ld/#index-maps

[dylanahsmith]

I agree with @leebyron about the solution to the original problem.

In the case of user-defined JSON data, can we just make it clear that custom scalars aren't restricted to how they can be serialized. E.g. with graphql-js you can serialize a custom JSON scalar using arrays, objects, etc.

[jackielii]

alternatively:

item {
  titles(languages: ["en", "fr"])
}

result:

"item": {
  "titles": ["Hello", "Bonjour"]
}

You can specify the list of language in a variable

Advantage of this approach:

1. static query: no need to specify languages before writing the fragment (versus approach 1 in @leebyron 's comment
2. didn't create a new Object type. (versus approach 2 in @leebyron 's comment)

[Nebulai]

+1 for map support. This "anti-pattern" logic seems like over-thinking it to me. Sure some people will abuse it but that is true for anything..

[LucasIcarus]

+1, map support is useful for some situations that the system always return less data than expect, I think.

[juancabrera]

+1 Currently I'm dealing with an API that data is arbitrary (as users can create their own contentTypes on the CMS), so there is no way I can create an entryType as I have no idea what fields are in there. Having a mapType will be super useful for these cases.

Update: Just tried this and is working great so far!: https://github.com/taion/graphql-type-json (thank you @taion).

[jarwol]

I have the same use case as @juancabrera. Our API delivers content defined and managed through a custom CMS, and a main feature is being flexible and dynamic. In the application, we model the objects as a base set of common fields, and then a "fields" map with all of the dynamic properties that we don't know about on the server.

{
  "contentId": 123456,
  "type": "video",
  "fields": {
    "title": "fake video 1",
    "releaseDate": "...",
    ...
  }
}

We'd like to give our client app devs the ability to query and transform the content in the way they want, but without having to create a strict schema on the server.

[th-ko]

+1 for maps. In my use-case I have objects of this shape (in typescript):

interface GenericObject { 
  id: string, 
  label: string,
  types: string[], 
  objectProps: {[key: string]: GenericObject[]}
  dataProps: {[key: string]}: string[]} 
}

Using the list of key-values would mean I am doing a transformation on the server to send the data and then do the reverse transformation on the client to build up that map again. Using the JSON value type would mean I can not fetch related objs (via objectProps) with their respective fields. Using arguments would mean I need to know all the properties in advance, which is not possible.

Also the size of data we talk about here is rather small. So paging is not an issue.

[rncry]

We have the exact same need as @jarwol

[aweiker]

I am in agreement with @leebyron after watching what has happened in other "JSON" type apis at my company. The path of least resistance is the path most often traveled.

This means that if there is a choice of creating a new type that has more semantic meaning to the UI or creating a map that has no tooling support or contract, but is "quick" to create on the server, then a map is going to be used.

[GroofyIT]

I would personally opt for 2 seperated types. As I see it there are 2 use cases of the data: 1) view, the data is shown to a user in his/her 's preferred language 2) admin, the data is to be edited (translations added / changed / removed) by an administrative user

This might come across as perhaps anti-pattern, though in my opinion it is not. Since these are 2 completely separated concerns / representations even if the source for both is the same in your db.

Thus: 1) VIEW: Type is the minimal set, language is given as a variable and resolver is responsible of choosing the correct one to return. Changes are high that you want localised data for more types then just "Item", therefore using a variable is also reusable across your schema queries, mutations, ...

type Item {
   title: String
 }

Selected by:

query ($language: String!) {
    Item(language:$language) {
        title
    }
    variables {
        language:'en-US'
    }
}

2) ADMIN: Type is extensive administrative set. Here you can choose your own preferred medicine: Array based

type Language {
    code:String
    text:String
}
type Item {
    title: [Language]
}

Unified languages type (listing all possible?)

type Languages {
    nl-BE:String
    en-US:String
    fr-FR:String
    nl-FR:String
}

type Item {
    title:Languages
}

Plain object

scalar Object

type Item {
    title: Object
}

[limscoder]

We have the same issue as @OlegIlyenko: user defined data types. Would be interested to hear how other users are tackling this.

[amannn]

I have a similar use case. Instead of the value of the Map being a simple string, I'd need GraphQL types as I'd like to use field resolvers there.

Consider a schema like this:

type Book {
  id: ID
  name: String
}

type User {
  name: String

  # This is invalid, but roughly what I'd need
  favouriteBooks: Map<String, Book>
}

A response should look like this:

{
  name: "Jane Doe",
  favouriteBooks: {
    science: {
      id: '1'
    },
    novel: {
      id: '2'
    },
    ...
  }
}

The book category names are dynamic and change often, therefore I'd like to not specify them in the GraphQL response. Note that this is just a contrived example. I'm not looking for a solution to this particular problem.

I understand the value of using a list instead, but it would be great to use this to work with clients / server responses that were designed before the GraphQL layer was in place.

[serle]

I would like to support ES6 Map construction directly from json. The ES6 Map constructor accepts and array of Entries where the Entry "type" is a two element array where the elements are of different underlying types one for the key and one for the value e.g. [string object]. I can't do this currently in GraphQL.

[wincent]

@amannn: Even "with clients / server responses that were designed before the GraphQL layer was in place", isn't a schema like the following similarly easy to produce/consume?

type Book {
  id: ID!
  name: String
}

type User {
  name: String
  favouriteBooks: [FavoriteBook]
}

type FavoriteBook {
  category: String
  book: Book
}

The obvious use case for having favouriteBooks be a map would be to do O(1) look-up by category name, but if that's what I care about I can create a look-up structure easily enough.

const booksByCategory = {};
books.forEach(({category, book}) => booksByCategory[category] = book);

[amannn]

@wincent Thanks for your reply!

Sure, the consumption of a list is no problem. But to use this with existing infrastructure, I'd need to do the following this:

• In my case, my GraphQL layer wraps existing REST endpoints. I'd have to transform favouriteBooks in REST responses into a list, so they match the schema.
• Either I rewrite the parts of my UI code that consume favouriteBooks to work with the new data structure, or I have to transform them back into a map in a wrapper component. If I do the latter and the UI components have a callback to update the data, I have to transform them back into a list before calling a mutation (given a similar input type as outlined above).
• When the GraphQL layer receives the mutation, it has to transform the list back into a map in order to pass it on to a REST endpoint.

So it could work, but what I'm trying to say is that it will probably end up in quite a bit of back-and-forth transformation which increases complexity. It would be great if the GraphQL- and the UI layer could use the same data structure as they are already known to REST endpoints.

[serle]

@wincent. If we ignore the fact that we can treat {} as a map of sorts and use the new ES6 Map object, my current code avoids things like:

books.forEach(({category, book}) => booksByCategory[category] = book);

in favor of:

map = new Map([ entry1, entry2, ... ]) where each entry is [key:string, value:object]

that way I get the underlying library to do all the looping for me. Since this is now part of the language and many languages have a map construct, surely graphql schema should support a construct like this more directly. If we can achieve the Entry construct then a map is just a list of Entries, however this is very JavaScript specific

[wincent]

If we can achieve the Entry construct then a map is just a list of Entries, however this is very JavaScript specific.

True, @serle. GraphQL is very intentionally language-agnostic, so it's good to point that out.

[taion]

There's nothing JS-specific about building maps from lists of key-value pairs, though. That's the standard map constructor in a huge number of programming languages, from C++ down to Python.

That said, though, it does feel really awkward to represent maps as key-value pairs. To the extent we're interfacing with e.g. REST APIs, it's an extra step in marshalling and unmarshalling data in the GraphQL layer, and it feels clunky.

[calebmer]

If you want maps or other interesting GraphQL transforms then I recommend you look at graphql-lodash. I don’t think there’s a strong argument (yet) for maps in the GraphQL specification, but client tooling can definitely fill the gaps! @wincent’s example could become a map with GraphQL Lodash with this query (assuming User is the root query type):

{
  favoriteBooks @_(
    keyBy: "category"
    mapValues: "book"
  ) {
    category
    book { name }
  }
}

Exactly the shape you want without having to change your UI code! If it turns out a lot of people use GraphQL Lodash for maps then maybe we should consider adding maps to the specification.

[taion]

I think the issue is more having seamless serialization/deserialization support on the client. Not sure Relay is quite there.

[serle]

Given that GraphQL fundamentally serves out data structures and most of them are now fundamentally baked into each language in order to promote standardization, I feel that GraphQL should have a rich type system to promote API clarity and eliminated unnecessary workarounds and transformations.

With regard to maps, maybe a more general approach is to introduce a Tuple type i.e. a fixed size array where each element has a predefined type which can either be a primitive or custom type. In this way we could achieve both the Entry concept and thus a Map as well as other more general use cases. However, I still feel that it is cleaner to explicitly have a Map type with standard language mappings.

[taion]

A tuple type isn't really relevant here because there's no particular relevant ordering between the key and the value in an entry struct.

[serle]

@taion for the ES6 map the key is always Entry[0] and the value is always Entry[1]

[xealot]

As a huge fan of Typescript, I am really enjoying trying to get a nascent GraphQL version of our API started. But, as many others have mentioned the lack of a basic Map type is a huge hinderance.

We don't always know what we're going to receive, especially in the case of user generated content.

And, unlike the JSONType mentioned above, or the list of tuples idea, I would prefer a fully typed solution as well.

Something like:

Map<String,Type>

I see this as a massive handicap for an API that deals with non-deterministic data.

[taion]

I don't think anybody's recommending the use of the JSON scalar type for typed data, BTW – for now the best workaround is to serialize as a set of entries.

I still agree that it would be better if GraphQL had first-class support for mapping types, like Flow, TypeScript, &c.

[miracle2k]

Note that in practical terms, I was able to define a custom scalar and just output the map as a JSON structure. This worked fine with relay classic, but doesn't with relay modern, since the code tries to process these nested objects here: https://github.com/facebook/relay/blob/9a401c625ad6bb4632478c14c9f54eb1f47179d9/packages/relay-runtime/util/recycleNodesInto.js#L19

[Waitak]

I have a use case with a slight wrinkle. My back end is running Django + graphene. The backend has a model with a field that is stored in the database as JSON. The front end is Angular 4. It constructs a JSON map representing what amounts to a parse tree for a small expression language, and then wants to pass the tree to the back end. Obviously it can be stringified before the GraphQL call and destringified on the back end, but it would be very nice to avoid that, with the added benefit of letting the platform ensure that the front end is really passing a valid JSON map to the back end.

[amannn]

I just stumbled upon another use case: highlights in elastic search responses. They look like this:

{
  "source": {
    "message": "Hi Jane!",
    "recipient": {
      "name": "Jane"
    }
  },
  "highlight": {
    "message": [
      "Hi <em>Jane</em>!"
    ],
    "recipient.name": [
      "<em>Jane</em>",
    ]
  }
}

Here message and recipient.name are paths to fields in the source that was found. You could try to create a SearchResponseHighlight type that contains all the fields but I can think of the following issues:

• I guess it's easy for this to get out of sync as at least in my case the type of the source is defined somewhere else as the types related to search.
• Queries for retrieving such data would look a bit redundant and the user has to make sure that highlight is always in sync with the requested fields.
• GraphQL doesn't support field names with dots in them currently.

[stubailo]

I think a map type can be quite useful to replace something like:

{
  map {
    key
    value {
      fieldOnValue
    }
  }
}

Into:

{
  map {
    fieldOnValue
  }
}

I think this might also make some kinds of connections simpler, since the edge cursor could maybe just be a key on the object?

One question to answer would be if the map is ordered or unordered. Not all languages support ordered maps so some cases would still have to use the first approach.

[acjay]

I was initially shocked at the lack of a native map type in GraphQL. It really kind of defeats the concept of GraphQL serving data in exactly the format in which clients need it.

Now, I feel differently. As is, aside from type recursion, there is only one unbounded type in GraphQL, which is List. This actually makes it a lot more tractable to think about how to build abstractions on top of GraphQL.

The cost of this is that clients have to manage their own data. But maybe this is as it should be. After all, while GraphQL is meant to reproduce graphs of data, in fact, queries are trees. You bring your own graphification. Maybe maps and other higher-level structures should similarly be deserialized out of more elemental pieces. Then something like a directive could be used in the schema to specify that a given list field should be interpreted as a map, either by pulling out a given field as the key or identifying key and value fields of entries.

[xealot]

I don't think this changes the fact that being able to limit values in a map would be a huge benefit. If the argument is that maps are harder to reason about I don't buy it.

Think of maps as keys not with simple values but with huge objects underneath. If you're only interested in the name of each person (Map<id, Person>) it would be more efficient to simply ask for the name.

mapOfPeople {
   name
   photo {
    small
  }
}

But, it's impossible to model the data THROUGH the map. So anything that has a set of dynamic keys has to be emitted as json data and is opaque to the client.

[taion]

I wonder if there's a way to get best of both worlds as in protobuf: https://developers.google.com/protocol-buffers/docs/proto#maps

[alkismavridis]

+1 I was really surprized that there is no way to produce a hashMap with graphQL. I really loved graphQL and I was ready to convince my colleagues to migrate from Rest to GraphQL.

But the absence of maps would make my argument imposible. We would have to massively refactor our frontend application.

I appreciate the people who claim that this is an antipattern. But I personally belive that there is a reason why we use both in computer science. Some times the right thing do is using an array, and some times is using a hashmap. There is not one answer here. So, I really believe that is should be up to the users to decide.

Some people (like us) have already frontend applications that use Maps all over the place. Some people simply like it.

Sometimes accessing stuff like userGroups[ myUser.groupId ] is easier and cleaner than doing the more "manual" version: userGroups.find (g => g.id == myUser.groupId) It seems much more performent too (in js at least)

And after all, the idea behind graphQL is that the client asks the structure that it needs.

I really encourage the graphql people to consider this.

[Bessonov]

I don't see this as an anti-pattern. Map, Dict, Object, KeyValue(Stores) are ubiquitous. Why? Because they makes our life easier. And access faster.

I just experimenting with GQL in an app which use map-like structures... of course we can represent a map with a tuple and convert it in both directions, but, well, why? It's so painful!

And after all, the idea behind graphQL is that the client asks the structure that it needs.

A big +1. If a car is designed to get you from A to B, then to push the car seems very strange.

[DylanVann]

You often want to work with normalized data on the client. It can be easier and more performant.

Something like getting a few users and merging them into a redux store is simpler if they are returned as Map<string, User>. These are still well defined types.

[acjay]

@DylanVann But that's also an argument for not having a formal Map type. We use Redux too, at my company. We ingest data from the network via our reducer layer into a normalized representation in the Redux store, and then use a selector layer to present a more convenient denormalized view of the data to our React view layer.

From that perspective, it's only very slightly more difficult to ingest an array of entities by adding them to Redux's key-value representation than it is to merge a map of entities. It's basically something like Object.assign({}, prevState, ...inputArray.map(obj => { [obj.id]: obj })), which is only slightly more difficult than Object.assign({}, prevState, inputMap).

My point is, either way, you usually have to have some sort of ingestion of the raw data from GraphQL. Adding a native Map type to GraphQL makes ingestion slightly mildly easier but multiplies by some small constant factor the complexity of all tooling, while not actually reducing complexity of GraphQL schemas themseleves (field of type map<ID, Entity> vs field of type [Entity]). I would posit that the best extensions to GraphQL are the things that significantly impact that last point.

[alkismavridis]

@acjay, Sure, I could convert the data myself, from a list to a map. You are right on this. Likewise, I could survive if there were no aliases in graphQL. I could convert the field keys myself on the client, to match my needs. But, I cannot imagine anyone who would prefer writing JS code to apply aliases, instead of defining them in the graphql query. The second is much simpler, more clear and more straight-forward.

Same is true for maps. Yes, we can do it ourselves, but it would be great if we didn't have to.

[DylanVann]

You can use something like normalizr on the client to deal with arrays being returned, but it would be nicer in some cases if the API could just represent a Map.

The existence of a workaround for not having a feature isn't an argument for not having the feature.

edit: It actually could be an argument, but in this case I think the workaround is overly complex.

[taion]

Tricky without something like generic types or extensions of lists in GraphQL, though. How do you know if a list is supposed to be a map?

[psigen]

This missing Map type is a problem for me as well.

I recognize that any fundamental change to the GraphQL spec is difficult, but the lack of this feature is particularly problematic because it's supported in many of the type systems and services that GraphQL interfaces with.

This leads to implementing custom resolvers on the server, followed by custom transforms on the client, to deal with situations where my server is sending a Map, and my client wants a Map, and GraphQL is in the middle with no support for Maps. Yes, it is possible, and I have done it, but it is fair bit of boilerplate and abstraction that seems to defeat the purpose of writing the API spec in GraphQL.

Note that storing data in this way is actually recommended by services such as Firebase -- it is amenable to mutations or referencing of subsets of items in the collection. You don't need to ensure consistency of the index location over time as other elements are added or removed. This is a common pattern in APIs where pagination is not required and/or element ordering is not important to the spec.

(This is the case I am facing as well: I am returning a set of objects from which I individually reference items for further updates, meaning that with an array I must re-index the payload into a map on the client to avoid an O(N) search for each update.)

---

Here are some examples of how this type is handled in specifications that have close adjacency to GraphQL. This is a hypothetical object Apple with a field seeds containing one or more objects indexed by a string key. Apologies in advance for any minor typos.

// Example Apple Payload (JSON)
{
  "seeds": {
    "alpha": { "weight": 1 }, 
    "beta":  { "weight": 2 },
    "gamma": { "weight": 3 },
  }
}

Typescript

https://www.typescriptlang.org/docs/handbook/interfaces.html#indexable-types

interface Apple {
  seeds: {
    [key: string]: {
      weight: number;
    };
  };
}

Javascript + Flow

https://flow.org/en/docs/types/objects/#toc-objects-as-maps

type Apple = {
  seeds: {
    [string]: {
      weight: number
    }
  }
}

Python (via typings)

https://www.python.org/dev/peps/pep-0484/#the-typing-module

class AppleSeed:
  weight: int

class Apple:
  seeds: Dict[str, AppleSeed]

JSONSchema (Used by MongoDB)

https://spacetelescope.github.io/understanding-json-schema/reference/object.html#pattern-properties

{
  "type": "object",
  "properties": {
    "seeds": {
      "type": "object",
      "patternProperties": {
        ".+": {
          "type": "object",
          "properties": {
            "weight": "integer"
          }
        }
      }
    }
  }
}