Query all fields of a type in a simple expression [wildcard]

[AhmadAudeh]

Let us say this is the type :

type Human {
  id: String
  name: String
  homePlanet: String
}

Then to query all the fields the query is :

query HumanNameQuery {
   Human {
     id
     name
     homeplanet
    }
 }

Should be something like this exist ??

query HumanNameQuery {
   Human {
     *
   }
 }

[rmosolgo]

I think this was left out of GraphQL on purpose. Although it seems convenient at first, it may actually lead to some maintainability troubles down the road. Consider some possibilities:

• A field is removed.

  Let's say the database changes and Human.homeplanet doesn't exist anymore. Any query Human { * } would execute successfully, but the client may encounter an error because it depends on homeplanet, which is gone now.

  By requiring an explicit list of fields, you get notified right away that a query is invalid -- you don't have to trace the error down through the client or UI.

• A field is added.

  Imagine that Human.friends is added, which returns a list of other Human objects. Does Human { * } include those objects? If so, any client which requests Human { * } but doesn't need friends has caused a lot of waste!

  It's better to request exactly which fields you need so that the server doesn't do any extra work. This also allows backend developers to implement resource-intensive fields without the fear that they would be requested by accident.

For those two reasons, it's better to make the client request fields explicitly!

[dschafer]

Yep, this was indeed deliberately omitted for the reasons @rmosolgo mentioned!

In places where we've needed things like this (a good example might be a Date type where you pretty much always want to fetch year, month, and day), we could create a fragment DateFields on Date { year, month, day } and use that everywhere that we would have used * before.

[AhmadAudeh]

@rmosolgo : thanks for your explanation, it makes sens. @dschafer : seems a usable solution, I will try that, thanks.

[amunna]

Interesting reasoning to remove the "*". It's nice to explicitly mention the search fields but it wouldn't hurt to have an "wild card" option just to explore the fields. To me this is just limiting the possibilities with search and doesn't make sense to me.

[acjay]

@amunna I agree that it's awkward sometimes to issue a bunch of queries to explore an API, rather than have an easy way to see a sample of all of the data. But this seems to me like an issue that should be solved in clients like GraphiQL. It could automatically expand a * to be all fields that can be called without arguments. Which, by the way, is another wrinkle. GraphQL doesn't currently assign any special meaning to fields with or without arguments, so it's a little arbitrary that introducing a wildcard in the language itself would create that difference.

[MorganConrad]

@rmosolgo Your point#2 (adding a field) is a red herring. If the client cares about performance, they can still be explicit in their fields. If it is just an exploratory client who doesn't care about performance, they could use "*" (were it supported). Exactly how SQL has done it for a long time.

Your point#1 has some validity, but, again, if the client really cared, they could be explicit.

[fsc7]

One more to disagree with the absence of the possibility of a wildcard like '*' to retrieve all fields. This feature is very useful if you are exploring an API.

[tomprogers]

I want a wildcard so that I can fetch a dictionary from the server. This is not exploratory, it's a real case that comes up when internationalizing a web application.

I don't know the names of the fields, so this is not solvable using fragments.. Converting the structure to an array of key:value on the server and then back into a dictionary on the client is painful when using apollo tooling, and also makes it much clearer that I'm just working around arbitrary shortcomings of GQL semantics.

query {
  user {
    firstname
    lastname
    preferredLanguage
  }

  languages {
    code
    label
  }

  strings {
    * // I want strings to be a dictionary of key:value pairs
  }
}

[juancampa]

@tomprogers for those cases you might want to model your strings field as a list of key-value pairs

[PSAustin]

We need to return some very deep, fully hydrated, object graphs. Without this feature that task is very laborious. This could even be a deal breaker for our current project where we were planning to use GraphQL.

[stubailo]

I think this is a duplicate of: https://github.com/facebook/graphql/issues/101

The ideal solution here IMO is to introduce a way to return objects, rather than make all objects able to be queried without specifying fields.

[jlouis]

GraphQL currently has the property that the client is explicit in everything it queries. This suggestion would mean we lose this property. Thus, it has to be done with a lot of care: clients using * will eventually open themselves up to getting lots of data they didn't ask for. Also, the server has no way of knowing which fields might be good candidates for deprecation, if the * query is used.

You can always define scalar JSON and just return a blob of JSON in place of a query. You cannot query inside the structure, but it can be arbitrary, deeply nested objects returned from it.

Currently, the best bet is to model it with a parameter to the query:

query Q($locale : LocaleCode) {
    user(login: "foo") {
        givenName(locale: $locale)
        familyName(locale: $locale)
        fullName(locale: $locale)
    }
}

Personally, I think what is needed is some kind of "implicit parameters", so you can define an implicit binding over a Selection Set. That means, for every subquery, you have access to the implicit parameter, if it has not been overridden. In turn, you can set a locale high up the query and then have everything under in scope pick up the implicit param naturally.

This is akin to digging out a Accept-Language header from the HTTP query and stuffing it into the context of the GraphQL execution, using it as a default value.

As I've said elsewhere: GraphQL is a programming language. You send small programs to the server for it to interpret, and that language could as well contain a let-bind construct of some kind.

[acjay]

@jlouis

As I've said elsewhere: GraphQL is a programming language. You send small programs to the server for it to interpret, and that language could as well contain a let-bind construct of some kind.

^ This.

I wrote about this in detail: http://artsy.github.io/blog/2018/05/08/is-graphql-the-future/

[tomprogers]

you might want to model your strings field as a list of key-value pairs

@juancampa No, I don't want to model my strings that way. I thought I made that clear in my original post.

[mjmahone]

It would probably be more productive to add a concrete proposal on a new issue rather than re-hashing a nearly three year old, closed thread. Especially as it sounds like there are ~3-4 different potential problems and solutions that have come up here:

• Having "*" somehow be used for "explore available fields": I don't know what that looks like as part of the language specification (as opposed to a client utility), so I'd be interested in seeing a concrete idea around that. That idea should be careful not to compromise forwards-compatibility (i.e. as the server adds or removes fields, the response shape shouldn't change: this would indicate it should be tooling built on top of GraphQL rather than core to GraphQL's specification, I think).
• Figuring out performance vs. developer ergonomics around a "wildcard" approach.
• Fetching a dictionary from a server. This may or may not be best solved with a wildcard: you can use things like a "Dictionary" scalar field to do this. Additionally, there may be room to clarify in the specification how to resolve this (for example using key-value objects in a sorted array, which is isomorphic to a dictionary), but again, that seems like a different request than this issue.
• Defining variable bindings somehow (maybe this would involve fragment variable definitions?)

Each of these all sound like interesting ideas that should be discussed! I'd be interested in digging deeper into each of these proposals, but it's difficult to do without having the discussion derailed when there are many different threads pulling apart on a closed, only-loosely-related issue.

[Danilo-Araujo-Silva]

This is somehow nonsense. When we are developing and we need to inspect a table on a database, for example, we don't write the 20 columns a table may have, we use "*"? ^^ Here should be the same.

[unleashit]

Plus one. Learning that a fields wildcard isn't possible, and that the thread to request has even been closed is a disappointment. I understand the reasoning for wanting to keep it explicit, but there are times, especially when one is testing or wanting to develop quickly, that it would be no less than a god send to be able to say "just gimme all the fields". The fragment doesn't help with this use case.

What about a possible solution of a flag on the server that enables support? The default could even be not to accept if you wanted to keep it strict.

[Jaikant]

I was searching for a wildcard option as well, sad it isn't yet available.

[brian-1901]

To be honest, you can make fragments work which leads to better organization of your code. 🤷‍♂️

[unleashit]

No need to lecture friend. Sometimes better organization isn't the goal. Sometimes speed is. Read my comment again. If you feel like over engineering should be enforced on everyone and for all use cases, you have the right to your opinion but we disagree.

[brian-1901]

Guess if your goal is speed, than my comment doesn't apply to you.

[rodrigopires]

Plus one for having the wildcard possibility implemented.

[FrankSD]

This is non-sense.

[winni4eva]

This issue was closed with an issue that linked back to this issue..nice.

[jdspugh]

NoSQL databases are based on graph data structures rather than tables. We should be able to query any graph node in a NoSQL database and receive its entirety, including children, to the depth we want with its query language (GraphQL). If you think in graph terms, you are simply returning a graph node and its children which is one of the most fundamental, basic and efficient operations on any graph data structure.

Anybody used jQuery? It's a graph query language also. The most common query operation $('#node-id') returns the object, its attributes and its children without specifying what the attributes and children are upfront. This should have an equivalent in GraphQL.

[armaandh]

I agree with the suggestion that at least while developing..we should have a way to be able to quickly wildcard all the fields. This is a nice productivity boosting during rapid development, and does not necessary imply it should be something used in production.

[boazsegev]

I'm fairly certain that exposing an "all fields" (*) operator should also be considered a security risk when exposing large data-sets.

Seriously, what is the amount of data provided by a User(id: X) {*} query on facebook? all posts, all likes, all links... how much data would that be? How easy would it be for a client to mistakingly send such a query?

I'm just starting to learn GraphQL and I wanted a * operator for my exploration... but I think the world is safer without it.

[unleashit]

Just adding my 2 cents a final time. What sort of bugs me about this isn't really the actual issue, but the broader problem of big tech companies who've in recent years taken over open source from grassroots groups who would usually listen to their communities a bit more. Of course they're going to write software in a way that works for them first and that's their right. But in FB's case I'm not a fan of how they have an agenda to try to subject everyone else to their opinions. Like don't even think of writing html to the DOM. If you're want to try that (say the project isn't paying enough to parse/react.createElement all of the WYSIWYG output, etc.), then we're going to make you look like a stunt man by forcing you to use dangerouslySetInnerHTML. In this case, they don't even give the option.

I agree with you @boazsegev that having a wild card in general is counter to the point of GraphQL. What I disagree with, is that it wouldn't be useful in some cases like quickly exploring an API when developing, quick toy projects, etc. This could easily be solved with a flag in the same way Typescript deals with stuff like no implicit any, strict null checks, etc. In the scheme of things, not a big deal. But Obviously a lot of ppl are ending up on this thread going WTH.

[aoakeson]

I honestly dont see the need for this. There are so many tools and libraries out there that will show you your schema, and even have auto complete. I dont see a practical use case that you need to display 20+ fields even for developing something. From my experience I only need a few fields at a time.

I dont see how this saves any time. It only opens up a lot of issues.

[fsc7]

I'm fairly certain that exposing an "all fields" (*) operator should also be considered a security risk when exposing large data-sets.

Seriously, what is the amount of data provided by a User(id: X) {*} query on facebook? all posts, all likes, all links... how much data would that be? How easy would it be for a client to mistakingly send such a query?

@boazsegev I don't think this is an argument because the fields are exposed anyhow. And you can make the same request on purpose if you have the authorization to do so. This must be controlled with authorization controls and through limitations in how deep a query can go in the graph. By not implementing it you are just limiting the ease of use for developers. If your argument was valid then '*' wildcard in SQL should also be banned and also JOINS etc and that's not the point.

[boazsegev]

@fsc7 , You are right, I'm not thinking about malicious attacks from behind an authentication / sanitization layer - I'm thinking of that new developer that just got into the team and wants to look at the schema or an example record or to try some shortcut and how their developer access can bring down the production backend...

... and I'm not even inventing this s#$t. Development APIs can be a double edged sword.

Besides, the fact that SQL has the * operator doesn't really mean we should propagate past mistakes :-p

[jdspugh]

I can develop more rapidly and without repeating myself (DRY) with a *. Have a option to switch on a 'strict mode' which disallows it if that suits your purposes.

[acjay]

I'm still not clear -- does anyone actually want to use this in code? Because I do my API exploration in tools like Graphiql. It would be straightforward to just make this a feature there.

[unleashit]

For those who may not understand why people would want this, imagine you want to quickly prototype something. Say a user profile. You know you're going to need most of the user fields, and at least for now your concern is not production... but getting something on the screen that looks halfway right. With REST, it's not problem... you write less code and can get there faster. Graphiql (the CURL -X/Postman for gql) isn't going to help with that and last time I checked, it still requires you to autocomplete fields one by one, which is a bit of a pain when there are a ton of fields. This is a relatively minor gripe I admit, but it's how I've worked for "years" with REST and don't see why GraphQL can't be made to work for everyone (again, I'm talking about prototyping or a quick Codesandbox or something, I personally would "never" want a wildcard in production).

The one further thing I can think of worth adding is if this ever gets added to the spec (doen't look promising ATM), in my opinion a wildcard should only affect fields for that resolver. Not cascade down to children, which could be a problem if the schema is deep.

[jdspugh]

@unleashit Yes, that's exactly my use case. I have a user profile I want to display and edit. My code picks up the fields to render from the db. This way I just need to add a new field to the database and it appears in my app without any coding required.

With regards to security issues, permissions should be specified on an object, field and user basis separately to queries. This is far more efficient, effective and powerful than trying to restrict or disable queries in some manner. If permissions set in this manner the full power of GraphQL queries can be unleashed, including '*' operators and queries that traverse to any depth (you can restrict the overall return size of a query on the db end if this is a concern). This way I can even define subobjects for my user profile and have my app render those in my app on a subscreen.

There will be public permissions (which could be set to 'none' in the case of a private db), and per user permissions which specify which object and fields are available for each user role. I'm using Graph.Cool and it has this type of functionality.

[unleashit]

@jdspugh you should be allowed to do that if you want whether some disapprove, I definitely agree. That said, I think a big part of the advantage of GraphQL is type safety. IMO unless you are referring to team process, having to specify the fields is less about security and more about ensuring say that if someone adds a field to the schema, it doesn't push out to the client by surprise (or vice versa). Even if its just you, that's a possible source of a bug. I'm glad that GQL lets us run a tighter ship, but a tools should be flexible enough when we need them to be.

I could be wrong (I'm pretty new to GQL), but I don't think security is a problem with wildcards (vs REST), except that it could allow a hacker to send a more resource intensive DoS type attack if the schema was big. I would assume on the server end you're gong to control which fields you expose and to who. A wildcard shouldn't be able to bypass that. That's different than SELECT as someone said, since databases will typically spit everything out (SELECT I agree is dangerous, but a godsend from a tinkering point of view!)

[jdspugh]

For sure, both options should be supported. Specifying which fields to return, and wildcards (including traversal depth specification).

[IvanGoncharov]

Since this issue became the most discussed closed issue I think we need clarify why it was closed.

From the top of my head, here are just a few reasons why we can't add * in its pure form:

• It makes adding computationally expensive fields (e.g. aggregations, sentiment analysis, etc.) extremely risky.
• By adding new fields you inflate payloads of existing clients
• You lose the ability to track if someone still using deprecated fields so you can't safely remove them.
• If your API using field-based cost analysis by adding new fields you can break old clients

So as you can see these concerns are not security related but instead focused around the evolution of your schema.

But I completely agree with the point of developer productivity. GraphiQL for long time include this feature:

But I think we need to extend it and make it more discoverable for new users (I personally found out about it by reading source code) and hopefully forming GraphQL Foundation will accelerate GraphiQL development.

[nodkz]

@IvanGoncharov I suggested adding { * } to GraphiQL, Playground here https://twitter.com/nodkz/status/1106395018573557761

If the user typed {*} in the query tab and press Enter, it should be replaced by all fields.

The current behavior - add the only required fields, not all. But users want to request all fields.

[nodkz]

With such thing, when users will ask - have GraphQL a *. We may answer, yep it has it in GraphiQL 😂

And it will be really what they want.

[fsc7]

@nodkz I think it is a good in-between solution for this problem. And should not be hard to implement.

[IvanGoncharov]

@nodkz I don't think having a special syntax that works only in Playground is a good idea. It will cause a lot of confusion for new GraphQL users since query that worked in the Playground stops working when you sent it from source code. It also can lead to some people thinking it's part of official syntax and fragment the community.

I think CMD+* shortcut would serve the same purpose. GraphiQL already supports collapsing query so it could be collapsed by default With * UI element instead of ↔

[jdspugh]

In the two most widely used graph query languages you can get all properties on nodes with these commands. I don't see any good reason why GraphQL should be different:

In Cypher MATCH (n) RETURN n

In Gremlin g.V().valueMap()

[IvanGoncharov]

@jdspugh GraphQL also has no support for aggregation, predicates, deep data traversing, graph specific operations and many other features that other query languages have.

It's because Cypher, Gremlin, SQL etc. are query languages for Databases. GraphQL is:

A query language for your API

And that's why GraphQL was designed with a different set of constraints and tradeoffs in mind. For example, being able to extend your Schema without breaking existing clients is extremely important for GraphQL.

[AhmadAudeh]

This discussion has been going on for a good time now. Even thought that I would be very happy to have the wildcard option, but I have to say that I agree with the GraphQL team to not adding it. Having a more consistent, functional and a higher quality product is more important than making life easier for us as developers. The most pretty thing about GraphQL is that you are getting EXACTLY what you are asking for in a clear and explicit way, keeping in mind that GraphQL is an API query language existed to describe how to ask for/manipulate data. Wildcard option without any conditions would make complications regarding type composition and recursive queries in addition to become a bug generator and a performance reducer with uncareful use or while developing multiple versions of a product. I agree it is a little bit of pain, but I think it does worth it and saves a lot of pain later on.

[jdspugh]

Completely disagree. I'm a full stack developer. I know exactly what code I'm generating on the server and on the client. Why would I want to double my work by defining/changing my schema on the server and then on the client as well? I can do it once on the server and have the client pick up the changes. Is faster, less maintenance and less potential for errors.

Sure I understand GraphQL is an API query language. As I see it there are two extra concerns as an API query language: security and server side resource usage. Security we already dealt with in earlier comments and it appears nobody has issues with it. For resource usage you can use API keys and/or user accounts to rate limit anyone who goes over predetermined system resource limits such as bandwidth and CPU. No need to reduce the potential power of sophisticated queries that high end developers can harness.

[fgeorgsson]

Wildcards would make all of the queries we run fit comfortably into a GET request which would make CDN caching a lot simpler. Over fetching is not a problem in this application, lacking CDN cache is the problem.

[hi2u]

These "you might shoot yourself in the foot" concerns are valid, at the "per-project" level depending on what the use case, and who the consumer is.

But having this as a limitation on the entire the protocol itself seems a bit short sighted to me, if you take the broader use cases into account.

Not just regarding this wildcard question itself... there's many large assumptions being made here that everyone is doing exactly the same thing with it, has exactly the same priorities, and only has same types of consumers.

Is GraphQL really only going to be something for frontend-only devs only to write hardcoded queries to servers that someone else controls... forever?

Yes performance (by not over-fetching) is one of the reasons GraphQL was initially created. But in certain cases, not having wildcards is going to make your system slower anyway (more on this below).

But what about all the others reasons... power, flexibility, recursive structures and less hard-coding everywhere? These are why I'm using GraphQL.

For anyone doing anything where they want to loop over whatever all the current fields are (and which are expected change in the future without further redundant frontend changes being needed also), you're actually making things much more complex and breakable for them by forcing them to write their own code generators for simple things that could be done with * in almost any other similar query language.

Now I need to recompile and redeploy my entire frontend every time I add a field to the system, even though the frontend is actually looping over what it receives from the server anyway. Either that, or slow everything down with more GraphQL queries that shouldn't be needed at all - wasn't 'preventing extra queries' also one of the reasons GraphQL was created?

A couple of analogies...

SQL

Performance is also one of the most important features of SQL databases too... so does that mean that the SQL standard should disallow SELECT * ?

I don't see how this is any different. In fact you do see recommendations everywhere not to overuse SELECT *in SQL, for the same reasons. And it's good advice, when relevant.

But imagine not having it at all, over "you might hurt yourself" concerns by the people that wrote the original SQL standards decades ago.

Should you let any and all 3rd parties run queries that would do SELECT * on all your tables? Of course not, but you put that limitation in place elsewhere, where it's appropriate. No need to handicap the entire query language itself.

cmd.exe/powershell/bash shells

Giving the user the power of using wildcards means they might mess up and delete a file they didn't mean to. I also run into performance problems and failures when I use wildcards in a shell sometimes. So should wildcards be removed to protect us? ...forcing us all to write our own (likely more buggy) workarounds.

The future of the protocol

I see big things for GraphQL beyond the typical common "frontend webdev hard coding things" use-case it was originally intended for. It would be a shame if the protocol itself is going to box itself into this one-and-only use-case.

Personally I'm using it with Postgraphile, I'll be using it with 100+ tables, and 100s of SQL views - many of which are auto-generated. Like many projects, I'm working on both frontend and backend, and the admin parts of my system need to show all the fields defined in the SQL VIEWs being shown. There's also backend-to-backend use cases too, and we're starting to see more of this with things like Prisma - which can't even be used from a browser to begin with anyway.

And while I don't use NoSQL much myself, one issue is that they all have their own query languages - something GraphQL could allow them to standardize on, assuming it doesn't box itself into it's original use case only. Even SQL is starting to do more recursive stuff with JSON now, and recursive CTEs etc. They could potentially open up to include native GraphQL querying too, without needing the current proxies like Postgraphile - depending on how GraphQL presents itself.

And in general, many of us are moving away from old paradigms of large swathes of very-similar copy-and-pasted imperative code for every table/form/field in our system to a more declarative way of doing things where a lot of this stuff is looped over from the base canonical definitions. Having to define the same fields redundantly in multiple places is the antithesis of this.

So for these kinds of use cases, the only options I can think of right now are...

• Copy and pasting fields names from your backend into your frontend (100s or 1000s for some systems)
• Run twice as many queries every time (introspection query + the data query)... kind of defeats the whole "prevents-over-fetching" thing - slowing the system down... also leads to potential race condition and caching issues
• Write your own code generators to redundantly store field lists in your frontend - then recompile and redeploy your whole frontend for every minor change
• Go back to using plain REST or some other non-standardized protocol :(

For me, GraphQL is a big piece of these new ways of doing things, but right now this limitation is forcing me to generate more bloated code or run twice as many queries as I should need to in some areas. Both of which harm performance, and create more opportunities for bugs.

This limitation is also going to make it harder for people to build general purpose GUI tools to use with GraphQL in general, which is a bit of a shame. And those tools that do get made are going to be slower because of this.

The point @pcguru made above is a good one here too, one of the big concerns people have with GraphQL is the added difficulties in caching. So having more potential solutions to this is a good thing, even if they're not appropriate for everything.

Whether or not your own server should accept wildcards is a project-level or who-is-consuming decision, and even then it likely makes sense to allow it in some places (internal/admin/automated/dev usage), and not others (3rd party + hardcoded frontend dev consumers).

Nobody is making an argument that every server should allow any query from everyone. Of course in most cases, if they shouldn't be allowed to do that, you're not going to let them. We already have things like persisted query whitelisting to handle this + more, and in the appropriate place.

We're just saying handicapping the entire protocol specification itself isn't where that limitation should be enforced for absolutely everything, regardless of use case or skill level.

[acjay]

Wildcards would make all of the queries we run fit comfortably into a GET request which would make CDN caching a lot simpler. Over fetching is not a problem in this application, lacking CDN cache is the problem.

Have you considered storing queries? There's no rule that you must send the entire query document over. The fact that queries have optional names and fixed parameters is tailor made for letting a GraphQL server store query documents via POST and execute them via GET, using URL query parameters to populate GraphQL query parameters.

[tejasrivastav]

When using NoSql databases tend to utilize the schema-less feature for storing unstructured data. With Graphql already inplace with almost all api(s) Any suggestion on how should you approach to structures like

{
    "title": "Form",
    "stages": [{
        "type": "Input",
        "label": "Name 1",
        "value": "default",
        "X-custom-field": "for type input"
    },
    {
        "type": "Choice",
        "name": "Name 2",
        "value": null,
        "choices": ["a","b","c"],
        "X-custom-field": "for type choice"
    }]
}

any other dynamic field that could be added to the structure dynamically from UI.