search

tailcallhq / tailcall

High Performance GraphQL Runtime
https://tailcall.run
Apache License 2.0
1.26k stars 237 forks source link

JIT fails in conformance tests #2740

Open karatakis opened 3 weeks ago

karatakis commented 3 weeks ago

Prerequisites

Describe the bug

When I enable JIT, all tests should pass, but when I do, I get a set of tests failing. The current implementation does not fail in any of the tests.

Steps to reproduce

  1. cargo test --test execution_spec --features force_jit

Expected behavior

No failing tests.

Actual behavior

The following list of tests fail:

karatakis commented 3 weeks ago

graphql-conformance-http-004.md

This test fails because JIT fails to prepare the query parameters for all query parameters. It just succeeds to prepare for the first query parameter.

Expected HTTP Requests

 INFO GET http://localhost:3000/user?id=4 HTTP/1.1
 INFO GET http://localhost:3000/pic?id=4&size=64&width&height HTTP/1.1
 INFO GET http://localhost:3000/pic?id=4&size=1024&width&height HTTP/1.1

Actual HTTP Requests

 INFO GET http://localhost:3000/user?id=4 HTTP/1.1
 INFO GET http://localhost:3000/pic?id=4&size&width&height HTTP/1.1
 INFO GET http://localhost:3000/pic?id=4&size&width&height HTTP/1.1

Notice the size=... part

This problem appears in the following tests too:

karatakis commented 3 weeks ago

graphql-conformance-http-006.md

This test fails because JIT @expr directive is not implemented correctly.

Given the following query, we get the wrong result for the current implementation and JIT.

This problem appears in the following tests too:

{
  user(id: 4) {
    friends(first: 10) {
      id
      name
      profilePic(size: 50)
    }
    mutualFriends(first: 10) {
      id
      name
      profilePic(size: 50)
    }
  }
}

Expected Result

{
  "data": {
    "user": {
      "friends": [
        {
          "id": 1,
          "name": "friend_1",
          "profilePic": "1_50__"
        },
        {
          "id": 2,
          "name": "friend_2",
          "profilePic": "2_50__"
        },
        {
          "id": 3,
          "name": "friend_3",
          "profilePic": "3_50__"
        }
      ],
      "mutualFriends": [
        {
          "id": 1,
          "name": "mutual_friend_1",
          "profilePic": "1_50__"
        },
        {
          "id": 2,
          "name": "mutual_friend_2",
          "profilePic": "2_50__"
        },
        {
          "id": 3,
          "name": "mutual_friend_3",
          "profilePic": "3_50__"
        }
      ]
    }
  }
}

Actual Result

{
  "data": {
    "user": {
      "friends": [
        {
          "id": 1,
          "name": "friend_1",
          "profilePic": "1___"
        },
        {
          "id": 2,
          "name": "friend_2",
          "profilePic": "2___"
        },
        {
          "id": 3,
          "name": "friend_3",
          "profilePic": "3___"
        }
      ],
      "mutualFriends": [
        {
          "id": 1,
          "name": "mutual_friend_1",
          "profilePic": "1___"
        },
        {
          "id": 2,
          "name": "mutual_friend_2",
          "profilePic": "2___"
        },
        {
          "id": 3,
          "name": "mutual_friend_3",
          "profilePic": "3___"
        }
      ]
    }
  }
}

Notice the profilePic field

karatakis commented 3 weeks ago

graphql-conformance-http-005.md

The JIT implementation does not format custom types into String, instead, it returns null

Expected value:

{
  "data": {
    "customer": {
      "id": 4,
      "name": "Tailcall",
      "date_of_birth": "2000-01-01"
    }
  }
}

Actual value:

{
  "data": {
    "customer": {
      "id": 4,
      "name": "Tailcall",
      "date_of_birth": null
    }
  }
}

Notice the date_of_birth field

karatakis commented 3 weeks ago

graphql-conformance-http-013.md

This fails because introspection is not implemented for JIT

The same problem for: graphql-conformance-013.md

karatakis commented 3 weeks ago

All of those cases except 013 case are fixed by https://github.com/tailcallhq/tailcall/pull/2709

karatakis commented 1 day ago

Introspection will be supported in later PR