sagastume / boltx

Neo4j driver for Elixir
Apache License 2.0
24 stars 5 forks source link

Error getting a node with a Datetime field #61

Closed aakashjhawar closed 7 months ago

aakashjhawar commented 8 months ago

I got the following error while running the query. Boltx.query(Boltx, "MATCH(n) RETURN n")

After debugging more, I found out if the query has MATCH command in it, then it throws error.

** (DBConnection.ConnectionError) bad return value: {:error, %{code: :failure, message: %FunctionClauseError{module: Boltx.PackStream.Unpacker, function: :unpack, arity: 1, kind: nil, args: nil, clauses: nil}}}
    (db_connection 2.4.3) lib/db_connection.ex:1267: DBConnection.handle_common_result/3
    (db_connection 2.4.3) lib/db_connection.ex:1508: DBConnection.run/6
    (db_connection 2.4.3) lib/db_connection.ex:644: DBConnection.parsed_prepare_execute/5
    (db_connection 2.4.3) lib/db_connection.ex:636: DBConnection.prepare_execute/4
    (boltx 0.0.4) lib/boltx.ex:149: Boltx.do_query/4

Can you please help me?

sagastume commented 8 months ago

Thanks for your report, @aakashjhawar.

I don't think the issue is related to the MATCH clause; most likely, it's a structure or data type that Packstream doesn't recognize. There might be a node in your data containing a structure or data that boltx cannot decode.

I attempted to reproduce the problem with different datasets both on Neo4j Aura and locally, but I couldn't replicate it.

I can suggest two ways to obtain more information:

  1. Instead of obtaining all the nodes with "MATCH(n) RETURN n" you can segment the queries to find which nodes cause the problem.
  2. The second thing is to be able to activate the logs

To investigate further, we could try to gather additional information through the logs. Please make sure that when sharing logs, no sensitive or confidential information is included.

Incorporate this configuration into the config.exs or test.exs file as needed.

#config/test.exs
# Print only warnings and errors during test
config :logger, level: :debug

config :boltx,
  log: true,
  log_hex: false

config :boltx, Bolt,
  hostname: "127.0.0.1",
  auth: [username: "neo4j", password: "password"],
  user_agent: "boltxTest/1",
  pool_size: 1,
  prefix: :default,
  scheme: "bolt",
  port: 7687,
  name: Boltx

Navigate to your project's test/test_helper.exs file and add the following:

#test/test_helper.exs
ExUnit.start(capture_log: true)

Add a file to perform a test with the query.

test/{your_project}_boltx_test.exs

#test/boltx_test.exs
defmodule Blog.BoltxTest do
  use ExUnit.Case, async: true

  describe "MATCH" do
    @tag :debug
    test "get all nodes" do
      response = Boltx.query!(Boltx, "MATCH(n) RETURN n")
      assert %Boltx.Response{results: []} = true
    end
  end
end

Run: mix test --only debug

The output will be something similar to this:


     The following output was logged:
     02:22:16.985 [debug] C: MESSAGE_TYPE ~ ["MATCH(n) RETURN n", %{}, %{mode: "w", bookmarks: [], db: nil, tx_metadata: nil}]
     02:22:16.994 [debug] S: SUCCESS ~ %{"fields" => ["n"], "t_first" => 2}
     02:22:16.997 [debug] C: MESSAGE_TYPE ~ [%{n: -1, qid: -1}]
     02:22:17.003 [debug] S: SUCCESS ~ %{"bookmark" => "FB:kcwQGIH38AzVQcSbFJ8oxrKsLQOQ", "db" => "neo4j", "t_last" => 0, "type" => "r"}

As a reminder, please ensure that logs do not contain private or sensitive information.

aakashjhawar commented 8 months ago

Hi @sagastume, thank you for your response.

I found out the data type that gives an error. It's datetime(). When I create a node with TIMESTAMP() temporal function, it works as expected. However when I use DATETIME(), it throws an error because of Packstream.

I have been using Bolt.Sips, but since it does not support the latest versions of Bolt, I am looking for alternative libraries. DATETIME() works fine in Bolt.Sips (might be because it doesn't use PackStream) but fails if I use Boltx.

Steps to replicate it:

If you create a node using TIMESTAMP, then you will get the user if you want to read it from database.

Boltx.query(Boltx, "CREATE (user:User{name: 'David', created_at: TIMESTAMP()})")
Boltx.query(Boltx, "MATCH (user:User{name: 'David'}) RETURN user")

But if you create a node using DATETIME, then you will get Packstream error.

Boltx.query(Boltx, "CREATE (user:User{name: 'John', created_at: DATETIME()})")
Boltx.query(Boltx, "MATCH (user:User{name: 'John'}) RETURN user")

Error:

** (DBConnection.ConnectionError) bad return value: {:error, %{code: :failure, message: %FunctionClauseError{module: Boltx.PackStream.Unpacker, function: :unpack, arity: 1, kind: nil, args: nil, clauses: nil}}}
    (db_connection 2.4.3) lib/db_connection.ex:1267: DBConnection.handle_common_result/3
    (db_connection 2.4.3) lib/db_connection.ex:1508: DBConnection.run/6
    (db_connection 2.4.3) lib/db_connection.ex:644: DBConnection.parsed_prepare_execute/5
    (db_connection 2.4.3) lib/db_connection.ex:636: DBConnection.prepare_execute/4
    (boltx 0.0.4) lib/boltx.ex:149: Boltx.do_query/4

%{"domain":["otp"],"erl_level":"error","error":{"initial_call":null,"reason":"** (DBConnection.ConnectionError) client #PID<0.977.0> stopped: ** (DBConnection.ConnectionError) bad return value: {:error, %{code: :failure, message: %FunctionClauseError{module: Boltx.PackStream.Unpacker, function: :unpack, arity: 1, kind: nil, args: nil, clauses: nil}}}\n    (db_connection 2.4.3) lib/db_connection.ex:1267: DBConnection.handle_common_result/3\n    (db_connection 2.4.3) lib/db_connection.ex:1508: DBConnection.run/6\n    (db_connection 2.4.3) lib/db_connection.ex:644: DBConnection.parsed_prepare_execute/5\n    (db_connection 2.4.3) lib/db_connection.ex:636: DBConnection.prepare_execute/4\n    (boltx 0.0.4) lib/boltx.ex:149: Boltx.do_query/4\n    (stdlib 3.17.2) erl_eval.erl:685: :erl_eval.do_apply/6\n    (elixir 1.14.5) src/elixir.erl:297: :elixir.eval_forms/4\n    (elixir 1.14.5) lib/module/parallel_checker.ex:110: Module.ParallelChecker.verify/1\n    (iex 1.14.5) lib/iex/evaluator.ex:329: IEx.Evaluator.eval_and_inspect/3\n    (iex 1.14.5) lib/iex/evaluator.ex:303: IEx.Evaluator.eval_and_inspect_parsed/3\n    (iex 1.14.5) lib/iex/evaluator.ex:292: IEx.Evaluator.parse_eval_inspect/3\n    (iex 1.14.5) lib/iex/evaluator.ex:187: IEx.Evaluator.loop/1\n    (iex 1.14.5) lib/iex/evaluator.ex:32: IEx.Evaluator.init/4\n    (stdlib 3.17.2) proc_lib.erl:226: :proc_lib.init_p_do_apply/3\n\n    (db_connection 2.4.3) lib/db_connection/connection.ex:198: DBConnection.Connection.handle_cast/2\n    (connection 1.1.0) lib/connection.ex:810: Connection.handle_async/3\n    (stdlib 3.17.2) gen_server.erl:695: :gen_server.try_dispatch/4\n    (stdlib 3.17.2) gen_server.erl:771: :gen_server.handle_msg/6\n    (stdlib 3.17.2) proc_lib.erl:236: :proc_lib.wake_up/3\n"},"error_logger":{"report_cb":"&:gen_server.format_log/1","tag":"error"},"logging.googleapis.com/sourceLocation":{"file":"gen_server.erl","line":949,"function":"gen_server.error_info/7"},"message":"GenServer #PID<0.1167.0> terminating\n** (DBConnection.ConnectionError) client #PID<0.977.0> stopped: ** (DBConnection.ConnectionError) bad return value: {:error, %{code: :failure, message: %FunctionClauseError{module: Boltx.PackStream.Unpacker, function: :unpack, arity: 1, kind: nil, args: nil, clauses: nil}}}\n    (db_connection 2.4.3) lib/db_connection.ex:1267: DBConnection.handle_common_result/3\n    (db_connection 2.4.3) lib/db_connection.ex:1508: DBConnection.run/6\n    (db_connection 2.4.3) lib/db_connection.ex:644: DBConnection.parsed_prepare_execute/5\n    (db_connection 2.4.3) lib/db_connection.ex:636: DBConnection.prepare_execute/4\n    (boltx 0.0.4) lib/boltx.ex:149: Boltx.do_query/4\n    (stdlib 3.17.2) erl_eval.erl:685: :erl_eval.do_apply/6\n    (elixir 1.14.5) src/elixir.erl:297: :elixir.eval_forms/4\n    (elixir 1.14.5) lib/module/parallel_checker.ex:110: Module.ParallelChecker.verify/1\n    (iex 1.14.5) lib/iex/evaluator.ex:329: IEx.Evaluator.eval_and_inspect/3\n    (iex 1.14.5) lib/iex/evaluator.ex:303: IEx.Evaluator.eval_and_inspect_parsed/3\n    (iex 1.14.5) lib/iex/evaluator.ex:292: IEx.Evaluator.parse_eval_inspect/3\n    (iex 1.14.5) lib/iex/evaluator.ex:187: IEx.Evaluator.loop/1\n    (iex 1.14.5) lib/iex/evaluator.ex:32: IEx.Evaluator.init/4\n    (stdlib 3.17.2) proc_lib.erl:226: :proc_lib.init_p_do_apply/3\n\n    (db_connection 2.4.3) lib/db_connection/connection.ex:198: DBConnection.Connection.handle_cast/2\n    (connection 1.1.0) lib/connection.ex:810: Connection.handle_async/3\n    (stdlib 3.17.2) gen_server.erl:695: :gen_server.try_dispatch/4\n    (stdlib 3.17.2) gen_server.erl:771: :gen_server.handle_msg/6\n    (stdlib 3.17.2) proc_lib.erl:236: :proc_lib.wake_up/3\nLast message: {:\"$gen_cast\", {:stop, #Reference<0.3299003733.3598581762.19225>, %DBConnection.ConnectionError{message: \"client #PID<0.977.0> stopped: ** (DBConnection.ConnectionError) bad return value: {:error, %{code: :failure, message: %FunctionClauseError{module: Boltx.PackStream.Unpacker, function: :unpack, arity: 1, kind: nil, args: nil, clauses: nil}}}\\n    (db_connection 2.4.3) lib/db_connection.ex:1267: DBConnection.handle_common_result/3\\n    (db_connection 2.4.3) lib/db_connection.ex:1508: DBConnection.run/6\\n    (db_connection 2.4.3) lib/db_connection.ex:644: DBConnection.parsed_prepare_execute/5\\n    (db_connection 2.4.3) lib/db_connection.ex:636: DBConnection.prepare_execute/4\\n    (boltx 0.0.4) lib/boltx.ex:149: Boltx.do_query/4\\n    (stdlib 3.17.2) erl_eval.erl:685: :erl_eval.do_apply/6\\n    (elixir 1.14.5) src/elixir.erl:297: :elixir.eval_forms/4\\n    (elixir 1.14.5) lib/module/parallel_checker.ex:110: Module.ParallelChecker.verify/1\\n    (iex 1.14.5) lib/iex/evaluator.ex:329: IEx.Evaluator.eval_and_inspect/3\\n    (iex 1.14.5) lib/iex/evaluator.ex:303: IEx.Evaluator.eval_and_inspect_parsed/3\\n    (iex 1.14.5) lib/iex/evaluator.ex:292: IEx.Evaluator.parse_eval_inspect/3\\n    (iex 1.14.5) lib/iex/evaluator.ex:187: IEx.Evaluator.loop/1\\n    (iex 1.14.5) lib/iex/evaluator.ex:32: IEx.Evaluator.init/4\\n    (stdlib 3.17.2) proc_lib.erl:226: :proc_lib.init_p_do_apply/3\\n\", severity: :error, reason: :error}, %{code: :failure, message: %FunctionClauseError{module: Boltx.PackStream.Unpacker, function: :unpack, arity: 1, kind: nil, args: nil, clauses: nil}}}}\nState: {Boltx.Connection, %{code: :failure, message: %FunctionClauseError{module: Boltx.PackStream.Unpacker, function: :unpack, arity: 1, kind: nil, args: nil, clauses: nil}}}","severity":"ERROR","time":"2024-01-19T11:24:12.664Z"}

Thank you.

sagastume commented 8 months ago

Thank you for your reports; they have helped make Boltx more stable.

I have managed to reproduce the problem by following your instructions. I will work on fixing the problem.

Thank you.

sagastume commented 8 months ago

In the Bolt v5 version, two new temporary structures have been added.

  1. Datetime structure is added to replace Legacy Datetime.
  2. DateTimeZoneId structure is added and replaces Legacy DateTimeZoneId.

Driver clients will not notice the difference. The decoding is done to the structures: DateTimeWithTZOffset for Datetime and TimeWithTZOffset for DateTimeZoneId.

This fix will be in Boltx version 0.0.5

Cypher example:

cypher_create = """
CREATE (user:User{
  uuid: $uuid,
  date_time_offset: DATETIME('2024-01-20T18:47:05.850000-06:00'),
  date_time_with_zona_id: DATETIME('2024-01-21T14:03:45.702000-08:00[America/Los_Angeles]'),
  time: TIME("15:41:10.222000000Z"),
  time_with_offset: TIME("15:41:10.222000000-06:00")
})
#...
assert {:ok, "2024-01-20T18:47:05.850000-06:00"} ==
               DateTimeWithTZOffset.format_param(date_time_offset)

assert "2024-01-21 14:03:45.702000-08:00 PST America/Los_Angeles" ==
               DateTime.to_string(date_time_with_zona_id)

assert {:ok, "15:41:10.222000+00:00"} == TimeWithTZOffset.format_param(time)

assert {:ok, "15:41:10.222000-06:00"} == TimeWithTZOffset.format_param(time_with_offset)

"""
aakashjhawar commented 8 months ago

Thank you @sagastume. I'll test out the latest version once it's published on Hex.

sagastume commented 8 months ago

Thanks to you, @aakashjhawar, your tests has been a great help.

The version has been released :).