chores: port some of the official Neo4j driver tests to Bolt.Sips, using boltkit

[florinpatrascu]

The official Neo4j drivers are using more and more the support offered by the boltkit toolkit, and we too would like to use it.

boltkit is extremely useful for development, offering a great test-bed for simulating server responses in various scenarios. You can see how the above mentioned drivers are implementing their tests with it, for example these "mocks".

To integrate boltkit in our tests, we'd have to start the boltkit (stub)server at the beginning of our tests, and kill/stop it when they end; we could most probably use System.cmd/3, for bootstrapping it. Then, with the boltkit server running as a background process, we can send and receive scripted scenarios, making our tests depending much-much less on having various Neo4j server instances, of different versions, clustered servers or not clustered, simulate nodes going away or corrupted routing tables and so on. The later being a very tedious and time consuming process, especially given the hardware resources eaten up from our dev machines, when in conjuncture with docker/docker-compose & Co. 😢

We'd appreciate any support for donating from your skills and dev time to improving our development environment and for also making our core-driver development easier and more robust nevertheless.

Thank you!
❤️

[dominique-vassard]

Very very nice idea. I've already know about bolt_kit but get stuck at the point where I asked for AWS credentials to have a dev cluster running in one command but never get any answer back. Anyway, I didn't look much into boltstub but it's really interesting, especially if we can use the scripts already written by Neo4j's team.

I've just spent some time today on the subject and it seems that there's a little more involved than expected:

• as you said, boltstub should run on background, this means something like a GenServer or something. I've made a quick test with Task.async (and a Process.sleep...) and it works almost fine. There's just some --- Logging error --- Python error popping
• It will be necessary to start new DbConnection, I don't know if this is problematic or not (especially if think about running tests concurrently -> port mess ahead maybe)
• ConfigAgent should be more flexible and not rely only on config

Some of these concerns are closely related to something I wanted to implement after you completed routing: the ability to connect to multiple database. Then I don't know if now is the right to add the bolt_kit support or if it's better to wait for routing and all its induced changes. First option leads to merge hell, the second leads to test pain for you.
Maybe the better should be to work directly on your dev branch (on branch checkout from your dev branch precisely) when you think it's stable enough.

[florinpatrascu]

boltstub should run on background

actually I was thinking to wrap it in its own .sh scrip and/or starting it at the beginning of the tests, something like this:

options = [
  :binary,
  :stream,
  :use_stdio,
  :stderr_to_stdout,
  :exit_status,
  args: ["-v", 7687, "init_stubs.bolt"]
]

case System.find_executable("boltstub") do
  boltstub when not is_nil(boltstub) ->
    Port.open({:spawn_executable, boltstub}, options)
  _ ->
    {:error, :no_boltstub}
end

so no GenServer, in this case. Mind you, the above is not tested :)

It will be necessary to start new DbConnection

not needed either, if the scenario above works?!

ConfigAgent should be more flexible and not rely only on config

The ConfigAgent is gone :) The "routing" code acts as a "hub" for connections, and each connection maintains its own configuration; the latter inferred from the initial user's own. I hope I have this version stable enough very soon, as per our earlier chat.

I believe the approach I suggested above, using Port or maybe even System.cmd, could be experimented with without even needing to start the driver, or at least that's what I thought when I opened this request for help. But you're very right, given the new routing code, the changes are too many to reimport the master in, unless there are some hotfixes we absolutely need.

I totally support your concern, about timing. I'll need few more days and have a working prototype that can be used for adding more meat to it. Unfortunately, I'll probably have one next weekend, as I am not sure how much time I can find after my day-job schedule ends. So let's wait few more days :)

Thank you so much for feedback!!

[dominique-vassard]

Your code works, but we still need to feed a new script to boltstub for each test. boltstub is just a dummy server that will match a client command and answer what the script told it to answer. And nothing more. If you consider the following script:

!: AUTO INIT
!: AUTO RESET

C: RUN "RETURN 1 AS num" {}
   PULL_ALL
S: SUCCESS {"fields": ["num"]}
   RECORD [1]
SUCCESS {}

It will fail for any query that is not RETURN 1 AS num with out parameters.

-> It will be necessary to start new DbConnection not needed either, if the scenario above works?!

boltstub is just a dummy server, we still need the client to perform connections and queries :)

And alas, it won't save us from using different "real" Neo4j versions, at least for writing the scenarii. We have those written by Neo4j teams but if we want more, we have to write them ourselves.... and know the subtle changes between the bolt version (and I'm not speaking of seemingly unused data in query/response)

But even with these considerations, it will be great for every new collaborator who work on the high level part of the driver to have those and not to have to run (as I do, as you may do) 3 (or more) Neo4j instances, event if it's easier now with Neo4j platform.

Considering a real server, boltkit has one-command server launch for single server, cluster and multicluster which is nice but require some AWS credentials....

[florinpatrascu]

BTW since the boltkit is written in python, this fresh off the press story might be relevant to our story here: Mixing Python with Elixir?!

[florinpatrascu]

hi just published this: https://github.com/florinpatrascu/fixex, maybe useful?! ;)

[florinpatrascu]

closing this as I am able to use the official test stubs provided by the Python driver, with the FixEx. Still tinkering at it, but this how a simple test looks like now, with FixEx:

defmodule Bolt.Sips.PythonStubTest do
  use Fixex,
    adapter: Fixex.Python,
    config: [
      python_path: [
        to_charlist("test/stub"),
        to_charlist("test/stub/boltkit")
      ]
    ]

  describe "Boltkit" do
    @fix [:server, :start, ["test/stub/scripts/return_1_in_tx.script", 9001]]
    fix "test with stub", ctx do
      neo_conf = [
        url: "bolt://localhost:9001",
        role: :boltkit
      ]

      assert %{fix: true} = ctx
      assert {:ok, neo} = Bolt.Sips.start_link(neo_conf)
      assert conn = Bolt.Sips.conn(:boltkit)

      assert %Bolt.Sips.Response{results: [%{"1" => 1}]} 
               = Bolt.Sips.query!(conn, "return 1")
    end
  end
end

useful for covering a good amount of use cases.

[florinpatrascu]

update:

• while the above works for some simpler use cases, and it's clean too, for the most complicated scenarios, the ones where the python support in boltkit is expecting generators, from us, the Erlport package fails - and since I am just an amateur in Python, I should probably delegate this to more Python-savvy troubleshooters. Will try using the stubs via Porcelain and see where I end up. Otherwise we might need to rewrite a lot of routing-related tests :/