Using bulk insert without command / Click

[AubinMoreau]

Hi, sorry to bother you with a question which is probably a simple one but is that possible (and if it is, how ?) to launch the code without using click or command and without giving every arguments ?

I just want to do this :
bulk_insert(graph, host, port, nodes, relations)

Without putting all parameters like this : bulk_insert(graph, host, port, password, nodes, relations, separator, max_token_count, max_buffer_size, max_token_size, quote, field_types, skip_invalid_nodes, skip_invalid_edges)

But for now, the only way i found is (as you do on your test) by using Click :

from click.testing import CliRunner
from redisgraph_bulk_loader.bulk_insert import bulk_insert

def addToGraph(port,nodes,relations,graphName):
    runner = CliRunner()
    res = runner.invoke(bulk_insert, [
        "--port", port,
        "--nodes", nodes,
        "--relations", relations,
        graphName])

Thanks for your help.

[jeffreylovitz]

Hi @AubinMoreau,

That is a good question that I had not previously considered!

The default arguments are set as Click options, then the actual parameters for bulk_insert does not have defaults: https://github.com/RedisGraph/redisgraph-bulk-loader/blob/master/redisgraph_bulk_loader/bulk_insert.py#L362

As such, you currently cannot call bulk_insert without specifying all parameters, though this can easily be fixed by adding the appropriate defaults on the bulk_insert signature.

However, you will still encounter issues if avoiding Click entirely, because we use Click's progress bars to wrap CSV processing: https://github.com/RedisGraph/redisgraph-bulk-loader/blob/master/redisgraph_bulk_loader/bulk_insert.py#L202

The bulk loader wasn't written to expose the function bulk_insert, so I expect your current approach will cause some problems. That said, it seems like a sensible strategy! I will give some thought to how to make Click optional and let bulk_insert work as a standalone function rather than just as a script.

[AubinMoreau]

Thanks for your quick reply @jeffreylovitz !

I will continue to use click if there's no other way, i just didn't wanted to do it without exploring other options (if i get satified by the first solution i found, i won't progress).

Have a nice day !

[ghukill]

I'm curious if there has been any additional thought on using this utility from a python interperter context, vs calling as a command line option? more specifically, without using Click?

Or, if another approach for bulk loading has emerged since then?

Either way, it remains a great utility!