rapidsai / gpu-bdb

RAPIDS GPU-BDB
Apache License 2.0
106 stars 43 forks source link

Investigate dask task objects being held b/w queries. #238

Open VibhuJawa opened 2 years ago

VibhuJawa commented 2 years ago

When using the benchmark runner it seems that there are some dask-tasks that are present b/w runs. We should investigate these objects.

This can be checked by adding client.has_what() after the query has been started.

print(client.has_what(), flush=True)
tables after drop dict_keys([])
tables after drop dict_keys([])
{'tcp://10.120.104.11:34983': (), 'tcp://10.120.104.11:37581': (), 'tcp://10.120.104.11:37865': (), 'tcp://10.120.104.11:37907': (), 

'tcp://10.120.104.11:38843': (), 'tcp://10.120.104.11:39089': (), 'tcp://10.120.104.11:39709': (), 'tcp://10.120.104.11:39811': (), 

'tcp://10.120.104.11:42075': (), 'tcp://10.120.104.11:43463': (), 'tcp://10.120.104.11:44441': (), 'tcp://10.120.104.11:44785': (), 'tcp://10.120.104.11:45285': (), 'tcp://10.120.104.11:45387': (), 

'tcp://10.120.104.11:45421': ("('rename-27090da64a4622f983c72e463860351d', 0)", "('wrapped_f-1c0784c4941eec03448ca1c143d2d63e', 0)", "('rename-aedc4989a050a4c2a2dbd7ca44429d6a', 0)"), 

'tcp://10.120.104.11:46109': ()}