ddrindex delete failing: AttributeError: 'Elasticsearch' object has no attribute 'delete_by_query'

[pkikawa]

running ddrindex delete results in attribute error

(ddrlocal) ddr@DDREditor:~$ ddrindex delete --recurse --index ddrpublic-20180713a --hosts 192.168.0.23 entity ddr-pc-41-51 --confirm
Traceback (most recent call last):
  File "/opt/ddr-local/venv/ddrlocal/bin/ddrindex", line 14, in <module>
    load_entry_point('ddr-cmdln==0.9.4b0', 'console_scripts', 'ddrindex')()
  File "/opt/ddr-local/venv/ddrlocal/lib/python2.7/site-packages/click/core.py", line 722, in __call__
    return self.main(*args, **kwargs)
  File "/opt/ddr-local/venv/ddrlocal/lib/python2.7/site-packages/click/core.py", line 697, in main
    rv = self.invoke(ctx)
  File "/opt/ddr-local/venv/ddrlocal/lib/python2.7/site-packages/click/core.py", line 1066, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/opt/ddr-local/venv/ddrlocal/lib/python2.7/site-packages/click/core.py", line 895, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/opt/ddr-local/venv/ddrlocal/lib/python2.7/site-packages/click/core.py", line 535, in invoke
    return callback(*args, **kwargs)
  File "/opt/ddr-local/venv/ddrlocal/lib/python2.7/site-packages/ddr_cmdln-0.9.4b0-py2.7.egg/DDR/cli/ddrindex.py", line 367, in delete
    click.echo(docstore.Docstore(hosts, index).delete(object_id, recursive=recurse))
  File "/opt/ddr-local/venv/ddrlocal/lib/python2.7/site-packages/ddr_cmdln-0.9.4b0-py2.7.egg/DDR/docstore.py", line 832, in delete
    return self.es.delete_by_query(
AttributeError: 'Elasticsearch' object has no attribute 'delete_by_query'

[pkikawa]

from geoff jost in slack: That's an error in ddr-cmdln's usage of the Python Elasticsearch API. Our dependencies are pegged at certain versions or version ranges. For elasticsearch it's elasticsearch>=2.0.0,<3.0.0. I haven't updated this in awhile, but there must have been an update that removed delete_by_query after I created that function.

[gjost]

Yep, they removed that function in this commit: https://github.com/elastic/elasticsearch-py/commit/fa62ae71bf068f9d1e97a415a6fe0fc4c32669b1#diff-2696ade2db2bc9506566dc21df9e00e8

[gjost]

There is a plaintive GitHub issue too -- looks like Elasticsearch moved delete_by_query to a plugin for some reason. I'll have to reimplement recursive delete.

[gjost]

Also note that I spoke to someone at the Elasticsearch booth at OSCON who was surprised somebody was still using the 2.* branch of the software.

When stuff like this happens I think about my experience with Elasticsearch and I remember certain articles I've read about how to implement full-text search in PostgeSQL or some actual database.

[gjost]

For now, let's peg elasticsearch-py at the last version that includes delete_by_query. Not a real solution but lets us do the other things we need to do right now.

[gjost]

Weird, could not find delete_by_query in the 2.x branch of elasticsearch-py.

Gonna have to go medieval and delete individually:

curl -XDELETE 'http://192.168.0.20:9200/ddrpublic-20180502a/entity/ddr-testing-40982-1'

[gjost]

Fixed in ddr-cmdln commit 78d3a94 for package ddrlocal-develop_2.8.9-3~deb9.

[sarabeckman]

Tested on ddrlocal-develop_2.8.9-3

Tried to run ddrindex delete --recurse --index ddrpublic-20180713a --hosts 192.168.0.23:9200 entity ddr-densho-12-8 --confirm

Received error: Usage: ddrindex delete [OPTIONS] OBJECT_ID Error: Got unexpected extra argument (ddr-densho-12-8)

[sarabeckman]

Tested on ddrlocal-develop_2.8.9-3

Ran ddrindex delete --recurse --index ddrpublic-20180713a --hosts 192.168.0.23:9200 ddr-densho-12-8 --confirm Worked as expected. Deleted entity, master, and mezzanine files.