[Doc] improve documentation of opensearchpy.client.bulk() (the bulk() function's return value is always empty)

[calebplum]

What is the bug?

When inspecting the return value of the opensearch-py bulk() function, it returns the following value (for example when writing 70 items):

>>> response = bulk(client, items, index=opensearch_index)
>>> print(response)
(70, [])

No matter what I try, the list in the return value's second item is always empty. I'd like to understand what this list is for and how I can handle errors returned from Opensearch.

Is there any existing documentation that describes this? The bulk() function's documentation doesn't have any information about its return value.

How can one reproduce the bug?

Make a call to the opensearch-py bulk() function and inspect its response.

What is the expected behavior?

I would like to know what the list in the return value is supposed to represent and how I can check for and handle errors returned from Opensearch.

Also, is there any way to access the underlying response from Opensearch?

What is your host/environment?

Ubuntu 22.04

Do you have any screenshots?

N/A

Do you have any additional context?

N/A

[saimedhi]

Hello, @calebplum. Thank you for pointing out the gaps in the documentation. We will update it soon.

I tried using the example in the userguide. I got response as below:

Adding bulk documents: {'took': 195, 'errors': False, 'items': [{'index': {'_index': 'index-2022-06-08', '_id': '1', '_version': 1, 'result': 'created', '_shards': {'total': 2, 'successful': 1, 'failed': 0}, '_seq_no': 0, '_primary_term': 1, 'status': 201}}, {'index': {'_index': 'index-2022-06-09', '_id': '2', '_version': 1, 'result': 'created', '_shards': {'total': 2, 'successful': 1, 'failed': 0}, '_seq_no': 0, '_primary_term': 1, 'status': 201}}, {'index': {'_index': 'index-2022-06-10', '_id': '3', '_version': 1, 'result': 'created', '_shards': {'total': 2, 'successful': 1, 'failed': 0}, '_seq_no': 0, '_primary_term': 1, 'status': 201}}]

Here "took" refers to time taken to process bulk request in milliseconds, "errors"= False implies no errors, "errors"=True implies has errors "items" gives the imformation about each individual operation status.

[calebplum]

Thank you for providing that information @saimedhi.

Can you confirm which version of opensearchpy was used in your example? I'm using v2.2.0.

If it's not a version issue, can you please help me troubleshoot why my opensearchpy client might be returning different output to yours?

[calebplum]

After further troubleshooting I've discovered my issue was that I was using opensearchpy.helpers.bulk() rather than opensearchpy.client.bulk(). The output I described in my original post appears to be expected.

I'll close this issue now.

[dblock]

@calebplum is there something we should do to help the next person not run into this issue?

[calebplum]

@dblock Here's a couple things I can think of that would help:

• Update the documentation for the opensearchpy.client.bulk() method to include its return value and an error-handling example.
• Add documentation for opensearch.helpers.bulk(), including its return value and an error-handling example. For reference, here's the documentation for the equivalent Elasticsearch method. This doesn't explicitly show its return value, however it does explain what it's supposed to return and how to access errors returned in the Elasticsearch response. It would be helpful to include all these details in the equivalent Opensearch documentation.
• Include return values for the existing examples in the Opensearch User Guide for both opensearchpy.client.bulk() and opensearchpy.helpers.bulk(). It'd also be helpful to include error-handling examples, but I'm not sure if this would be the best place for that. I could probably raise a PR to include the return values if you'd like, but I'm not sure what the error-handling best practices are for these methods.

[dblock]

@calebplum Makes sense. Would you care to contribute? I'm going to reopen this issue and rename it to a doc one.

[saimedhi]

@calebplum, Please feel free to take up this issue and raise a PR with updated docs or user guide. Thanks :)

[tannerjones4075]

I would like to work on this issue. I will work through the documentation.

[tannerjones4075]

Could you assign this issue to me?

[dblock]

@tannerjones4075 Are you still working on this?

[tannerjones4075]

@tannerjones4075 Are you still working on this?

Yes, I am still working on this.

[Plutone11011]

@dblock related to this, I think other methods of opensearch.client may benefit from similarly upgraded documentation, like delete or create . I would gladly contribute to this, should I open a new issue or a PR referencing this issue is sufficient?

[wbeckler]

Feel free to open a related issue. I'm sure your contributions will be appreciated

On Sat, Apr 13, 2024, 12:36 PM Lorenzo Borelli @.***> wrote:

@dblock https://github.com/dblock related to this, I think other methods of opensearch.client may benefit from similarly upgraded documentation, like delete or create . I would gladly contribute to this, should I open a new issue or a PR referencing this issue is sufficient?

— Reply to this email directly, view it on GitHub https://github.com/opensearch-project/opensearch-py/issues/380#issuecomment-2053696145, or unsubscribe https://github.com/notifications/unsubscribe-auth/AA5PRLURNOJORV377SWUDW3Y5FNIDAVCNFSM6AAAAAAXRNKO6OVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDANJTGY4TMMJUGU . You are receiving this because you are subscribed to this thread.Message ID: @.***>

[dblock]

@dblock related to this, I think other methods of opensearch.client may benefit from similarly upgraded documentation, like delete or create . I would gladly contribute to this, should I open a new issue or a PR referencing this issue is sufficient?

Yes please!