Compatibility with opensearch

[johndpope]

https://opensearch.org/docs/latest/clients/python-low-level/

Going to attempt to swift over the base elasticsearch over to opensearch-py

UPDATE good news - switched these lines and everything still works.

manager.py

from opensearchpy import OpenSearch
from opensearchpy.exceptions import RequestError

https://gist.github.com/johndpope/0f77156bfc1d43bd0c23df0d34d86b01

tests seem to be passing fine.

now to attempt to wire up to opensearch end point.

[johndpope]

might make more sense to rename this repo activityfeeds-opensearch (I do love elasticfeeds / but activityfeeds probably get more traction) can keep class names the same.

[qlands]

Cool! Can you make a pull request with your changes? Then I can change the readme and add some parameters to support either ES or OpenSearch.

[johndpope]

so lot of progress - some bumps. would like to wrap this up in a bow - but consider this is going to be in parts. there's probably 5-6 tickets that need to be separated. I fixed it all - so it's working with aws lambda - had to do some surgery on manager.py

to get the opensearch to actually work - I attempted to overload the manager - but eventually gave up and used a (bring your own connection) hack https://gist.github.com/johndpope/370753393ab8c0bc5ea6bc13983e5d8b

the manager then revealed itself that it was doing many / all over connection = create_connections() so i just added some tests to test if there was an existing connection - solved that problem.

I use a few helpers to create aws lambda layers - I attached to gist

https://gist.github.com/johndpope/370753393ab8c0bc5ea6bc13983e5d8b#file-manager-py-L637

The logging on aws lambda in python is really bad - fails so hard so often - using aws_lambda_powertools makes it way more usable.

so my approach makes elastic search obsolete - but opensearch is working.

I also chopped black from it - haven't used this before. I see it's specific to ci/cd.

fyi - here's some testing from opensearch https://github.com/opensearch-project/opensearch-py/blob/main/test_opensearchpy/test_connection.py

lambda execution from gist above

with the powertools - the logging successfully captures dictionaries without blowing up.

in your opensearch / dashboard - here is the button to show all

https://gist.github.com/johndpope/370753393ab8c0bc5ea6bc13983e5d8b#file-test-json