Handle using the sync client in an async context for local tokenizers

[abdullahkady]

Addressing this: https://github.com/langchain-ai/langchain-cohere/issues/19

[harry-cohere]

We discussed and you'd like to try an alternative approach. Let me know when you'd like a review!

[abdullahkady]

Looking good, thanks!

Would you be able to add integration tests?

@harry-cohere I added a test but it doesn't make a lot of sense, now that we have split the functions (just like the rest of the SDK). The test would need to be something like:

• async test (in terms of the test context itself)
• initialize sync client
• run all the sync client calls (defined in the test_client.py in the test's async context)

[harry-cohere]

Looking good, thanks! Would you be able to add integration tests?

@harry-cohere I added a test but it doesn't make a lot of sense, now that we have split the functions (just like the rest of the SDK). The test would need to be something like:

• async test (in terms of the test context itself)
• initialize sync client
• run all the sync client calls (defined in the test_client.py in the test's async context)

Perhaps just integration tests around the functionality, as it's intended to be used? Up to you though.