chore(langchain): disable flaky tests

[Kyle-Verhoog]

There appear to be stability issues with using snapshots and/or LangChain in general.

There are failures in the mocked tests that look like:

builtins.AssertionError: assert 0 == 1
 +  where 0 = <MagicMock name='LLMObsSpanWriter().enqueue' id='127482146130048'>.call_count
 +    where <MagicMock name='LLMObsSpanWriter().enqueue' id='127482146130048'> = <MagicMock name='LLMObsSpanWriter()' id='127482147073440'>.enqueue

as well as failures with snapshot based tests:

builtins.Failed: At request <Request GET /test/session/snapshot >:
   At snapshot (token='tests.contrib.langchain.test_langchain_community.test_lcel_chain_simple'):
    - Directory: /go/src/github.com/DataDog/apm-reliability/dd-trace-py/tests/snapshots
    - CI mode: 1
    - Trace File: /go/src/github.com/DataDog/apm-reliability/dd-trace-py/tests/snapshots/tests.contrib.langchain.test_langchain_community.test_lcel_chain_simple.json
    - Stats File: /go/src/github.com/DataDog/apm-reliability/dd-trace-py/tests/snapshots/tests.contrib.langchain.test_langchain_community.test_lcel_chain_simple_tracestats.json
    At compare of 1 expected trace(s) to 0 received trace(s):
Did not receive expected traces: 'langchain.request'

While we investigate a more stable method of testing it makes sense to disable the tests to avoid noise to our neighbours in the library :).

DOWN WITH FLAKY TESTS

Checklist

• [ ] PR author has checked that all the criteria below are met
• The PR description includes an overview of the change
• The PR description articulates the motivation for the change
• The change includes tests OR the PR description describes a testing strategy
• The PR description notes risks associated with the change, if any
• Newly-added code is easy to change
• The change follows the library release note guidelines
• The change includes or references documentation updates if necessary
• Backport labels are set (if applicable)

Reviewer Checklist

• [ ] Reviewer has checked that all the criteria below are met
• Title is accurate
• All changes are related to the pull request's stated goal
• Avoids breaking API changes
• Testing strategy adequately addresses listed risks
• Newly-added code is easy to change
• Release note makes sense to a user of the library
• If necessary, author has acknowledged and discussed the performance implications of this PR as reported in the benchmarks PR comment
• Backport labels are set in a manner that is consistent with the release branch maintenance policy

[github-actions[bot]]

CODEOWNERS have been resolved as:

tests/contrib/langchain/test_langchain_community.py                     @DataDog/ml-observability
tests/contrib/langchain/test_langchain_llmobs.py                        @DataDog/ml-observability

[pr-commenter[bot]]

Benchmarks

Benchmark execution time: 2024-11-22 19:41:32

Comparing candidate commit 971dad7b34d58e3d2d55ee3fb17e5891693a4630 in PR branch kylev/flaky-tests-should-be-shot-into-the-sun with baseline commit d792c3dc3c7452ed64524ea38d0b9c9116330a73 in branch main.

Found 0 performance improvements and 0 performance regressions! Performance is the same for 388 metrics, 2 unstable metrics.