search

langchain-ai / langchain-nvidia

MIT License
48 stars 15 forks source link

Scheduled test cases fix #71

Closed raspawar closed 1 month ago

raspawar commented 1 month ago

Fix for the scheduled integration test_chat_model test case failure

Screenshot 2024-07-12 at 7 42 32 PM

Reason for failure: mistralai model does not allow AIMessage followed by SystemMessage

Successful Workflow

mattf commented 1 month ago

@raspawar ok, if this is the issue then the server side implementation must have changed. our options are to 0. change the default model to one that passes all the tests, 1. mark this test as xfail.

go w/ (1) for now and investigate (0).

raspawar commented 1 month ago

@mattf My guess is ==> It is not server side implementation This test cases were added in recent PR: https://github.com/langchain-ai/langchain-nvidia/pull/63 And Integration test cases run only in Scheduled tests github workflow action, so after it got merged, scheduled test cases started failing.

mattf commented 1 month ago

My guess is ==> It is not server side implementation This test cases were added in recent PR: #63 And Integration test cases run only in Scheduled tests github workflow action, so after it got merged, scheduled test cases started failing.

63 added test_stop while the failures are in test_messages. the test_stop tests are already marked xfail

change test_messages to also be xfail for now.

raspawar commented 1 month ago

ok, I will keep looking, meanwhile I have marked test_messages as xfail, so can we close this as the scheduled test cases will keep failing