LangSmith config Weird checker leads to error (0.8.0 problem)

dchang10 commented 1 month ago

Self Checks

[X] This is only for bug report, if you would like to ask a question, please head to Discussions.
[X] I have searched for existing issues search for existing issues, including closed ones.
[X] I confirm that I am using English to submit this report (我已阅读并同意 Language Policy).
[X] [FOR CHINESE USERS] 请务必使用英文提交 Issue，否则会被关闭。谢谢！:）
[X] Please do not modify this template :) and fill in all the required fields.

Dify version

0.8.0

Cloud or Self Hosted

Cloud

Steps to reproduce

I tried adding a langsmith config to a project, but the newly added get_project_url() function keeps returning "LangSmith get run url failed: xyz" error.

✔️ Expected Behavior

It is supposed to be an easy link to langsmith.

❌ Actual Behavior

LangSmith get run url failed: xyz

dosubot[bot] commented 1 month ago

The "LangSmith get run url failed: xyz" error in the get_project_url() function occurs because an exception is raised when attempting to get the run URL. This exception is caught and logged, and then a ValueError is raised with the message "LangSmith get run url failed: {str(e)}".

Here is the relevant code snippet:

def get_project_url(self):
    try:
        run_data = RunBase(
            id=uuid.uuid4(),
            name="tool",
            inputs={"input": "test"},
            outputs={"output": "test"},
            run_type=LangSmithRunType.tool,
            start_time=datetime.now(),
        )
        project_url = self.langsmith_client.get_run_url(
            run=run_data, project_id=self.project_id, project_name=self.project_name
        )
        return project_url.split("/r/")[0]
    except Exception as e:
        logger.debug(f"LangSmith get run url failed: {str(e)}")
        raise ValueError(f"LangSmith get run url failed: {str(e)}")

The error occurs because the get_run_url method call within the try block fails, leading to the exception being caught and the error message being raised ^[1].

Possible reasons for the get_run_url method to fail include:

Invalid run_data: If the run_data object is not properly constructed or contains invalid data.
Missing project_id or project_name: The method requires either project_id or project_name to generate the URL.
API Client Issues: Issues such as incorrect API key, endpoint, or other configuration problems.
Exception Handling: Any other unexpected exceptions during the method execution ^[1].

Ensure that the project_id or project_name is correctly set in the LangSmithClient configuration to avoid this issue ^[1].

To continue talking to Dosu, mention @dosu.

dchang10 commented 1 month ago

What dosu commented does not help with this bug. Yes, it is an error thrown by the get_run_url() function. However, it is not clear why that check is even needed, and why the behavior of this simple langsmith config setup deviates from previous dify versions.

dosubot[bot] commented 1 month ago

The get_run_url() function check is needed to ensure that the LangSmith client can successfully generate a URL for a run, which is crucial for tracking and accessing run details. This check helps verify that the client is correctly configured and that the necessary parameters (like project_id and project_name) are valid and functional ^[1].

The behavior of the LangSmith config setup in version 0.8.0 deviates from previous Dify versions because it introduces a more structured and detailed tracing mechanism. This includes handling various types of trace information (e.g., WorkflowTraceInfo, MessageTraceInfo, ModerationTraceInfo, etc.) and creating corresponding LangSmithRunModel instances for each type. The new setup also includes additional metadata handling, error logging, and specific run types (e.g., LangSmithRunType.chain, LangSmithRunType.tool, LangSmithRunType.llm, etc.), which were not as explicitly managed in previous versions. This structured approach allows for more granular and comprehensive tracing of different operations within the Dify platform ^[1].

To continue talking to Dosu, mention @dosu.

charli117 commented 1 month ago

@dchang10 The test did not reproduce your problem, make sure that the https://smith.langchain.com network is reachable

dchang10 commented 1 month ago

@charli117 When you are setting config for langsmith, do you have to match the project name with something? Or could I name it anything? https://smith.langchain.com is reachable.

charli117 commented 1 month ago

The project name must be the same as the langsmith project name, otherwise the project address will not be obtained

dchang10 commented 1 month ago

@charli117 Langsmith project creation is done when the first log is sent in. The only "name" that is specified is the "Description" you enter on Langsmith when an API key is created.

charli117 commented 1 month ago

Project value correspond to the following fields

dchang10 commented 1 month ago

@charli117 This is when you connect it to an existing project. However, when a user wants to link a new dify application to a new langsmith project, this is when you would see the error.

crazywoola commented 1 month ago

Please try to upgrade to v0.8.3 to see if this persists.

langgenius / dify