[Bug]: to_replace for edit_file_by_replace will be changed with CodeActAgent + GPT-4o

[nanjiangwill]

Is there an existing issue for the same bug?

• [X] I have checked the troubleshooting document at https://docs.all-hands.dev/modules/usage/troubleshooting
• [X] I have checked the existing issues.

Describe the bug

When agent calls edit_file_by_replace, they will be provided with to_replace, I noticed that sometimes agent does not receive correct to_replace and it seems it was "formatted" by LLM.

For example, when the actually to_replace is

def tensor_map(fn: Callable[[float], float]) ->Callable[[Storage, Shape,
Strides, Storage, Shape, Strides], None]:
    """
    CUDA higher-order tensor map function. ::

      fn_map = tensor_map(fn)
      fn_map(out, ... )

    Args:
        fn: function mappings floats-to-floats to apply.

    Returns:
        Tensor map function.
    """
    raise NotImplementedError('IMPLEMENT ME HERE')'

the to_replace used in edit_file_by_replace changed to

def tensor_map(fn: Callable[[float], float]) ->Callable[[Storage, Shape, Strides, Storage, Shape, Strides], None]:
    """
    CUDA higher-order tensor map function. ::

      fn_map = tensor_map(fn)
      fn_map(out, ... )

    Args:
        fn: function mappings floats-to-floats to apply.

    Returns:
        Tensor map function.
    """
    raise NotImplementedError('IMPLEMENT ME HERE')'

where you can see the first new line token is removed

When I change the LLM to claude, this bug disappeared.

I am wondering if the process of getting to_replace depends on LLM?

Current OpenHands version

ghcr.io/all-hands-ai/runtime:24de744e15729595fde7d6b73be9c936

Installation and Configuration

[core]
workspace_base="./workspace"

[llm]
model="gpt-4o"
api_key=""
embedding_model="openai"

Model and Agent

Model: GPT-4o Agent: CodeActAgent

Operating System

Mac OS

Reproduction Steps

Run

func_details =  def tensor_map(fn: Callable[[float], float]) ->Callable[[Storage, Shape,
Strides, Storage, Shape, Strides], None]:
    """
    CUDA higher-order tensor map function. ::

      fn_map = tensor_map(fn)
      fn_map(out, ... )

    Args:
        fn: function mappings floats-to-floats to apply.

    Returns:
        Tensor map function.
    """
    raise NotImplementedError('IMPLEMENT ME HERE')'

poetry run python ./openhands/core/main.py \
        -t "implement function that is 'raise NotImplementedError('IMPLEMENT ME HERE')' {func_details}" \
        -c CodeActAgent \
        -l llm

Logs, Errors, Screenshots, and Additional Context

No response

[github-actions[bot]]

This issue is stale because it has been open for 30 days with no activity. Remove stale label or comment or this will be closed in 7 days.

[github-actions[bot]]

This issue was closed because it has been stalled for over 30 days with no activity.