Skyvern-AI / skyvern

Automate browser-based workflows with LLMs and Computer Vision
https://www.skyvern.com
GNU Affero General Public License v3.0
10.73k stars 739 forks source link

extend auto completion coverage #1165

Closed LawyZheng closed 2 weeks ago

LawyZheng commented 2 weeks ago

[!IMPORTANT] Extend auto-completion by incorporating user goals and details, adding location input detection, and refining element handling.

  • Behavior:
    • Extend auto-completion prompts in auto-completion-potential-answers.j2 and auto-completion-tweak-value.j2 to include user goal and user details.
    • Add is_location_input attribute to parse-input-or-select-context.j2 and InputOrSelectContext class in actions.py to identify location input fields.
    • Modify handle_input_text_action() in handler.py to use is_location_input for auto-completion.
  • Functions:
    • Add remove_duplicated_HTML_element() in handler.py to clean up HTML elements.
    • Update input_or_auto_complete_input() in handler.py to handle new auto-completion logic.
  • Misc:
    • Add hasASPClientControl() in domUtils.js and page.py to check for ASP client control.
    • Minor logging and error handling improvements in scraper.py and dom.py.

This description was created by Ellipsis for 2b8ad184ca6127320a0d99047709f75fd8b5c927. It will automatically update as commits are pushed.