search

yeagerai / genlayer-simulator

MIT License
14 stars 8 forks source link

SIM-BE-Improve webrequest module #357

Open AgustinRamiroDiaz opened 2 months ago

AgustinRamiroDiaz commented 2 months ago

We should use a different browser and image to ensure it works on most platforms. More info: https://github.com/ultrafunkamsterdam/undetected-chromedriver

AgustinRamiroDiaz commented 2 months ago

@cristiam86 could you add more context on the problems we are trying to solve here?

cristiam86 commented 2 months ago

sure @AgustinRamiroDiaz! The main problems that we currently have are:

  1. The docker image being used is too heavy
  2. Some platforms (some Windows versions) fail to build and execute because of all the modules and dependencies needed.
  3. The browser (chrome) used needs to be simplified to what we are doing. Bulat suggested the undetected chrome, which seems lighter and better.
YeagerAI-Bulat commented 2 months ago
  1. Maybe https://seleniumbase.io/help_docs/uc_mode/#uc-mode would be a better option
AgustinRamiroDiaz commented 1 month ago

Extra: After this, we wouldn't require to target a particular architecture since chromium is available in other architectures (now we depend on Chrome) FROM --platform=linux/amd64 python:3.12.5-slim