Lookyloo / PlaywrightCapture

Capture a URL with Playwright
Other
30 stars 3 forks source link

Favicon fetching #45

Closed Rafiot closed 6 months ago

Rafiot commented 1 year ago

Playwright blacklists requests to fetch the favicons (see https://github.com/microsoft/playwright/issues/7493), so we need an alternative to that.

For that to work, we need to do the following:

PlaywrightCapture

  1. Add a method to get all the possible favicons from the rendered page (either embedded, or fetch them with requests) - See AIL implementation for reference - the response of this method is a dictionary: {"URL_or_reference": b"favicon as a bytestream"}
  2. Add an optional parameter to the capture_page method - something like with_favicons (False by default)
  3. Call the new method after the call to get the screenshot https://github.com/Lookyloo/PlaywrightCapture/blob/7e6242890a7cb40a5e097ab3f54e2992e59abf0b/playwrightcapture/capture.py#L518
  4. Update CaptureResponse accordingly https://github.com/Lookyloo/PlaywrightCapture/blob/7e6242890a7cb40a5e097ab3f54e2992e59abf0b/playwrightcapture/capture.py#L48

LacusCore

  1. Update the encode method to encode the favicons: https://github.com/ail-project/LacusCore/blob/86b4b51dcdc297b0dbb3e69eae05ab61766c6a49/lacuscore/lacuscore.py#L339
  2. Update the TypedDict for the JSON-compatible dump (CaptureResponseJson): https://github.com/ail-project/LacusCore/blob/86b4b51dcdc297b0dbb3e69eae05ab61766c6a49/lacuscore/lacuscore.py#L84
  3. Update CaptureSettings and enqueue method to support the new with_favicon keyword: https://github.com/ail-project/LacusCore/blob/86b4b51dcdc297b0dbb3e69eae05ab61766c6a49/lacuscore/lacuscore.py#L100
  4. Update _capture to pass with_favicon to PlaywrightCapture.capture_page

PyLacus

  1. Update CaptureResponse to map with CaptureResponseJson from LacusCore
  2. Update CaptureSettings to be the same as in LacusCore
  3. Update enqueue to be the same as in LacusCore

Lookyloo

  1. Add with_favicon as a parameter when enqueuing a capture in the API and in the UI
  2. Pass it to LacusCore or PyLacus
  3. Display the favicon(s) somewhere in the tree view
Rafiot commented 6 months ago

Done.