realitykings scraper cant fine chrome

mayonnaise3141 commented 3 years ago

when scraping a realitykings scene url (https://www.realitykings.com/scene/2940891/makeup-titorial), i get the following error: exec google-chrome: executable not found in $PATH

stash is running in a docker container (with sudo) which presumably does not have access to my user path

google-chrome is in my user path

$ which google-chrome
/usr/bin/google-chrome

how can i make stashapp (running in a docker container) see the path for $USER?

JaseNZC commented 3 years ago

That scraper uses CDP so you need to make sure that section is completed in stash settings it's under scrapers

You would have to map a path from your host to your container if you wanted to go the path route or install a headless version of chrome in a container in docker. Easy way would be to run this command in terminal to create the instance of the required docker container, then complete the settings in cdp location as mentioned above. docker run -d -p 9222:9222 --rm --name headless-shell chromedp/headless-shell

mayonnaise3141 commented 3 years ago

thanks, so is just wondering how to get the version of the headless shell running at localhost:9222? I think the chome cdp path requires a version, but the output of docker run did not seem to provide it:

Unable to find image 'chromedp/headless-shell:latest' locally
latest: Pulling from chromedp/headless-shell
b380bbd43752: Pull complete
a5737bd1fafe: Pull complete
e166cadac687: Pull complete
741419281822: Pull complete
Digest: sha256:7f52a4a033d515de1769e5d9cc014a3181cc9041d4a0e2b515f2366705211a58
Status: Downloaded newer image for chromedp/headless-shell:latest
91070ab84743d23bdb41e7c1c4eba8f2bca84bd8b9f4b385bccf21e8402e8b5f

JaseNZC commented 3 years ago

thanks, so is just wondering how to get the version of the headless shell running at localhost:9222? I think the chome cdp path requires a version, but the output of docker run did not seem to provide it:
Unable to find image 'chromedp/headless-shell:latest' locally
latest: Pulling from chromedp/headless-shell
b380bbd43752: Pull complete
a5737bd1fafe: Pull complete
e166cadac687: Pull complete
741419281822: Pull complete
Digest: sha256:7f52a4a033d515de1769e5d9cc014a3181cc9041d4a0e2b515f2366705211a58
Status: Downloaded newer image for chromedp/headless-shell:latest
91070ab84743d23bdb41e7c1c4eba8f2bca84bd8b9f4b385bccf21e8402e8b5f

https://developers.google.com/web/updates/2017/04/headless-chrome

bnkai commented 3 years ago

thanks, so is just wondering how to get the version of the headless shell running at localhost:9222? I think the chome cdp path requires a version, but the output of docker run did not seem to provide it:

The cdp remote path doesnt require a version just the ip. The version is just a needed string eg in your case i would run ip addr from the host machine that runs the docker containers and get the ip addr eg 192.168.1.2 (or 10.0.0.2) and then use http://192.168.1.2:9222/json/version as the cdp path, provided ofc that the chromedp/headless-shell container is running. The /json/version string at the end stays as it is.

For a more advanced usage i would have a look here https://github.com/stashapp/stash/issues/1023 The docker compose file from that issue for example

version: '3.4'
networks:
  stash-net:
    name: stash-net
services:
  stash:
    image: stashapp/stash:latest
    container_name: stash
    restart: unless-stopped
    ports:
      - "9999:9999"
    networks:
      - stash-net
    logging:
      driver: "json-file"
      options:
        max-file: "10"
        max-size: "2m"
    environment:
      - STASH_STASH=/data/
      - STASH_GENERATED=/generated/
      - STASH_METADATA=/metadata/
      - STASH_CACHE=/cache/
    volumes:
      - /etc/localtime:/etc/localtime:ro
      - ./.stash:/root/.stash
      - ./:/data
      - ./.stash/metadata:/metadata
      - ./.stash/cache:/cache
      - ./.stash/generated:/generated
  headless-shell:
    image: chromedp/headless-shell:latest
    container_name: headless-shell
    restart: unless-stopped
    network_mode: "service:stash"

combines stash and headless-shell as services and as stated works even if you use localhost as the ip (you might need to adjust the image and volumes part)

bnkai commented 2 years ago

Closing as the issue seems resolved

stashapp / CommunityScrapers

realitykings scraper cant fine chrome #758