termux / termux-packages

A package build system for Termux.
https://termux.dev
Other
13.4k stars 3.09k forks source link

termux selenium-webdriver #2149

Open delmarocks opened 6 years ago

delmarocks commented 6 years ago

Is there any way to run webdriver in termux when automating browsers using python in termux?

Hax4us commented 6 years ago

I am also curious about this

Serkan-devel commented 6 years ago

Selenium is written in java so it's portable, I think it would work

delmarocks commented 6 years ago

would be great if termux can run a headless browser like phantomjs to be used in python

Serkan-devel commented 6 years ago

Why not hook it to a native browser app? I think that would require adb enabled and phantomjs has been discontinued

Serkan-devel commented 6 years ago

Or one can create a headless chrome package compiled to arm

ambroisehdn commented 5 years ago

so it is possible to use selenium on termux ? if no witch alternatif can help to automatise browser in termux

SwapnilSoni1999 commented 5 years ago

So anyone got anything?

SwapnilSoni1999 commented 4 years ago

package request tag is added So will you work on this? @xeffyr

ghost commented 4 years ago

So will you work on this?

Does someone have submitted a pull request with build script and patches for Chromium or Firefox ? Seems no. Adding Selenium webdriver will be useless in such case.

Tag "help wanted" should clearly describe the status of that issue...

AhmadNaruto commented 4 years ago

is always error 13 . isue is about permision to execute driver chrome or gecko

tathastu871 commented 4 years ago

just install ubuntu or any other distro on termux and therein install firefox or chromium and selenium Succesfully tested. However google chrome does not work on mobile platform as its not open source and there is no arm version of google chrome while chromium and firefox works successfully.

For chromium Specific settings needed 1) --no-sandbox arguments needs to be supplied in selenium when using interactive or python script as root user. 2) binary path to be setted to path of chromium because selenium is made to use google chrome and not chromium. But webdrivers for chromium and google chrome are same. So selenium needs to be instructed to use chromium and not chrome otherwise it will throw exception error that chrome executable not available or not in path.

eg.

For selenium 3 or less: driver = webdriver.Chrome(executable_path ='usr/bin/chromium')

For latest selenium 4, executable path option is deprecated so use object method as below: from selenium.webdriver.chrome.options import Options op = Options() op.binary_location('/usr/bin/chromium') op.add_argument('--no-sandbox') If you are not having a desktop environment or x windows system use headless op.add_argument('--headless') Finally driver = webdriver.Chrome(options=op)

kcubeterm commented 4 years ago

@tathastu871 yea, that's work, but we want same in termux itself

tathastu871 commented 4 years ago

For using it in termux itself following succesfully tested solution. Connect phone to pc with usb debugging enable adb start-server chromedriver Note down port number from output In python on termux from selenium import webdriver

capabilities = { 'chromeOptions': { 'androidPackage': 'com.android.chrome', } } driver = webdriver.Remote('http://localhost:9515', capabilities) # Specify your port number value driver.get('http://google.com') driver.quit()

tathastu871 commented 4 years ago

Using a distro on termux will allow for portability for selenium automation however for using it on termux only we need to use adb. In case tried adb over wifi works and does the trick thereby bypassing use of usb

tathastu871 commented 4 years ago

Final solution is to use remote capabilities where phone to be connected over wan using port forwarding and on pc which is kept running all day long with browser installed and whenever we are away from house or pc and want to use selenium on termux use remote capabilities using port and ip address of device.

Port forwarding required only in case of wan when our pc and phone are on different network.

While on lan and on same wifi no port forwarding require.

tathastu871 commented 4 years ago

adb, chromedriver or geckodriver, selenium are only needed for automating web on android browsers. Among them adb and selenium are both available as termux binaries without need of computer. The only workaround needed is to get chromedriver recoded to run on termux.

Just as adb and fastboot binaries have been remade for termux if chromedriver is also rewritten and made to work in termux everything will be good.

Then there will be no need to use computer or prooted linux distros, just a termux is sufficient

xloem commented 3 years ago

This is the termux-packages repository. It's a little helpful to remember this works via proot or tethering, but the issue is around introducing termux packages to provide for systems that aren't tethered and don't have the space for a prooted distro.

bitcoinmeetups commented 3 years ago

I also would like to be able to use Selenium with Termux or a similar solution

EnricoBDev commented 2 years ago

@tathastu871 tried doing what you suggested but I always have a chrome driver not in path error, how do I fix this? Thanks

tathastu871 commented 2 years ago

@tathastu871 tried doing what you suggested but I always have a chrome driver not in path error, how do I fix this? Thanks

driver = webdriver.Chrome('/path/to/chromedriver')

Thisismo7a commented 2 years ago

Need help I can't install 'selenium' on termux any solution ????🥺🥺🥺

tathastu871 commented 2 years ago

Need help I can't install 'selenium' on termux any solution ????🥺🥺🥺

Use andronix app to install ubuntu or any other distro and install chromium or firefox with selenium using apt.

However latest version of firefox may have some problems with selenium. if encountered try downgrading firefox and firefox-geckodriver packages to 75.0+build3-0ubuntu1

incase of chromium extra steps of specifying binary location is required

luanon404 commented 2 years ago

Need help I can't install 'selenium' on termux any solution ????🥺🥺🥺

I have written a package like selenium but only need termux to use, no pc require, you can do many simple tasks like login, submit form, .... here is link if you need (i dont know if i can use selenium on android without pc so i made this package)

tathastu871 commented 2 years ago

Also theres a package called selendroid. Wonder anyone figured how to use it its a jar binary to control android apps https://github.com/selendroid/selendroid/releases/download/0.17.0/selendroid-standalone-0.17.0-with-dependencies.jar

bitcoinmeetups commented 2 years ago

Also theres a package called selendroid. Wonder anyone figured how to use it its a jar binary to control android apps https://github.com/selendroid/selendroid/releases/download/0.17.0/selendroid-standalone-0.17.0-with-dependencies.jar

Selendroid. No updates. 4 years.

luanon404 commented 2 years ago

Also if adb enabled you can use uiautomator on termux to automate any app including browsers Just enable adb in tcpip mode once using pc.Then no use of pc. Only termux would be sufficient to automate every action on android

Also.on android 11 you dont need pc to enable tcpip adb. Just use wifi setting to do it

So what is the choice for old Android devices and no pc? I've been searching for two years but still can't find an easy way, most need pc or root.

AndreAlmeida09 commented 2 years ago

just install ubuntu or any other distro on termux and therein install firefox or chromium and selenium Succesfully tested. However google chrome does not work on mobile platform as its not open source and there is no arm version of google chrome while chromium and firefox works successfully.

For chromium Specific settings needed

  1. --no-sandbox arguments needs to be supplied in selenium when using interactive or python script as root user.
  2. binary path to be setted to path of chromium because selenium is made to use google chrome and not chromium. But webdrivers for chromium and google chrome are same. So selenium needs to be instructed to use chromium and not chrome otherwise it will throw exception error that chrome executable not available or not in path.

eg.

For selenium 3 or less: driver = webdriver.Chrome(executable_path ='usr/bin/chromium')

For latest selenium 4, executable path option is deprecated so use object method as below: from selenium.webdriver.chrome.options import Options op = Options() op.binary_location('/usr/bin/chromium') op.add_argument('--no-sandbox') If you are not having a desktop environment or x windows system use headless op.add_argument('--headless') Finally driver = webdriver.Chrome(options=op)

I can't get this to work.. I used Andronix to install Ubunut 20.04 , Desktop Environment LXDE.

Any tips? I keep getting chrome driver not in path error....

tathastu871 commented 2 years ago

Now Geckodriver is available for termux. Method1) wget https://github.com/mozilla/geckodriver/releases/download/v0.32.0/geckodriver-v0.32.0-linux-aarch64.tar.gz Method2) cargo install geckodriver

Try enabling: adb start-server --> adb tcpip 5555 then on Termux --> run adb connect localhost:5555 geckodriver it will show port --> Use selenium remote capabilities to connect to localhost:port number and we can control firefox app using selenium directly from termux

Can anyone try it i cant enable adb on my device

luanon404 commented 2 years ago

Now Geckodriver is available for termux. Method1) wget https://github.com/mozilla/geckodriver/releases/download/v0.32.0/geckodriver-v0.32.0-linux-aarch64.tar.gz Method2) cargo install geckodriver

Try enabling: adb start-server --> adb tcpip 5555 then on Termux --> run adb connect localhost:5555 geckodriver it will show port --> Use selenium remote capabilities to connect to localhost:port number and we can control firefox app using selenium directly from termux

Can anyone try it i cant enable adb on my device

solution for android 11 above only

tathastu871 commented 2 years ago

Now Geckodriver is available for termux. Method1) wget https://github.com/mozilla/geckodriver/releases/download/v0.32.0/geckodriver-v0.32.0-linux-aarch64.tar.gz Method2) cargo install geckodriver Try enabling: adb start-server --> adb tcpip 5555 then on Termux --> run adb connect localhost:5555 geckodriver it will show port --> Use selenium remote capabilities to connect to localhost:port number and we can control firefox app using selenium directly from termux Can anyone try it i cant enable adb on my device

solution for android 11 above only

no works for all.

Once enable adb tcpip 5555 from pc. Then remove usb

and Now on termux adb connect localhost:5555 geckodriver selenium remote capabilities

My usb us brokenso icant adb but here is selenium output

raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.WebDriverException: Message: No Android devices are online

Means it was working and scanning for localhostjust that my adb is not enabled it says no device

SunPodder commented 1 year ago

We have Firefox, chromium + chrome-driver in TUR. Firefox with geckodriver from the official repo works in termux.

SunPodder commented 1 year ago

@xtkoba completed

Nothig00 commented 1 year ago

@xtkoba completed

Can i get littel guide for how to use TUR with selenium

SunPodder commented 1 year ago
apt install tur-repo
apt install firefox
# you can also use chromium instead
# firefox is light weight

This will install tur-repo and firefox.

Download gecko driver Run:

tar -xf geckodriver-v0.32.2-linux-aarch64.tar.gz
mv geckodriver $PREFIX/bin

You may now use selenium as usual. Here is a python code

from selenium import webdriver

driver = webdriver.Firefox()

# This will require a vnc connection
# you may run this in headless mode
# you will have to pass some arguments to the driver
driver.get("https://google.com")

Edit: Geckodriver can be installed directly from the tur-repo.

apt install geckodriver
Nothig00 commented 1 year ago

@SunPodder do I need proot-distro ?

SunPodder commented 1 year ago

@SunPodder do I need proot-distro ?

No

iamstarcode commented 1 year ago

Hi @SunPodder please both the firefox and the chromium cannot be found even after "apt update and apt upgrade"

sylirre commented 1 year ago

chromium is part of TUR: https://github.com/termux-user-repository/tur

pkg install tur-repo
pkg update
pkg install chromium

firefox is part of X11 packages set:

pkg install x11-repo
pkg update
pkg install firefox

Pay attention to the order of pkg install commands. It matters because *-repo packages change the sources.list configuration of package manager.

iamstarcode commented 1 year ago

Oh! Thanks i didn't add x11-repo earlier, after that i could see both chromium and firefox, please just wondering if you have a repo where you have succesfull scrapped a website with selelium using Node.js in termux, I will so much appreciate it thanks. And this wouldn't require PRoot i suppose.

tathastu871 commented 1 year ago

always use headless mode to prevnt excess memory use and crash

SunPodder commented 1 year ago

Oh! Thanks i didn't add x11-repo earlier, after that i could see both chromium and firefox, please just wondering if you have a repo where you have succesfull scrapped a website with selelium using Node.js in termux, I will so much appreciate it thanks. And this wouldn't require PRoot i suppose.

Any code that works on linux should work on termux. Just make sure that code uses headless mode if you don't have a xserver.

iamstarcode commented 1 year ago

Thank you very much, 😊

Akianonymus commented 1 year ago

Even if one doesn't want to use headless mode, then install xorg-server-xvfb and use the fake display, will work without headless.

luanon404 commented 1 year ago

for who dont know how to install, i made this tutorial https://github.com/luanon404/Selenium-On-Termux-Android

xAffan commented 1 year ago

Selenium-Manager is not compiled for arm64 so you are unable to select chromedriver.

dr1408 commented 10 months ago

how can i use termux tool that use selenium and python and require web browser to make it work on my android firefox app without the need to vnc what codes i need to edit in the tool

Shpantze commented 9 months ago

for who dont know how to install, i made this tutorial https://github.com/luanon404/Selenium-On-Termux-Android

I'm getting... PID 19150 does not have permission android.permission.CLEAR_APP_USER_DATA to clear data of package org.chromium.chrome.stable after trying to set driver = *(for brevity) in python.

update: So I figured out you have to disable permission monitoring but now the version of chrome driver installed with tur or chromium (wherever it comes from...idk) says it's only compatible with Chrome version 121 ...I only see chromium versions up to 120 in your provided link and I'm not sure really how to set the driver otherwise

2nd update: Looks like simply changing the options to point at the chrome package did the trick. .... 🤦🏻‍♂️uggghh cleared my data though... very convenient lol

nikhilgrajput commented 3 months ago

Hi @luanon404 I am able to open the chromium from termux (Andrpoid 10). I am facing mainly four issues.

  1. In every run it's open as new instance, I am want to use the old session. I tried to use options.add_argument('--profile-directory=Default') and options.add_argument("--user-data-dir=/data/data/org.chromium.chrome.stable/app_chrome") but no benefit. Below is the driver code

       @staticmethod
       def getTermuxDriver(logger):
              try:
                   ser = Service("/data/data/com.termux/files/usr/lib/chromium/chromedriver")
                   options = webdriver.ChromeOptions()
                   options.add_argument("--no-sandbox")
                   options.add_argument("--disable-dev-shm-usage")           
                   options.add_argument('--profile-directory=Default')
                   options.add_experimental_option("androidPackage", "org.chromium.chrome.stable")
                   options.add_argument("--display=:1")            
                   driver = webdriver.Chrome(service=ser, options=options)
                   return driver
              except Exception as e:
                   logger.exception("getTermuxDriver")
                   return None
  2. paperclip() is not working, I mean not able to use copy paste as my text have have seme special character in it, so instead of using send_keys, I want to user paperclip.copy(text) and then send_keys(Keys.CONTROL + 'v') to paste the text into desired element.
  3. Not able to open Firefox, below is code for firefox

        from selenium.webdriver.firefox.service import Service as FireService
        from selenium.webdriver.firefox.options import Options as FireOptions
    
        @staticmethod
        def getTermuxFirfoxDriver(logger):
               ser = FireService('/data/data/com.termux/files/usr/bin/geckodriver')
               options = FireOptions()
               options.add_argument("--no-sandbox")
               options.add_argument("--disable-dev-shm-usage")
               options.add_argument('--profile-directory=Default')
               driver = webdriver.Firefox(service=ser, options=options)
               return driver

    error : No browser open in my android device (In case of chromium it opens the chromium installed in my device) , but its seems like it is able to open browser some here in background, since I am able to see selenium webelement error which comes when you click a element which is obscured by some other element.

Below are some details of my environment Selenium : selenium-4.9.1 Chromium : arm64_ChromePublic_HEVC-125.0.6422.164.apk Firefox : 129.0 gecko path in termux : /data/data/com.termux/files/usr/bin/geckodriver firefox path in termux : /data/data/com.termux/files/usr/lib/firefox

  1. Not able to install uiautomator2 inside termux, pip install uiautomator2 hang when it tries to install lxml dependency.

Request you you if you can help me out here.

luanon404 commented 3 months ago

Hi @luanon404 I am able to open the chromium from termux (Andrpoid 10). I am facing mainly four issues.

  1. In every run it's open as new instance, I am want to use the old session. I tried to use options.add_argument('--profile-directory=Default') and options.add_argument("--user-data-dir=/data/data/org.chromium.chrome.stable/app_chrome") but no benefit. Below is the driver code

       @staticmethod
       def getTermuxDriver(logger):
              try:
                   ser = Service("/data/data/com.termux/files/usr/lib/chromium/chromedriver")
                   options = webdriver.ChromeOptions()
                   options.add_argument("--no-sandbox")
                   options.add_argument("--disable-dev-shm-usage")           
                   options.add_argument('--profile-directory=Default')
                   options.add_experimental_option("androidPackage", "org.chromium.chrome.stable")
                   options.add_argument("--display=:1")            
                   driver = webdriver.Chrome(service=ser, options=options)
                   return driver
              except Exception as e:
                   logger.exception("getTermuxDriver")
                   return None
  2. paperclip() is not working, I mean not able to use copy paste as my text have have seme special character in it, so instead of using send_keys, I want to user paperclip.copy(text) and then send_keys(Keys.CONTROL + 'v') to paste the text into desired element.
  3. Not able to open Firefox, below is code for firefox

        from selenium.webdriver.firefox.service import Service as FireService
        from selenium.webdriver.firefox.options import Options as FireOptions
    
        @staticmethod
        def getTermuxFirfoxDriver(logger):
               ser = FireService('/data/data/com.termux/files/usr/bin/geckodriver')
               options = FireOptions()
               options.add_argument("--no-sandbox")
               options.add_argument("--disable-dev-shm-usage")
               options.add_argument('--profile-directory=Default')
               driver = webdriver.Firefox(service=ser, options=options)
               return driver

    error : No browser open in my android device (In case of chromium it opens the chromium installed in my device) , but its seems like it is able to open browser some here in background, since I am able to see selenium webelement error which comes when you click a element which is obscured by some other element.

Below are some details of my environment Selenium : selenium-4.9.1 Chromium : arm64_ChromePublic_HEVC-125.0.6422.164.apk Firefox : 129.0 gecko path in termux : /data/data/com.termux/files/usr/bin/geckodriver firefox path in termux : /data/data/com.termux/files/usr/lib/firefox

  1. Not able to install uiautomator2 inside termux, pip install uiautomator2 hang when it tries to install lxml dependency.

Request you you if you can help me out here.

  1. Android dont have profile like that, so you cant use
  2. Did you allow the browser to copy and paste?
  3. Thats now how my example come, u need install firefox app before
  4. u can disable it, by pass --no-dependency maybe? i dont remember it, but u can, for lxml, you need install some lib by apt or pkg, you can ask gpt for this.