Setting up a test battery

[uazo]

I decided to start making this browser into something more serious. To date, there is no test in cromite that indicates what bug and regressions there are in each release, and more importantly, what the differences are compared to chromium, even from the point of view of security if not privacy. That is why I have always thought that automatic tests should be introduced.

What I need is an automatic test platform.

this is the idea:

• find a way to drive cromite from the outside
• reactivate the build of chromium tests, to date broken by my patches
• find the official list of chromium/blink tests and check how cromite performs
• define a list of chromium tests to be maintained (e.g. sandbox tests)
• building specific tests for cromite

this is a project within a project, so it is hard to imagine that it will not last for years.

[uazo]

reserved

[uazo]

let's start with how to drive cromite from the outside. first of all I start by saying that it is more complex than it looks (and excuse me if I write trivial things).

the choice is to use Appium + UiAutomator2 + chromedriver, driven by webdriver in nodejs, but I started out knowing nothing about all!

first of all, I discovered that there are 2 versions of appium, the one officially documented in browserstack (but old, the v1) and one undocumented (v2) which, without knowing it, I used. In fact, I found differences that I could not explain, but which I only realised later, since most of the things I tried still worked. this made it difficult for me to understand, also because the driver of the old version was called wd while that of the new webdriver and at first I thought it was just the initials :) so I developed a solution with wd only realising later that I was doing it wrong :(

Only after banging my head several times do I realise that:

• Appium is the box that enables communication between the client (webdriver) and the application through a driver
• WebdriverIO is the client library to use
• UiAutomator2 is the driver, to be installed in appium, to be used by the webdriver to communicate with an app in android as far as the native layer is concerned.
• chromedriver is another driver, the server for talking to chrome, activated by appium and proxied by UiAutomator2, which allows direct communication with the chromium developer tools

everything communicates via http, between the client and server part of the component, with the sole exception of UiAutomator2 which drives android via adb. chromedriver also talks to chromium via a http socket tunnelled by adb.

the various layers allow me different things

• UiAutomator2, together with the appium inspector, allows me to drive the elements of the application's native interface. I can also run adb commands, e.g. to grant permissions with pm, but it does not allow me to launch javascript code inside the browser. docs: https://github.com/appium/appium-uiautomator2-driver
• chromedriver allows me to work with the, shall we say, specific part of chromium, interacting with the dom, the page and, above all, the developer tools.

UiAutomator2 works flawlessly, chromedriver needs some hack, also because it is not officially supported by browserstack (and moreover not updated, being still at v123) and I need my solution to work on the emulator as on a real device. hacks cannot stay in chromedriver (that it would have been easier to), because it is not replaceable in browserstack, but directly in cromite.

the problem is that to get chromedriver to work with chromium forks (all of them, the problem is not just cromite) you have to cheat some of its basic settings. an internet search did not directly bring a solution, because it seems that no one has ever succeeded (or maybe I haven't found it). the problem is that appium + chromedriver are not made to talk with a fork but only with webviews or the content shell. an easy problem to change, if you could insert a few lines of code here and there but fortunately you just need to use in the capabilities this options, and it works:

         'appium:chromeOptions': {
             'androidDeviceSocket': 'chrome_devtools_remote',
             'androidExecName': 'unusedbutimportant',
         },

except in browserstack! that does not accept the passing of that information. i have asked their support, i have not currently received any technical response (if not look in the documentation :) so you need to modify chromium and enable the socket used by chromedriver as if it were a webview rather than a browser. fortunately it is very simple, I will put the patch.

done that I haven't done anything yet :) but it's a first step and I got here.

[uazo]

all potentially true, evidence is lacking. I would like to find a way to prove otherwise, even just to myself:

Cromite makes changes which significantly reduce security, and most of their privacy changes are highly questionable.

They do not implement many of the features correctly and take problematic shortcuts. It leads to them having a long list of features which sound useful but which are largely reducing security and not working properly.

They incorporate highly problematic third party code from Eyeo full of serious security bugs and full of invasive tracking code

from https://discuss.grapheneos.org/d/10550-mull-hardened-firefox-security/60