michallepicki
commented
5 years ago Tests are being executed in parallel/async
How many tests are being run in parallel? (what's the --max-cases/:max_cases ExUnit option?)
Open PragTob opened 5 years ago
michallepicki
commented
5 years ago Tests are being executed in parallel/async
How many tests are being run in parallel? (what's the --max-cases/:max_cases ExUnit option?)
PragTob
commented
5 years ago @michallepicki it's not set explicitly and I don't know how many cores the CI has off the top of my head and I don't have SSH access
edit:
lscpu says 4 CPUs so probably max-cases is set to 8. We thought about reducing it as we think it might be an issue with overloading chromedriver but not quite sure
PragTob
commented
5 years ago After changing tests to run with --max-cases 2 on CI we haven't seen the failures since. Will continue to observe. So, it might be that it's some sort of problem with overloading or state interfering. Question is just whether it's in wallaby or chromedriver or what not.
For whoever comes afterwards, a team member also had an issue where this occurred all the time but there the issue was non matching chromedriver/chromium versions. That's decidedly not the case on CI as linux package management enforces matching versions there.
WHile our problem seems solved I think I'd like to keep this open because it might still be a problem that's fixable or at least better highlightable in wallaby.
PragTob
commented
4 years ago For the record, the problem popped up again restricting it to --max-cases 2 fixed it again :man_shrugging:
I'm unsure what this is. Part of me believes chromedriver just dies.
Which is supported by the error I get sometimes being:
** (MatchError) no match of right hand side value: {:error, "unknown error: Chrome failed to start: exited abnormally\n (chrome not reachable)\n (The process started from chrome location /usr/lib/chromium/chrome is no longer running, so ChromeDriver is assuming that Chrome has crashed.)\n (Driver info: chromedriver=72.0.3626.121,platform=Linux 5.0.0-29-generic x86_64)"}
app_1 | stacktrace:
app_1 | (pricing_engine_web) test/support/feature_case.ex:35: PricingEngineWeb.FeatureCase.__ex_unit_setup_0/1
app_1 | (pricing_engine_web) test/support/feature_case.ex:1: PricingEngineWeb.FeatureCase.__ex_unit__/2
app_1 | test/pricing_engine_web/features/pricing_grid_test.exs:1: PricingEngineWeb.PricingGridTest.__ex_unit__/2
PJUllrich
commented
4 years ago Small note for future reference: Make sure that your chromedriver version is the same on your local machine and on your CI. Pin the version on your CI if necessary. Every now and then my CI broke because chromedriver got a new version, which wasn't installed on my local machine, leading to the problem that the tests ran locally, but broke in the CI with the above mentioned error meassage.
josephan
commented
4 years ago I had the same issue as well, however my tests were passing on CI but not local (macOS).
My CI was running chromedriver 79 and my local was running chromedriver 77.
I fixed it by running brew cask upgrade chromedriver
sveredyuk
commented
4 years ago +1 to @PragTob
I had the same issue on GitlabCI and local machine.
Gitlab CI + Wallaby.Experimental.Chrome
PJUllrich
commented
4 years ago I ran into this issue again locally and would like to recommend to verify that your chromedriver has the same version as your Chrome-Browser, if you have one installed. In my case, Chrome updated itself to Version 81, but my chromedriver was still at 79. That lead to the {"error" ,"invalid session id"} error. After updating chromedriver to 81, my tests ran through successfully again.
bitboxer
commented
4 years ago I also ran into this problem and have no clue on how to make chrome+wallaby work on travis ci. Details here. Would love some help.
eahanson
commented
3 years ago I had the same problem in the past, and my fix was what @PJUllrich recommended: make sure Chrome and Chromedriver are at the same version. Here's a bash script that I run as part of my apps' doctor script:
#!/usr/bin/env bash
chromedriver_path=$(command -v chromedriver)
chrome_path="/Applications/Google Chrome.app/Contents/MacOS/Google Chrome"
chromedriver_version=$("${chromedriver_path}" --version)
chrome_version=$("${chrome_path}" --version)
chromedriver_major_version=$("${chromedriver_path}" --version | cut -f 2 -d " " | cut -f 1 -d ".")
chrome_major_version=$("${chrome_path}" --version | cut -f 3 -d " " | cut -f 1 -d ".")
if [ "${chromedriver_major_version}" == "${chrome_major_version}" ]; then
exit 0
else
echo "Wallaby often fails with 'invalid session id' if Chromedriver and Chrome have different versions."
echo "Chromedriver version: ${chromedriver_version} (${chromedriver_path})"
echo "Chrome version : ${chrome_version} (${chrome_path})"
exit 1
fi
lpender
commented
3 years ago @eahanson Thanks for the script it worked great for me!
Little update for Linux compatibility in CI:
#!/usr/bin/env bash
chromedriver_path=$(command -v chromedriver)
unameOut="$(uname -s)"
case "${unameOut}" in
Linux*) chrome_path="/usr/bin/google-chrome";;
Darwin*) chrome_path="/Applications/Google Chrome.app/Contents/MacOS/Google Chrome";;
esac
chromedriver_version=$("${chromedriver_path}" --version)
chrome_version=$("${chrome_path}" --version)
chromedriver_major_version=$("${chromedriver_path}" --version | cut -f 2 -d " " | cut -f 1 -d ".")
chrome_major_version=$("${chrome_path}" --version | cut -f 3 -d " " | cut -f 1 -d ".")
if [ "${chromedriver_major_version}" == "${chrome_major_version}" ]; then
echo "Chromedriver matches chrome version ✓"
exit 0
else
echo "Wallaby often fails with 'invalid session id' if Chromedriver and Chrome have different versions."
echo "Chromedriver version: ${chromedriver_version} (${chromedriver_path})"
echo "Chrome version : ${chrome_version} (${chrome_path})"
exit 1
fi
btihen
commented
3 years ago I have the same problem on MacOS X: 11.2.3 (20D91) I installed using ASDF: cat .tool-versions erlang 23.3.1 elixir 1.11.4-otp-23 nodejs 14.15.5 Postgres 13.2
wallaby 0.28
I have the same version of chromedriver as my browser I couldn't get wallaby to find chromedriver in the path until I installed it locally in my project.
If there is a script to change please tell me which file to change - (I am relatively new to Phoenix/Elixir and completely new to Wallaby)
I followed the setup instructions at: https://www.tddphoenix.com/setting-up/
✔ ~/devel/learning/germanvelasco/tdd_phoenix/chatter [install_wallaby L|✚ 9…9] 19:41 $ mix test ..
1) test user can visit homepage (ChatterWeb.UserVisitsHomepageTest) test/features/user_visits_homepage_test.exs:4 ** (RuntimeError) invalid session id code: |> visit("/") stacktrace: (wallaby 0.28.0) lib/wallaby/httpclient.ex:136: Wallaby.HTTPClient.check_for_response_errors/1 (wallaby 0.28.0) lib/wallaby/httpclient.ex:56: Wallaby.HTTPClient.make_request/5 (wallaby 0.28.0) lib/wallaby/webdriver_client.ex:329: Wallaby.WebdriverClient.visit/2 (wallaby 0.28.0) lib/wallaby/driver/log_checker.ex:6: Wallaby.Driver.LogChecker.check_logs!/2 (wallaby 0.28.0) lib/wallaby/browser.ex:1057: Wallaby.Browser.visit/2 test/features/user_visits_homepage_test.exs:6: (test)
..
Finished in 3.0 seconds 5 tests, 1 failure
Randomized with seed 381981 ✘-1 ~/devel/learning/germanvelasco/tdd_phoenix/chatter [install_wallaby L|✚ 9…9] 19:41 $ chromedriver Starting ChromeDriver 89.0.4389.23 (61b08ee2c50024bab004e48d2b1b083cdbdac579-refs/branch-heads/4389@{#294}) on port 9515 Only local connections are allowed. Please see https://chromedriver.chromium.org/security-considerations for suggestions on keeping ChromeDriver safe. ChromeDriver was started successfully.
matreyes
commented
2 years ago Hi!
I'm using Wallaby for scrapping so it failed randomly (with some pages on specific circumstances)
From developers.google.com:
By default, Docker runs a container with a /dev/shm shared memory space 64MB. This is typically too small for Chrome and will cause Chrome to crash when rendering large pages. To fix, run the container with docker run --shm-size=1gb to increase the size of /dev/shm. Since Chrome 65, this is no longer necessary. Instead, launch the browser with the --disable-dev-shm-usage flag.
So using:
Wallaby.start_session(
capabilities: %{
chromeOptions: %{
args: [
"--headless",
"--no-sandbox",
"window-size=1280,800",
"--fullscreen",
"--disable-gpu",
"--disable-dev-shm-usage"
]
}
}
)Worked for me (after lot of suffering).
bottlenecked
commented
1 year ago In case someone has troubles after the 115 release of chromedriver, here's an amended version of the script from @lpender and @eahanson that works for me
#!/usr/bin/env bash
# from https://github.com/elixir-wallaby/wallaby/issues/468#issuecomment-810518368
chromedriver_path=$(command -v chromedriver)
platform="mac-x64"
function get_download_url {
major_version=$1
curl -s curl "https://googlechromelabs.github.io/chrome-for-testing/latest-versions-per-milestone-with-downloads.json" \
| jq -r '.milestones."'$major_version'".downloads.chromedriver | .[] | select(.platform | contains("'$platform'")) | .url'
}
function download_chromedriver {
download_url=$1
curl -s $download_url -o "chromedriver_${platform}.zip"
}
function install_chromedriver_executable {
unzip -oq "chromedriver_${platform}.zip"
sudo mv "chromedriver-${platform}/chromedriver" "/usr/local/bin/chromedriver"
rm "chromedriver_${platform}.zip"
rm -rf "chromedriver-${platform}"
}
unameOut="$(uname -s)"
case "${unameOut}" in
Linux*) chrome_path="/usr/bin/google-chrome";;
Darwin*) chrome_path="/Applications/Google Chrome.app/Contents/MacOS/Google Chrome";;
esac
chromedriver_version=$("${chromedriver_path}" --version)
chrome_version=$("${chrome_path}" --version)
chromedriver_major_version=$("${chromedriver_path}" --version | cut -f 2 -d " " | cut -f 1 -d ".")
chrome_major_version=$("${chrome_path}" --version | cut -f 3 -d " " | cut -f 1 -d ".")
if [ "${chromedriver_major_version}" == "${chrome_major_version}" ]; then
echo "Chromedriver matches chrome version ✓"
exit 0
else
echo "Wallaby often fails with 'invalid session id' if Chromedriver and Chrome have different versions."
echo "Chromedriver version: ${chromedriver_version} (${chromedriver_path})"
echo "Chrome version : ${chrome_version} (${chrome_path})"
echo ""
echo "Install latest version? [y/n]"
read install
echo "install = $install"
if [[ "$install" == "y" ]]; then
echo "determining latest chromedriver version under ${chrome_major_version} ..."
url=$(get_download_url $chrome_major_version)
echo "downloading chromedriver version ${chrome_major_version}..."
$(download_chromedriver $url)
echo "installing chromedriver (requires sudo)..."
$(install_chromedriver_executable)
echo "Done :)"
else
exit 1
fi
fi
Howdy fellow wallabyers!
We have some tests that are stubbornly flaky on CI and sometimes locally and I can't for the live of me figure out why that might be happening.
I couldn't reproduce locally right now but on CI a colleague had to rerun the tests 5 times for them to pass.
Error
Test Code
Multiple tests are randomly failing, one of them being our easiest:
The others are usually simple CRUD tests (app isn't that complex yet) :) Often times 2 of them are failing. Sometimes it's just one.
There is no database access here. Just good old landing page. The session is also piped through from beginning to end so no issues with stale sessions.
Environment
The whole thing reminds me a bit of #221
Any ideas/help/input on how to remedy this would be highly appreciated :ok_hand: