Closed MarcoLeder-zz closed 4 years ago
Hello @MarcoLeder, Thanks for the support !
As debugging steps could you try :
to directly execute this command and look if you have any output (stdout and/or screenshot file)
/opt/phantomjs-2.1.1-linux-i686/bin/phantomjs --ignore-ssl-errors=true --ssl-protocol=any --ssl-ciphers=ALL "/usr/local/lib/python3.7/dist-packages/webscreenshot/webscreenshot.js" url_capture=http://192.168.0.220:80 output_file="/somepath/screenshots/http_192.168.0.220_80.png" width=1200 height=800 format=png quality=75 ajaxtimeout=1400 maxtimeout=1800
to directly execute this command to look for phantomjs
error (see the debug
flag) :
/opt/phantomjs-2.1.1-linux-i686/bin/phantomjs --debug=true --ignore-ssl-errors=true --ssl-protocol=any --ssl-ciphers=ALL "/usr/local/lib/python3.7/dist-packages/webscreenshot/webscreenshot.js" url_capture=http://192.168.0.220:80 output_file="/somepath/screenshots/http_192.168.0.220_80.png" width=1200 height=800 format=png quality=75 ajaxtimeout=1400 maxtimeout=1800
do you have an X server on your raspberry ? because if not, you should pass --no-xserver
to webscreenshot
(before that, apt-get install xvfb
)
Can you try another renderer ? Cheers.
Ok after quite some tinkering I think we are very close to getting it to work. I do not want to leave any steps out which I did or tried, in case they work for someone else :)
First I ran your 1st suggested command and got the following back:
Auto configuration failed
3069648912:error:25066067:DSO support routines:DLFCN_LOAD:could not load the shared library:dso_dlfcn.c:185:filename(libssl_conf.so): libssl_conf.so: Cannot open the shared object file: File or directory not found
3069648912:error:25070067:DSO support routines:DSO_load:could not load the shared library:dso_lib.c:244:
3069648912:error:0E07506E:configuration file routines:MODULE_LOAD_DSO:error loading dso:conf_mod.c:285:module=ssl_conf, path=ssl_conf
3069648912:error:0E076071:configuration file routines:MODULE_RUN:unknown module name:conf_mod.c:222:module=ssl_conf
Therefore I went to google a bit and found out that phantomJS is not supported for arm architectures.. (at least from what I found) and that this might have been the problem
I then switched to the chromium renderer binary since phantomJS does not seem to be an option anymore:
root@hostname:/somepath# python webscreenshot -vv --renderer-binary /usr/lib/chromium-browser/chromium-browser 192.168.0.220
webscreenshot.py version 2.91
[DEBUG][General] Options: Namespace(URL='192.168.0.220', ajax_max_timeouts='1400,1800', cookie=None, crop=None, format='png', header=None, http_password=None, http_username=None, imagemagick_binary=None, input_file=None, label=False, label_bg_color='NavajoWhite', label_size=60, log_level='DEBUG', multiprotocol=False, no_xserver=False, output_directory='/somepath/screenshots', port=None, proxy=None, proxy_auth=None, proxy_type=None, quality=75, renderer='phantomjs', renderer_binary='/usr/lib/chromium-browser/chromium-browser', ssl=False, timeout=30, verbosity=2, window_size='1200,800', workers=4)
[INFO][General] '192.168.0.220' has been formatted as 'http://192.168.0.220:80' with supplied overriding options
[+] 1 URLs to be screenshot
[DEBUG][http://192.168.0.220:80] Shell command to be executed
'/usr/lib/chromium-browser/chromium-browser --ignore-ssl-errors=true --ssl-protocol=any --ssl-ciphers=ALL "/usr/local/lib/python3.7/dist-packages/webscreenshot/webscreenshot.js" url_capture=http://192.168.0.220:80 output_file="/somepath/screenshots/http_192.168.0.220_80.png" width=1200 height=800 format=png quality=75 ajaxtimeout=1400 maxtimeout=1800'
[ERROR][http://192.168.0.220:80] Shell command PID 28298 returned an abnormal error code: '1'
[ERROR][http://192.168.0.220:80] Screenshot somehow failed
[+] 0 actual URLs screenshot
[+] 1 error(s)
http://192.168.0.220:80
Then I tried your first command again but with the chromium renderer binary now so I entered the following:
/usr/lib/chromium-browser/chromium-browser --ignore-ssl-errors=true --ssl-protocol=any --ssl-ciphers=ALL "/usr/local/lib/python3.7/dist-packages/webscreenshot/webscreenshot.js" url_capture=http://192.168.0.220:80 output_file="/somepath/screenshots/http_192.168.0.220_80.png" width=1200 height=800 format=png quality=75 ajaxtimeout=1400 maxtimeout=1800
This was the result:
[28363:28363:0616/104200.582523:ERROR:zygote_host_impl_linux.cc(89)] Running as root without --no-sandbox is not supported. See https://crbug.com/638180.
I therefore first assumed that I had picked the wrong filetype for rendering and went to usr/lib/chromium-browser/ where there are quite some files:
browser_tests_apprtc.runtime_deps
flash_manifest.json
master_preferences
top_domain_generator
bytecode_builtins_list_generator
headless_lib.pak
MEIPreload
torque
cddl
icudtl.dat
natives_blob.bin
transport_security_state_generator
chrome_100_percent.pak
libffmpeg.so
neon
v8_context_snapshot.bin
chrome_200_percent.pak
libpepflashplayer.so
plugins
v8_context_snapshot_v7.bin
chrome-sandbox
libs
protozero_plugin
xdg-mime
chromium-browser
libVkICD_mock_icd.so
resources.pak
xdg-settings
chromium-browser-v7
locales
snapshot_blob.bin
Flash.htm
make_top_domain_list_for_edit_distance
swiftshader
I then wanted to ask you if I should use something else than chromium-browser, given the list of files above.
I then took another look at the --no-sandbox error (which was caused because I used root I assumed) and tried to solve it. I went ahead and executed the script WITHOUT being root and this is what happened:
Chromium opens quite a lot of tabs (9 in total) with the first file showing your entire code. The other files all look rather the same, they all say not able to reach xxx (see screenshot).
I must say that is the first time the browser window actually opened! Has never happened before so we are definitely closer to the finish line. Thank you ever so much for your time :) Any ideas on what to do next?
Oh yeah and btw I have tried with other websites than just 192.168.0.220 but still no luck :( Xserver should be in use since I use Raspberry Pi with desktop interface:
whereis Xserver
Xserver: /usr/share/man/man1/Xserver.1.gz
Furthermore, when using the --no-xserver but leaving everything the same this happens:
renderer binary could not have been found in your current PATH environment variable, exiting
Oh yeah totally forgot - here the output with chromium and --debug flag:
/usr/lib/chromium-browser/chromium-browser --debug=true --ignore-ssl-errors=true --ssl-protocol=any --ssl-ciphers=ALL "/usr/local/lib/python3.7/dist-packages/webscreenshot/webscreenshot.js" url_capture=http://192.168.0.220:80 output_file="/somepath/
![IMG_0496](https://user-images.githubusercontent.com/55808530/84758094-4c323b80-afc5-11ea-97ac-9e8c146f38d2.jpeg)
screenshots/http_192.168.0.220_80.png" width=1200 height=800 format=png quality=75 ajaxtimeout=1400 maxtimeout=1800
[3907:4215:0616/112315.977653:ERROR:object_proxy.cc(632)] Failed to call method: org.freedesktop.DBus.Properties.Get: object_path= /org/freedesktop/UPower: org.freedesktop.DBus.Error.ServiceUnknown: The name org.freedesktop.UPower was not provided by any .service files
[3907:4215:0616/112315.978325:ERROR:object_proxy.cc(632)] Failed to call method: org.freedesktop.UPower.GetDisplayDevice: object_path= /org/freedesktop/UPower: org.freedesktop.DBus.Error.ServiceUnknown: The name org.freedesktop.UPower was not provided by any .service files
[3907:4215:0616/112315.979235:ERROR:object_proxy.cc(632)] Failed to call method: org.freedesktop.UPower.EnumerateDevices: object_path= /org/freedesktop/UPower: org.freedesktop.DBus.Error.ServiceUnknown: The name org.freedesktop.UPower was not provided by any .service files
Fontconfig error: Cannot load default config file
[3933:3933:0616/112316.741959:ERROR:gl_surface_presentation_helper.cc(259)] GetVSyncParametersIfAvailable() failed for 1 times!
[3933:3933:0616/112316.816580:ERROR:gl_surface_presentation_helper.cc(259)] GetVSyncParametersIfAvailable() failed for 2 times!
[3933:3933:0616/112316.893609:ERROR:gl_surface_presentation_helper.cc(259)] GetVSyncParametersIfAvailable() failed for 3 times!
[3933:3933:0616/112316.932119:ERROR:gl_surface_presentation_helper.cc(259)] GetVSyncParametersIfAvailable() failed for 4 times!
[3933:3933:0616/112316.981122:ERROR:gl_surface_presentation_helper.cc(259)] GetVSyncParametersIfAvailable() failed for 5 times!
[3933:3933:0616/112317.010311:ERROR:gl_surface_presentation_helper.cc(259)] GetVSyncParametersIfAvailable() failed for 6 times!
[3933:3933:0616/112317.039930:ERROR:gl_surface_presentation_helper.cc(259)] GetVSyncParametersIfAvailable() failed for 7 times!
[3933:3933:0616/112317.054077:ERROR:gl_surface_presentation_helper.cc(259)] GetVSyncParametersIfAvailable() failed for 8 times!
[3933:3933:0616/112317.089111:ERROR:gl_surface_presentation_helper.cc(259)] GetVSyncParametersIfAvailable() failed for 9 times!
[3933:3933:0616/112317.105537:ERROR:gl_surface_presentation_helper.cc(259)] GetVSyncParametersIfAvailable() failed for 10 times!
[3933:3933:0616/112317.169144:ERROR:gl_surface_presentation_helper.cc(259)] GetVSyncParametersIfAvailable() failed for 11 times!
[3933:3933:0616/112317.193274:ERROR:gl_surface_presentation_helper.cc(259)] GetVSyncParametersIfAvailable() failed for 12 times!
[3933:3933:0616/112317.222567:ERROR:gl_surface_presentation_helper.cc(259)] GetVSyncParametersIfAvailable() failed for 13 times!
[3933:3933:0616/112317.236980:ERROR:gl_surface_presentation_helper.cc(259)] GetVSyncParametersIfAvailable() failed for 14 times!
[3933:3933:0616/112317.246041:ERROR:gl_surface_presentation_helper.cc(259)] GetVSyncParametersIfAvailable() failed for 15 times!
[3933:3933:0616/112317.259452:ERROR:gl_surface_presentation_helper.cc(259)] GetVSyncParametersIfAvailable() failed for 16 times!
[3933:3933:0616/112317.265308:ERROR:gl_surface_presentation_helper.cc(259)] GetVSyncParametersIfAvailable() failed for 17 times!
[3933:3933:0616/112317.280241:ERROR:gl_surface_presentation_helper.cc(259)] GetVSyncParametersIfAvailable() failed for 18 times!
[3933:3933:0616/112317.285174:ERROR:gl_surface_presentation_helper.cc(259)] GetVSyncParametersIfAvailable() failed for 19 times!
[3907:4252:0616/112324.704247:ERROR:udev_watcher.cc(94)] Failed to begin udev enumeration.
Best regards Marco Leder
Therefore I went to google a bit and found out that phantomJS is not supported for arm
Indeed, but it seems that there are some unofficial builds here and here and here that you can try.
I then switched to the chromium renderer binary since phantomJS does not seem to be an option anymore
You forgot to specify the renderer -r
option, your tests show that you use the phantomjs
renderer
(default) while specifying a chrome binary (and of course, it cannot work).
Can you try this command:
$ python webscreenshot -vv -r chromium --renderer-binary /usr/lib/chromium-browser/chromium-browser 192.168.0.220
Oh my god thank you ever so much! Totally my fault, I was not aware that I have to specify both of them :) Thank you for taking the time to solve my issue 👍 Indeed it works now, however with using the chromium render binary I ran into this problem as well.. hope that you are able to add this feature soon !
Best regards Marco Leder
Dear Maaaaz
First of all thank you so much for such a great tool (eventhough I am not able to use it correctly). I am running latest raspian os (32bit) on my Raspberry Pi 4 and everything with installation worked out pretty darn well. However, if I run the script as intended, it does not save the screenshot nor does it return any errors.
All the folders in which I am working have permission 777, including the screenshots folder which was sucessfully created by your program. I have tried to force ssl and tried different websites but none of them works. I used to use the renderer binary from chromium but as I saw no results switched to PhantomJS, but still nothing.
Here is the debug info (though I don't know if this is worth anything for you):
As you can see I am running latest version of everything so and system is up to date :) Any ideas?
Best regards Marco Leder