spice html5 client not working

[henickr]

Hi, trying to use the new html5 web sockets client but after cloning websockify and starting it ./run --token-plugin TokenFile --token-source /tmp/kvm-vdi 5959...

root@titan:~/henick/websockify# lsof -i:5959 COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME python 30666 root 3u IPv4 2750944 0t0 TCP *:5959 (LISTEN)

then... visiting the url /kvm-vdi/client_pools.php

I am kicked to a login page to which I login with valid credentials to be allowed through only to receive the message LOGIN_FAILURE on the following page.

I haven't configured or installed anything so I assume that is the issue but the directions simply say run using that command and don't provide much else. Should I be building websockify from source or running it in some other way than I have said here?

i am posting this because i did not expect a problem with authentication ...

Thanks.

edit: i've since done an easy_install numpy and 'python setup.py build' then install on websockify in an attempt to omit usage of the provided copy in lieu of one built by this system. end result same.

[brianhenson]

I too have this issue

[Seitanas]

Hello, sorry for late answer - had a work trip. You have to create client (Clients>add client) in dashboard and also put it to atleast one pool (Add clients to pool). If user has no pools available, it will be ignored. If you are using AD/LDAP users, you must have to create at least one group for that user add it to pool (add AD group Add AD group to pool).

[brianhenson]

worked perfectly. Can you add that to the readme file? I used it to setup everything

[Seitanas]

Sure I will.

[henickr]

ok yeah, the confusing part for me here was that the client you add is not the same login as the dashboard itself. you add a completely different user which you login to at that page and then, providing you have a pool created with contained vms started, then the client_pools.php populates.

it does seem to 'work' in the sense that there aren't any errors. I can connect directly to the same vm using the spice client and it displays the desktop but the browser displays a black page with no errors.

I suspect that may be a browser issue or something so I am trying a few different methods now.

admittedly i'm at a bit of a loss as to how to troubleshoot this....

[Seitanas]

You should try running websockify in interactive mode (omit --daemon parameter). You will be able to see websockify output then. There are many issues with websockets. One of them - if dashboard is running on https, you also have to provide ssl certificate to websockify. Also you need to know, that self-signed certificates does not work with firefox browser. This is a known issue.

[henickr]

your documentation doesn't specify a --daemon parameter... it says ./run --token-plugin TokenFile --token-source /tmp/kvm-vdi 5959

but.... when I run it this way... ./run --token-plugin TokenFile --token-source /tmp/kvm-vdi 5959 --log-file=/tmp/kvm-vdi/websockets.log &

I have a log file to tail... then hit it in the browser gives me ... handler exception: need more than 1 value to unpack

So yeah its a websockets issue... I'll figure that out on the internet, I don't suspect I'd be lucky enough for you to know the answer to that one.

[henickr]

incorrect config file based on their issues page.... https://github.com/kanaka/websockify/issues/235

[Seitanas]

Are tokens in /tmp/kvm-vdi directory created? Also if you need to daemonize websockify, - do not use &. Use --daemon

[henickr]

using this now instead of documentation... ./run --token-plugin TokenFile --token-source /tmp/kvm-vdi 5959 --log-file=/tmp/kvm-vdi/websockets.log --daemon

same result. no there are no tokens generated in /tmp/kvm-vdi/, just the log i created.

[Seitanas]

This is odd. Perhaps there's anything in webserver error log? There should be token files created in /tmp/kvm-vdi Are you running websockify on the same server, on which dashboard service is deployed?

[henickr]

yes same server as dashboard. i would only expect me to have an odd issue. very typical lately...

only thing in the httpd_error log while client_pools.php is open is that it is very chatty now with repeated php errors stating undefined index 'ad_user' and 'state' for draw_html5_buttons.php ...

otherwise nothing special. i've considered switching this to debian but its been so much fun trying to fix centos why stop now ;)

[Thu Aug 11 15:37:26.112319 2016] [:error] [pid 26620] [client 129.173.67.12:61521] PHP Notice: Undefined index: ad_user in /var/www/html/kvm-vdi/draw_html5_buttons.php on line 33, referer: http://omitted.research.cs.dal.ca/kvm-vdi/client_pools.php [Thu Aug 11 15:37:26.117114 2016] [:error] [pid 26620] [client 129.173.67.12:61521] PHP Notice: Undefined index: state in /var/www/html/kvm-vdi/draw_html5_buttons.php on line 54, referer: http://omitted.research.cs.dal.ca/kvm-vdi/client_pools.php

[Seitanas]

strangely enough, it does create /tmp/kvm-vdi folder. Have no idea, why it does not create tokens inside that folder though. selinux perhaps?

[Seitanas]

Oh, try moving log file out of /tmp/kvm-vdi folder. Also delete the one that's already inside.

[henickr]

no i created that folder because it didn't create it and i needed a log to look at....so you're probably right there somehow, perms or something but....

SELinux status: disabled

and its /tmp so perms issue would be confusing.

when i delete it, i get this... so i will forego the log for now because i now have a bit of an idea whats going on where before i had no clue... so don't really need it at the moment. IOError: [Errno 2] No such file or directory: '/tmp/kvm-vdi/websockets.log'

and running this now, no it does not create anything again. ./run --token-plugin TokenFile --token-source /tmp/kvm-vdi 5959 --daemon

strace i guess.... :| side note, should probably make a systemctl command for running this thing so it would be easier to figure this out.... i'll do that in a sec.

[Seitanas]

If you've created /tmp/kvm-vdi by hand, you need to change permissions of that folder for webserver to be able to write there. Just delete that folder and let dashboard to create it by itself. Also do not put log into that folder. Logfile may be misinterpreted by websockify

[henickr]

already tried that, did nothing.... looking through strace now to find out why... this might take a while...

[Seitanas]

Doubt if strace will help you. You should debug web part. Web part creates directory and tokens. Websockify just reads files in directory on demand.

[henickr]

maybe i'm misunderstanding that but the run command has that path directly as an argument so in an strace i must see it do something with it somewhere... nothing appears in error_log and I don't see another area to look...

[Seitanas]

That path is accessed by websockify module only, when you try to access html5 console. It does not do anything when started or in idle mode. https://github.com/kanaka/websockify/blob/master/websockify/token_plugins.py#L46 The way it works: when you press on a pool icon on your client_pools.php site, KVM-VDI selects a VM from that pool for you, starts it, then calls websocket.php and creates token file in /tmp/kvm-vdi folder (also folder, if needed) with the name of that VM. It puts name, address and port of that VM inside the toke file. After that it calls html5 client and defines token, which it should access. Also a SPICE password.

[henickr]

ok great, i needed to find this

if (!file_exists('/tmp/kvm-vdi/')) {
    mkdir('/tmp/kvm-vdi', 0777, true);
}

so yeah, odd it doesn't error to httpd_error indeed....

[Seitanas]

okay, could you try changing code a bit: mkdir('/tmp/kvm-vdi');

[henickr]

i was just thinking the same thing.... i removed the kvm-vdi component completely as a test.... and had tokens generated directly into tmp temporarily... also changed the websockets command to point directly at /tmp....

still no tokens... i'll try that way now..

[henickr]

ok... friggin weird.... I commented out the if portion and forced the code to create regardless.

the error log throws this now

[Thu Aug 11 16:36:45.210846 2016] [:error] [pid 5252] [client 129.173.67.12:61946] PHP Warning: mkdir(): File exists in /var/www/html/kvm-vdi/websocket.php on line 21, referer: http://titan.research.cs.dal.ca/kvm-vdi/client_pools.php

It does not.

[Seitanas]

It works as intended. If directory already exists, it is impossible to create it.

[henickr]

of course. but it doesn't :)

root@titan:~/henick/websockify# ls /tmp systemd-private-a80e5c25bb9544fb9361e3c4b7dc1007-httpd.service-aO59Xa systemd-private-a80e5c25bb9544fb9361e3c4b7dc1007-mariadb.service-S1uw6F

[Seitanas]

maybe there's kvm-vdi file inside /tmp?

[henickr]

nadda. friggin weird.

[henickr]

ok it gets stranger... rm -rf /tmp/* same result.

[Seitanas]

So kvm-vdi directory is still not created?

[henickr]

nope... after doing the rm -rf i hit the browser again with a tail -f /var/log/httpd/error_log | grep -i mkdir & running so I can see it error out on the forced mkdir every time. still nothing there.

but i can see it trying every time i refresh.

times like this make me wish i chose to be a debian eng :|

touch /tmp/test works .......

[Seitanas]

Okay, just create a simple test.php file with contents: <?php mkdir('/tmp/test'); file_put_contents("/tmp/test/file","content"); ?> and then try to access it via web browser.

[henickr]

same result.

root@titan:~/henick/websockify# ll -d /tmp drwxrwxrwt. 7 root root 99 Aug 11 16:45 /tmp

[Seitanas]

it should have been created /tmp/test/ folder and /tmp/test/file inside of it

[henickr]

it does throw this error when it happens though

root@titan:~/henick/websockify# [Thu Aug 11 16:52:32.102536 2016] [:error] [pid 12342] [client 129.173.67.12:62142] PHP Warning: mkdir(): No such file or directory in /var/www/html/kvm-vdi/test.php on line 3

maybe the php version.... 7.0.9? I can't see it being THAT drastically different...

[Seitanas]

Don't think so. Should work with 7 out of the box. Just create /tmp/kvm-vdi directory and chmod it to 777 then try using html5 client. See if tokens are created. If this won't work, time to ticket RH I think :)

[henickr]

also tried to su apache --shell=/bin/bash; mkdir test and it works

[henickr]

also tried to make the test file relative. it can't create anything inside the kvm-vdi web directory either.

i have a php dev here with me testing another system.

[henickr]

oh my good god. ok its fixed but still doesn't work.

You need to set PrivateTmp=false in /usr/lib/systemd/system/httpd.service
then systemctl daemon-reload; systemctl restart httpd.service

i still have a black screen but the tokens are generated now -- with contents.

I guess centos 7 apache has a private temp location that sometimes interferes with this that should be shut off to use. so the tokens are there now but still black screen.

[Seitanas]

What does websockify log say?

[henickr]

says just the typical startup stuff and the token file includes the name of the (randomly?) selected available vm (I have 10 VDI systems free in the pool and it chose number 5 -- arbitrary but anyway) and also the blade.domain.tld:port it is running on in the token.

I see in the url bar, the information has all been passed along with the password... one point of note.... vm number 5 was not started before the token was created. now it is....

only things i can think of are to run websockify in verbose mode... with the log...

./run --token-plugin TokenFile --token-source /tmp/kvm-vdi 5959 --log-file=/tmp/kvm-vdi/websockets.log --daemon --verbose

the log file only contains multiple copies of the socket cleanly starting... and verbose adds nothing valuable... but the vm it chose is started... so it did do something....

and other idea to turn on browser errors in php.ini but this causes a bit more odd behavior... with errors to browser turned on, the popup does not open with the url... and with errors off like default and i click the pool, a new tab pops open with this as the target...

http://omitted.research.cs.dal.ca/kvm-vdi/spice_html5/?host=omitted.research.cs.dal.ca&port=5959?password=correctpasswordfromdb&vmInfoToken=nameofvm

in both instances, with errors on or off, whatever vm is chosen in the token is auto started.

just won't display. so there are the ideas i had.... interesting however that the popup doesn't open with error reporting on but possibly unrelated... any other ideas?

[Seitanas]

Maybe your dashboard service is running on https?

[henickr]

nah, no https. another interesting side note i see is that in top on blade03 (the hypervisor that contains the vm it randomly chose -- 05) I see for the last 23h vdi-wrapper running with the qemu process below it. vdi-wrapper is supposed to go away when it loads the qemu-kvm process isn't it?

when i start it using the dashboard directly, i normally don't see a vdi-wrapper process at all because it typically finishes quickly -- unless i'm mistaken. so interesting....

[Seitanas]

Yes, vdi-wrapper should go away. It just calls kvm with required parameters and exits.

[henickr]

so, i tried a few more things related to increasing memory and ttls/execution limits of the JVM because while using other actual hypervisors like openvz, memory is often the cause of sudden unexplainable stops... until finally I got a stack trace from websockify... its the same error though that suggests the configuration is wrong. its likely something centos7 specific with websockify so i will work with them and get it resolved then post the result back here when i am finished.

129.173.x.x: new handler Process handler exception: need more than 1 value to unpack exception Traceback (most recent call last): File "/root/henick/websockify/websockify/websocket.py", line 930, in top_new_client client = self.do_handshake(startsock, address) File "/root/henick/websockify/websockify/websocket.py", line 860, in do_handshake self.RequestHandlerClass(retsock, address, self) File "/root/henick/websockify/websockify/websocket.py", line 114, in init SimpleHTTPRequestHandler.init(self, req, addr, server) File "/usr/lib64/python2.7/SocketServer.py", line 649, in init self.handle() File "/root/henick/websockify/websockify/websocket.py", line 581, in handle SimpleHTTPRequestHandler.handle(self) File "/usr/lib64/python2.7/BaseHTTPServer.py", line 340, in handle self.handle_one_request() File "/usr/lib64/python2.7/BaseHTTPServer.py", line 328, in handle_one_request method() File "/root/henick/websockify/websockify/websocket.py", line 543, in do_GET if not self.handle_websocket(): File "/root/henick/websockify/websockify/websocket.py", line 485, in handle_websocket self.validate_connection() File "/root/henick/websockify/websockify/websocketproxy.py", line 52, in validate_connection host, port = self.get_target(self.server.token_plugin, self.path) File "/root/henick/websockify/websockify/websocketproxy.py", line 128, in get_target result_pair = target_plugin.lookup(token) File "/root/henick/websockify/websockify/token_plugins.py", line 51, in lookup self._load_targets() File "/root/henick/websockify/websockify/token_plugins.py", line 30, in _load_targets tok, target = line.split(': ') ValueError: need more than 1 value to unpack Ignoring interrupted syscall

[Seitanas]

You should try websockify from git to see if it works. Just clone it, change to its directory and use ./run command with your parameters.

[henickr]

same result. is there anything specific about the usage i should be aware of when presenting the problem to them short of providing the error? Anything specific that would help with the -way- the code uses the daemon?

[Seitanas]

I'm still nor very convinced, that its websockify problem. Are tokens now created in /tmp/kvm-vdi folder?

[henickr]

yes and everything expected appears in the URI but as soon as the browser window pops open to the black screen the error presented is the stack trace I posted immediately above this message with the error suggesting the configuration file is wrong. So likely centos 7 is somehow passing the config wrong or using it in some unexpected way, i'm guessing now.

root@titan:~/henick/git-websockify/websockify# 129.173.x.x: new handler Process
handler exception: need more than 1 value to unpack

[Seitanas]

And there are no other type of files in /tmp/kvm-vdi folder (like log file or something), just tokens?