Does they have iKVM support schedule?

[NorihisaNakai]

Hi, We want to connect to iKVM (Supermicro motherboard IPMI VNC Server) on noVNC. But, iKVM VNC protocol use original future on VNC protocol. if project have to support iKVM, would you tell me schedule?

Thank you for your cooperation.

Norihisa Nakai

[DirectXMan12]

So, it looks like someone took a shot at it. If you can provide some documentation on how the IPMI VNC Server works, we will consider attempting to add support. It's difficult, however, since we do not have an actual device to test against, AFAIK.

[jimdigriz]

I'm working on it (doing this on and off in my spare time):

https://github.com/jimdigriz/noVNC/tree/aten-ikvm (rebased regularly from pixelformat, which its-self is rebased against upstream master)

Had to first solve the pixel format constraint (iKVM uses RGB555 and noVNC is hardcoded for RGB888) which is done (https://github.com/jimdigriz/noVNC/tree/pixelformat), but it does bump the IE dependency to 10 as we have to handle endian/bitshifting conditions.

[jimdigriz]

My aten-ikvm branch seems to work (lots of wisdom from https://github.com/thefloweringash/chicken-aten-ikvm/blob/master/lens/lens.rb), needs others to test it though and things to bear in mind:

• to send your credentials, you use 'username:password' in the password field; similar to how xvp does things
• browser linky would look like http://localhost:6080/vnc.html?host=localhost&port=6080&logging=warn&password=USERNAME:PASSWORD
• not tried resolution changing, but should work
• no remote media support, but I recommend folks just use floppy images (uploadable with curl) with iPXE - http://www.digriz.org.uk/ipmi
• mouse support in there, not tested
• someone needs to try this against other 'normal' VNC servers, I have tested against Debian 'wheezy's tightvnc 1.3.9-6.4 server
• the Intel ATM tweak needs testing, but should work due to the pixelformat stuff

The pixelformat branch, which the aten-ikvm branch branches off, should hopefully lay down some of the code needed to address https://github.com/kanaka/noVNC/issues/396 too.

[jimdigriz]

Now tested against the following Supermicro motherboards:

• X8DTL
• X9SCL/X9SCM
• X9SCM-F - thanks to Brian Rak for testing!
• X9DRD-iF
• X9SRE/X9SRE-3F/X9SRi/X9SRi-3F
• X10SLD

Unfortunately it seems newer models (e.g. X10SLM-F and X10SLE) use some new encoding method 0x57 (87) which needs to be reverse engineered.

[DirectXMan12]

Sweet! We'd be glad to merge your branch once you think it's finished. Please include unit tests, if you can, and then just add a pull request (reference this bug).

[bkanuka]

Do we need websockify to use iKVM?

[jimdigriz]

Yes, it uses VNC over port 5900/tcp. ATEN have just 'forked' the VNC protocol into something...else :-/

[astrand]

I'm not sure it's a good idea to merge support for an undocumented "forked" VNC protocol. How are we supposed to be able to maintain that, without some reference? In my opinion, noVNC shouldn't implement anything that's not documented by https://github.com/rfbproto/rfbproto. So if you really care about this ATEN protocol, my suggestion would be to first submit patches to the rfbproto project that documents this stuff.

[jimdigriz]

Congratulations astrand for volunteering!

Alas, if this is where the party is going, I am going to get my coat.

DirectXMain12, if you find any technical problems with the patch (regressions, patch sucks, sets pants on fire, etc) then let me know and I will jump back in and fix it. At some stage I will dip in and try to figure out the encoding ATEN have started to use on their newer systems so we can get that squished and spare folks enduring the ghastly Java client.

[astrand]

I'm not volunteering for anything. I just want to make sure that me and my team doesn't need to spend many years trying to maintain code that implements some undefined hackish protocol that we have no interest in.

This is just my opinion, the other developers might think differently. Also, if you are willing to maintain the code for some time (rather than just providing a one time patch), that would also help.

[samhed]

@jimdigriz thanks for the work on this!

As mentioned, I don't think anyone in the noVNC team has any hardware for testing this though. This is why I agree with @astrand in the way that we need to be cautious with merging stuff regarding the RFB protocol which is undocumented.

@DirectXMan12 and @kanaka what do you think about this?

[DirectXMan12]

@jimdigriz : There's a couple things that I'd like you to fix on the pull request before we even consider merging it. I've left comments there

@samhed , @astrand , @kanaka : So, I'm a bit divided on this. One the one hand, supporting undocumented features is always tricky. On the other hand, I've used the Java client (it might not have been the ATEN one, but something similar), and, boy, was it horrible. Having an alternative, especially one that companies could easily embed in their standard interface (i.e. noVNC), would be really nice, and would also look good for the noVNC project itself ("Use noVNC to simplify administering your lab, even without an OS installed!").

I wonder if there's any way we could support this via some sort of plugin system? Between this and the issues with Tight, I'm beginning to wonder if we shouldn't have some sort of pluggable encoding architecture. @jimdigriz : could you envision this working inside something like that?

[bkanuka]

@DirectXMan12 I fall exactly into the group you mentioned. I was exposed to this bug report and noVNC by searching for alternatives to that horrible horrible Java client. Your hesitation to merge undocumented protocols is completely understandable, but man would I love to use noVNC instead of java :-)

[jimdigriz]

I originally started writing the ATEN support for Debian's tightvnc viewer, but as I prefer to code myself with a reference implementation to hand, I quickly shifted to using noVNC as everyone and their dog can read Javascript. This was my primary objective, to help someone else write their own client implementation. That my infrastructure team no longer has to endure using the Java client, was just a cherry on the cake. For me, thats "mission accomplished".

I would be careful with using the word 'plugin', I have no idea what the implications (or need) to have something working via "some sort of plugin system". Code is code, it can be written in whatever way needed. If you want to try your plugin approach, really, just write a prototype, if it takes more than two days, abandon it, what you have is good enough already. If you want to try anything, I would recommend noting that the RFB protocol lends its self to a client implementation that just builds a tree and walks up and down it, as in if there is a conditional (if/switch) statement in the code, it probably is being done wrong. The gripe I had with noVNC codebase was nothing to do with OO/seperation/etc, it was the byte queue/dequeuing handling and the twisty maze of if statements that staple together a state machine (the encoder handlers should not have conditionals in there).

To the patch, if you look at pixelformat..aten-ikvm, I actually thought it all slotted in rather neatly without too much pain. The auth hook could be made nicer and I agree it looks scary, but it does just seem to work automatically, including for tightvncserver. Writing the whole patch really was just me battling with Javascript and the quirks of the ATEN protocol; noVNC its-self was really nice to work with here. Thanks!

What concerns me though is that master..pixelformat patch is intrusive, and it was a complete ballache to write as noVNC completely ignored that maybe it would be forced to do local pixel transformation its self; all my time here was spent trying to minimise the impact of the patch. Being that that it covers something already documented plus a major (albeit hard) part of the RFB protocol, makes me feel that there is some bike shedding going on where the focus is on the minor patch and not the major intrusive work. Why is there even talk of plugins at this point?

I know this patch works with tightvnc and the seven different models of ATEN chaff I have access to at work and I have had one random Internet stranger I approached out of the blue try it with success too. Has anyone with write access to the project actually put aside what the patch is/does and just tested it for real regressions on real hardware in real life? If not, really can we deal with technical rather than abstract problems first. If they did, what did I break?

[DirectXMan12]

What concerns me though is that master..pixelformat patch is intrusive, and it was a complete ballache to write as noVNC completely ignored that maybe it would be forced to do local pixel transformation its self

I suspect @kanaka did that for convenience when he wrote it, but I'm not sure

Why is there even talk of plugins at this point?

I wouldn't use a plugin for the PixelFormat stuff -- it's just something we should have. Thanks for taking the time to work through that. If nothing else, I'll definitely merge that (although I really hope we can merge the rest as well ;-) ). The reason I was considering a plugin system is that it would make it easier for deployers of noVNC to chose which encodings they wanted to enable and disable.

I know this patch works with tightvnc and the seven different models of ATEN chaff I have access to at work and I have had one random Internet stranger I approached out of the blue try it with success too.

That's good to hear. Unfortunately, I'm not sure if any of us has access to hardware (I'm check with work to see if I do), so I'm mainly relying on you here to test the iKVM parts. I'll run some more local tests to make sure the patch works on normal RFB servers.

Thank you for all your hard work, and know that I really do want to see this in noVNC in some form. I just want to be careful not to get bitten by any bugs WRT "normal" RFB communication

[MartinVerges]

Hello,

we can provide some test systems for ATEN IPMI (Supermicro Servers) and would like to support your effort to implement it in noVNC. As you might know, IPMI is widely used on server systems and the java viewer is a pain in the a**. I personal want to get rid of that java viewer for better customer experience.

We are also willing to spend some money to get this solved and if somebody can do it, we would be glad to push noVNC to IPMI / KVM Support.

At the moment, it does not work in my test with X9SCM-F (other than stated) as it response "No supported sub-auth types!"

Best Regards Martin

[sbrynen]

When I try this against IPMI on a X9DRi-LN4F+ Supermicro, it hangs at "Starting VNC handshake"

[frovik]

I tested this software and can confirm it actually works quite well. The motherboard we tested it to work on was X7SPT-DF-D525+ . However, I expect most of x7-x8-x9 motherboards should work.

How can we get more momentum with this? I would be more than happy to provide access or even donate hardware to get more testing going here so we can have this integrated and even maintained.

[andrew-azarov]

We can provide the X10 system to test and debug

[DirectXMan12]

@andrew-azarov While I will not have time to work on this in the near future (I'm busy with some other stuff for work), we did, I some point, have some people interested in working on this. I will see if I can get them in touch with you.

[kelleyk]

I reverse-engineered the 0x57 format (and decent chunks of the iKVM client and server binaries) last year and got distracted by life before finishing a reimplementation of it. If there's renewed interest in merging 0x56/0x57 support into noVNC, I'd be happy to dust off my work.

It's worth nothing that the ATEN formats (and the underlying BMC hardware) support multiple different types of encoding and compression (one that's like JPEG with fixed tables and two types of color quantization, if memory serves), even if the current firmware only seems to use a very limited subset of that functionality (just the JPEG format, even at a black-and-white prompt; a fixed screen resolution; etc).

I have X9 (WPCM450) and X10 (AST2400) hardware handy to test with. There is also some X8/X9 hardware that uses a Renesas SH7757/7758, which I do not have access to.

[disaster123]

+1 that would be great the Java client just sucks

[andrew-azarov]

+1 That would be awesome. There is a way to connect media through samba, but the console client is not vnc compatible unfortunately.

[MartinVerges]

+1 that would be fantastic We would be happy to provide demo equipment for you

[disaster123]

@kelleyk any news on this topic?

[kelleyk]

@disaster123 I spent some time on this a few weeks ago, and I'm pretty sure that I have the JPEG side of things pretty well figured out. At that point, I got distracted by life.

When next I can set aside an afternoon, I'll try capturing some sample traffic and implementing something that can actually decode video frames. (It is also possible that I may need to dump the Huffman and quantization tables from the client, since they're hardwired; they may be fairly normal ones, or they may not be.) Once that's done, we'll just need to figure out what the best way of integrating it into noVNC is.

[kelleyk]

Just wanted to reassure everybody that I'm still working on this.

I've got it almost entirely figured out. There's a lot of dead code in the client corresponding to features that are simply never used, as far as I can tell, so I wound up reverse engineering way, way more than would otherwise have been necessary.

I've also finished reimplementing the majority of what I've learned about how the 0x57 video encoding works. Unfortunately, there's still a problem (or two); the output is not reasonable. (Drat.) At the moment I'm working on trying to determine whether this is because I screwed something up during the reimplementation effort, or because of something I'm misunderstanding about the data format.

Hope to have good news soon!

@disaster123 @MartinVerges @andrew-azarov What hardware (boards) are you guys using? I have a few to test with, but different BMCs and/or different firmware revisions might use different features of the protocol/format.

[disaster123]

X10sri and x10drh and x10drff and various x9 boards

[disaster123]

Thx for your work!

[andrew-azarov]

X10SLE-F X9SCD+-HF these mostly.

[MartinVerges]

X10SRi-F (01.89) X10SRi-F (01.67) X9DR3-F (03.15) X9DRW-F (2.35.0) X9SCM-F (03.19) and many more

[kelleyk]

Awesome; thanks, guys. I don't think that any of that hardware should be different in any important ways from the stuff that I have to test with, but at some point I'll ask for your help in testing with the various gear you've got just to make sure.

Also, I have some fun news: a few minutes ago, I successfully* decoded traffic that I captured moving across the wire into an image!

The asterisk is because something isn't quite right yet. In order to make sure that I was handling the JPEG-related parts of the format correctly, I wound up writing my own JPEG decoder in Python; that seems to be working (i.e. the output it generates looks just fine). However, the KVM image is fuzzy.

I have two thoughts as to what might be the problem:

• It could be that I screwed up in dumping one of the various tables that are hardwired into the server and client. (In a real JPEG image, these are specified at the beginning of the file, so this would explain why only the KVM stuff is having trouble.)
• It could be that the ATEN iVKM stuff depends on very particular rounding behavior; or, it could be that their math is "special" (not-quite-correct) and their server just happens to generate data that causes that special math to generate a correct-looking image. I'm hoping that that's not the case, because then I'd have to reverse-engineer that.

I'll keep investigating, but hopefully I've just done something stupid and it won't be hard to fix. Then all we need to do is port the decoder into an actual VNC client (like noVNC)!

[kelleyk]

I'm fairly positive that this is an issue with the quant tables.

[disaster123]

Thx that sounds great. Does anybody already Knie how to authenticate via novnc?

[kelleyk]

I think that that work has been done, since there is support for one of the other (older) ATEN protocols: https://github.com/kanaka/noVNC/commit/1db6fd83878aad22e082d049dff751f95d32be99

If not, we'll find out soon! ;)

[MartinVerges]

Hello,

X9DRW is a bit different. Its based on another vendor of IPMI. There are 2 types, ATEN and AMI based firmware. The X9DRW for example uses the "American Megatrends" AMI Firmware.

Thanks for your work so far, I'm going to test it today.

Best regards Martin

[kelleyk]

Aha, yes, the X9DRW--and the X9SRW, which I fortunately have access to--use the Rensas SH7757 BMC, so they may be a bit different (in fact, I'd bet that that the SH7757 boards don't use the 0x57 video format). The X10 boards (and many of the X9 boards) use the Aspeed AST2400.

In any event, I've done some more digging. All of the tables are right; it looks like it's a math problem. I still can't find any real problems with my code on normal JPEG images, but when I try decoding KVM traffic, it looks like I wind up getting very slightly different numbers early on and the error compounds as it carries through, leading to a slightly screwy picture.

Time to keep investigating!

[kelleyk]

Success!

I went back and implemented another IDCT routine; this time, I tried a fixed-precision one (similar to the "fast" integer AA&N implementation in libjpeg) that should be absolutely identical to what the ATEN client does... and what do you know, it works!

I'm going to take a few minutes to tidy up the code and put some notes on paper, and then I'll take a first look at how hard it will be to integrate into noVNC.

[disaster123]

great and thanks! i've seen SM has released the first REDFISH based ipmi fw's. Do you know if it will work with them too? https://www.supermicro.com/solutions/Redfish.cfm

[kelleyk]

I don't know, but I bet it will. Redfish is just an attempt at reinventing IPMI, as far as I'm aware; it shouldn't change how the iKVM stuff works.

[DirectXMan12]

@kelleyk let me know if you need any pointers on where to start with noVNC. The existing PR (#445) should be a good place to start, and should have some of the building blocks already implemented.

[kelleyk]

@DirectXMan12 Thank you for the offer! I'm sitting down now to take a look at getting some version of the aten-ikvm code that @jimdigriz contributed up-to-date with master and passing tests; that seems like the right place to start from.

Once I've got that set up, the integration points seem fairly straightforward: parse the 0x57 ("AST2100") format, pass data off to a JavaScript port of my reverse engineering work, and then hand back decoded blocks of image data.

If you have any more-detailed advice about the first part (I'll go take a look at the PR you linked and the other previous mentions of iKVM support), or if there are things you'd like me to keep in mind while cranking out code (to help the end product be the best it can be), I'd love to have the benefit of your wisdom.

Cheers!

[kelleyk]

Okay, it looks like your #445 is a rebased version of the goodies from @jimdigriz. Thank you for speaking up--saves me a bunch of time not to have to figure that out myself!

[DirectXMan12]

if there are things you'd like me to keep in mind while cranking out code

The main pieces of advice that I can give are:

1. Make a websockify recording of the exchanges if you can, which can be used to easily reproduce issues with vnc_playback.html
2. Try to use preallocated buffers (there's a general use buffer called _destBuff in rfb.js, and also a couple buffer for palettes), and don't create single-use objects if you don't have to (GC was/is one of the biggest slowdowns).
3. Try not to introduce any significant performance decreases (make a recording, and then compare with standard noVNC and/or noVNC before your changes)
4. See if you can get any of the canvas functions to do heavy lifting for you in terms of decoding data

I'd love to have the benefit of your wisdom

I'm not sure if I'd go so far as to call it "wisdom" :wink:

Let me know if you have any questions, though.

[phil21]

@kelleyk We've (by we, I mean largely @alantwentyseven) been working on a very similar patch for a number of months, but unfortunately stopped checking this thread so missed the great work you've completed.

We have fully functional/tested Dell iDrac (7 and 8) support working, and are able to provide a large variety of Supermicro testbed hardware if nothing else. We've been working on reverse engineering the X9/X10 protocols, but it appears you've made far more progress than we have.

What is the best way to contact you out of band? I would love to work together and get these patches combined and (eventually) merged into master where we've put a lot of effort to ensure it's maintainable long-term.

[kelleyk]

@phil21 The username and domain portion of my email address are both the same as my GitHub username, and the TLD is ".net". Happy to exchange other contact info over email.

@DirectXMan12 Okay, thanks -- those tests pass in Chrome.

Re: your point (3): performance (for this video format) won't be a goal until I have things working at all, but hopefully it is acceptable in the short-term* if there's no impact (as there shouldn't be) for any other formats.

While I will try hard not to do anything stupid (massive copying around of data, etc.), this seems like a reasonable place to start because (touching on your point (4)) I don't think that handing data off to the Canvas/etc. is going to be possible. A "whole JPEG" never appears; unless there's some way to reach inside the JPEG decoder and directly call the various subroutines, I think we're doing this the good, old-fashioned way.

*I have a performance engineering background... so once we've got the thing working at all, I bet I can make it go fast enough to not bother anyone.

[DirectXMan12]

Re: your point (3): performance (for this video format) won't be a goal until I have things working at all, but hopefully it is acceptable in the short-term* if there's no impact (as there shouldn't be) for any other formats.

Yeah, the reason that I brought this up is that the existing implementation's color conversion code causes a hit to performance (not a huge one, but a noticeable one), even when there's no conversion taking place. I'd like to eventually figure out how to minimize that for the non-iKVM cases, but that's another story.

A "whole JPEG" never appears; unless there's some way to reach inside the JPEG decoder and directly call the various subroutines, I think we're doing this the good, old-fashioned way.

Yeah, that's tough. We (the noVNC team) have been toying around with the idea of using asm.js to speed up performance, but most of our code that needs to be fast would need a big rewrite to make work with asm.js. It might be worth considering implementing the various decoding pieces in an asm.js friendly way, so that it's trivial to convert them over once you're done with the initial implementation (if you're going to be doing any "heavy-lifting" in the javascript, it might yield decent performance gains).

*I have a performance engineering background... so once we've got the thing working at all, I bet I can make it go fast enough to not bother anyone.

Sounds good! If you happen to have some free time hanging around, at some point, I'd love to have someone else take a look at the noVNC code in general for performance as well (I did a big refactor a while ago that increase performance significantly, but it would be nice to have someone with a performance background take a look and see if there's anything else obvious worth doing).

[kelleyk]

Success! I've got 0x57 support working.

There should not be any performance impact when not using it, but its performance is currently very poor. I've made some of the most obvious performance improvements (e.g. there is no memory allocation happening except up front, avoidable copying has been eliminated, etc.). Using the render queue mechanism is probably the next thing to try.

I did take a peek at asm.js, and my implementation shouldn't require any horrible re-working in order to be converted if that's something that needs to happen eventually.

[kelleyk]

It's very rough, but I would love field reports: https://github.com/kelleyk/noVNC/tree/ast2100-support

There's a text file ("ast2100-notes.txt") explaining a few things, including the fact that I'm aware that image quality is not quite perfect yet. (I'm hoping to iron this out in the next couple of days and push a fix. Seems like there's a rounding issue somewhere in the IDCT implementation.)

You can change video quality using the atenChangeVideoSettings() function from the console; atenChangeVideoSettings(0x0, 0x0, 422) would be the worst quality and with chroma subsampling, whereas atenChangeVideoSettings(0xB, 0xB, 444) would be the best quality and without chroma subsampling.

[subtato]

Just ran across this part of the project a couple days ago, just wanted to say thanks for the work you (and everyone else) have put into this -- you're going to be saving my team and I a lot of headaches with our internal management... :) Supermicro's finicky Java client really is a nightmare.

Went ahead and tried this on one of our servers, an X10SLD-F (within a SYS-5038ML-H8TRF chassis, firmware version 01.07), from Dallas to a server in Phoenix -- was pleasantly surprised how well it's working thus far: screenshot of ATEN iKVM over noVNC

I don't have a lot of time to test more at the very moment, but I'll circle back around later tonight or tomorrow after reading through ast2100-notes.txt.