Screen resolution problem

[tomsealy]

Hello, I am working with the FOGProject and we are currently getting ready to start on our new client which will include remote control of our clients. I have always used OSXvnc and love how fast and clean it is. I have recently started to work with your code and have come accross two issues. The first which I have fixed was an issue with initial rfb screen grab (rfbScreenInit). The second issue is with the screen size on the client. I was just wondering if you had any pointers to get this resolved. I will be happy to work with you to get this resolved. I should also state this is only an issue with Macs with Retina displays. Other than that your code works great!!

Thanks,

Ton

[stweil]

Hi,

the distorted screen when mirroring Retina displays is indeed the most important thing which I want to get solved. My knowledge of the Mac API is not good enough to find a quick solution, and up to now I did not find the time to dig deeper into this problem.

If you have any code improvements, don't hesitate to send a pull request on GitHub, so I can integrate them into the code.

Kind regards, Stefan

[tomsealy]

So I believe I have the problem identified. The XServer is calculating the wrong resolution. It is because of the HiDPI used in retina displays. So for example. The screen resolution is stated as 1280x 800 but in actuality the resolution is almost twice that. So I am unsure about how to achieve this. Since you know this code better than anyone, how would I go about either scaling the server or doubling the reported desktop size to the client?

[tomsealy]

So the code changes I have made are confirmed 100% working on non retina displays 10.7-10.11 beta. The only issue I still have and am trying to work out is for retina displays. I am aimlessly attacking this problem and as I am not familiar with the code. Again I am not trying to be a bother and I do apologize if I am coming of as a pain. I can determine if the screen is retina but I am unable to tell the vnc server to either double the resolution or whatever. I think this fix should not be too hard for someone like yourself.

Thanks in advance, Tom

[stweil]

How do you determine whether it is a retina screen or not? A code example would be helpful. And which code changes did you make?

Thanks, Stefan

[tomsealy]

Hello and thank you for the quick reply

All below changes are in OSXvnc server/main.c

To detect Retina Display I added:

float displayScale = 1;
if ([[NSScreen mainScreen] respondsToSelector:@selector(backingScaleFactor)]) {
    NSScreen *myScreen = [NSScreen mainScreen];
    float displayScale = [myScreen backingScaleFactor];
    if (displayScale > 1){
        rfbLog("Detected HiDPI Display with scaling factor of %f",displayScale);
    }
}

To fix the garbled image for retina display make this change which I found online: replace

if (floor(NSAppKitVersionNumber) > floor(NSAppKitVersionNumber10_6)) {
    rfbScreen.paddedWidthInBytes = rfbScreen.width*rfbScreen.bitsPerPixel/8;
} else {
    rfbScreen.paddedWidthInBytes = CGDisplayBytesPerRow(displayID);
}

with:

if (floor(NSAppKitVersionNumber) > floor(NSAppKitVersionNumber10_6)) {
    CGImageRef imageRef = CGDisplayCreateImage(displayID);
    rfbScreen.paddedWidthInBytes = CGImageGetBytesPerRow(imageRef);
    if (imageRef != NULL)
        CGImageRelease(imageRef); 
} else {
    rfbScreen.paddedWidthInBytes = CGDisplayBytesPerRow(displayID);
}

Wow that markdown looks horrible ;-) Thanks, Tom

(stweil: Fixed markdown)

[tomsealy]

So apparently the issue indeed lies in X11 and Xserver. They do not support HiDPI and do not plan on doing so. I would say that if OSXvnc is to keep on the glorious path of the past it will need to switch over to GTK or a true Cocoa rendering. Correct me if I am wrong but the x11/xserver code does not really get the pixel data but rather keep track of banded rects to in which the pixels are gathered from.

I have tried my best to study the code and learn the logic of the server. So my statements could/can be very much unfounded

I have been able to tinker around with getting pixel data from UI Images and scaling correctly for retina and non retina displays but without knowing what exactly is going on with some of your code I am unsure how much I can help. Please let me know what I can do to make this work, because as I said in the past, OSXvnc server is the best vnc server I have ever found for os x and believe that it will continue to do so if we can get this sorted out. However without Retina support I am concerned as Apple has made plans for all machine to be using a HiDPI version on all of their machines in the very near future

Regards,

Tom

[tomsealy]

Well making good head way. Got resolution to display correctly however it is inverting colors.....grrrr but hey at least got resolution working, will keep you updated

Tom

[tomsealy]

Got it!!!!!!!!!!!!!!!!! Whoot Whoot. Only change is in main.c. Can share but need to test some more but looks good. Do we need to support below 10.6?

So So happy, Tom

[stweil]

Congratulation! I am waiting for your pull request (please use a commit message similar to the other ones with a Signed-off-by, see https://github.com/stweil/OSXvnc/commits/master).

I think we can even drop support for 10.6.

Cheers, Stefan

[tomsealy]

I will get started cleaning up code tonight and hopefully have something for you very soon. Since 10.6 doesn't support retina, I was able to mess with libraries so that we won't have to remove "legacy" support.

Side topic: For Fog I was going to add a password argument to the vnc server, any objections to adding that to my commit as well? That way a single binary could handle all of our needs (no need for storepassword). I can also leave it out of usage function to keep it hidden if you like.

Thanks, Tom

[stweil]

I'd prefer a separate commit for the password argument, but it is sufficient to make a single pull request for all of your commits. Please add the new argument also to the usage function - I see no advantage in a hidden argument.

Stefan

[tomsealy]

Pull Request Pushed!! Please look at main.c auth.c and rfb.h. These files are up to date with yours as I did a pull before pushing. My SDK threw some errors while building the gui so again I would just focus on my changes but it has been tested on 10.9 - 10.11 both retina and not.

Hope this helps

Tom

[tomsealy]

The fix has been pushed to the master branch

Thanks for the help, Tom