search

joncampbell123 / dosbox-x

DOSBox-X fork of the DOSBox project
GNU General Public License v2.0
2.83k stars 383 forks source link

Pixel-perfect display #2167

Open ant-222 opened 3 years ago

ant-222 commented 3 years ago

This is a proposal to implement pixel-perfect scaling in DOSBox-X, similar to the way it was done in my original patch. To put it short, it calculates such integral vertical and horizontal scaling factors as best approximate the target pixel aspect ratio. The result is sharp, uniform pixels with correct proportions. Here is my fork of DOSBox-X with pixel-perfect scaling implemented as output=openglpp. Make sure to set scaler=none and doublescan=false before testing. The algorithm tries to fit the image into 4:3 with aspect=true and keeps square pixels with aspect=false.

Questions for discussion:

  1. @Wengier proposes that pixel-perfect mode should work on top of scalers (e.g. tv2x). Is it a good idea?
  2. @Wengier proposes to consider implementing pixel-perfect mode as a usual scaler (e.g. tv2x). Is is technically possible?
  3. If neither of the above, then shall I force scaler=none in case of openglpp, as I did in the original patch and in the implementation for dosbox-staging?
  4. If I shall, then how to pass that information to RENDER_Reset() in src/gui/render.cpp?
    I have an idea to do it from GFX_GetBestMode() via a new GFX_NOSCALING flag. But since GFX_GetBestMode() is currently invoked for SDL1 only, I shall have to change that, ignoring the other flags in case of SDL2.
ant-222 commented 3 years ago

I will soon prepare my solution for disabling other scalers in pixel-perfect mode.

ant-222 commented 3 years ago

The first working version is committed. Please, test the current implentation.

ant-222 commented 3 years ago

I have just made the window match the image at minimal scale. The question remains what to do with the doublescan option. Although it does not break pixelperfect mode, it limits its ability to approximate the target pixel proportions. Must we have it enabled by default? When does it make a difference? I think it would be a viable decision just to turn it off in the default configuration. What do you think?

Wengier commented 3 years ago

@ant-222 Thanks for the pixel-perfect patch! I have tried with output=openglpp and the features certainly works in DOSBox-X. I really appreciate it.

I think it is always better for the output to work with existing scalers. However, I am not proposing that pixel-perfect mode to work on top of other scalers (normal2x, xbrz, etc). Instead, the other scalers may work on top of the openglpp output, similar to how they work on top of the surface output etc, if this can be done. For example, if I maximize the DOSBox-X window the actual image can be pretty small within the DOSBox-X window, like this:

image

But if another scaler (like normal2x) can be applied on top of this, the image will clearly be larger. The other scalers like xbrz are useful if they can work in combination too.

I suggest implementing pixel-perfect mode as a usual scaler only if this can be done easily. Otherwise, just go with what we already have.

For passing information to RENDER_Reset() in case you need to do so, I think it can be done with for example the usage of a global variable. But there may be other solutions too depending on the actual code.

I am not quite familiar with the doublescan option. @joncampbell123 probably knows how this option works much better than me.

In any case, thank you so much for the pixel-perfect implementation for DOSBox-X. @Banjo-Oz probably wants to check this out too since you apparently want the pixel-perfect scaling in DOSBox-X, and here it is.

Banjo-Oz commented 3 years ago

Wow! Huge thanks to ant-222 for this! A really important element of any fork for me. :) Excited to try it once it's available! Really, really appreciate this being ported to X.

ant-222 commented 3 years ago

@Weigner and I are discussing his problem by mail, so here is a short report: the pixel-perfect mode works correctly, but the High-DPI awareness mechanism in Windows destroys it by lying about the actual window size and display dimensions. It may not happen on other OSes. Setting dpi aware=true fixes the problem, but predictably makes the initial DOSBox-X window too small, because of windowresolution=original means a pixel-to-pixel scale. I proposed setting a larger windowresolution to have a comfortable window by default, but that breaks window resizing and maximisation.

Anyway, using DOSBox-X on a Windows high-DPI device with dpi aware=false is a way to nowhere because it causes Windows quietly to resize the window using primitive bilinear interpolation, and to lie to programs by quietly recalculating all pixel dimensions in WinAPI calls. In this situation, no accurate scaling whatsoever is possible.

Since I use Widnows XP (which fortunately has no idea about high-DPI) and a 1600x1200 display, I cannot fix this problem without others' help.

Banjo-Oz commented 3 years ago

@ant-222 I use Windows 7 x64 (and have access to Windows 7 x32). Is there anything at all I can do to help? I sadly know zero about coding DOSBox but am happy to test anything and give any feedback that might help. No probs if not, just wanted to offer.

ant-222 commented 3 years ago

@Banjo-Oz :

Wow! Huge thanks to ant-222 for this! A really important element of any fork for me. :) Excited to try it once it's available! Really, really appreciate this being ported to X.

Thank you. If you have Windows, then please try this build of DOSBox-X with pixel-perfect scaling. Neglect not to persuse dbx-winexe.txt first :-) I hereby invite every body with Windows XP or later to alpha-test my pre-built implementation of pixel-perfect scaling for DOSBox-X. If you have another OS or wish to build it from source for another reason, help yourself to my fork.

P.S.: can you please delete one of your duplicated messages above? P.P.S.: it's the same thing that happened with me while posting from an old browser: attempt to edit was perceived as a new message.

ant-222 commented 3 years ago

@Wengier

I think it is always better for the output to work with existing scalers.

Yes, for reasons of generality, but notice that openglnb, openglhq, and openglpp here proposed, are not really output types, but different scalers specific to—and implemented inside—the opengl output. Those output-specific scalers are applied after DOSBox has processed the source display with whatever scaler is active (e.g. normal2x or xBRZ).

However, I am not proposing that pixel-perfect mode to work on top of other scalers (normal2x, xbrz, etc). Instead, the other scalers may work on top of the openglpp output, similar to how they work on top of the surface output etc, if this can be done.

As I explained to you by e-mail, this is impossible because the output method receives an image already processed by those built-in scalers. Furthermore, I think it would be useless, for normalNx scalers will only thwart the ability to approximate the required PAR by multiplying the dimensions of the input image, whereas advanced scalers such as xBRZ can upscale the image to any window or display size without the help of pixel-perfect scaling.

I simply do not see a case where pixel-perfect scaling could be useful in conjunction with another scaler. You mentioned it as a method of having a sufficiently large window on hi-res monitors with dpi aware=true, but that would be just a hack and an abuse of a scaler. A good solution is to specify a minimum initial window size as I propose here. By the way, something like windowresolution=50% (as I think dosbox-staging supports) would give a great device-independent sane default.

I suggest implementing pixel-perfect mode as a usual scaler only if this can be done easily. Otherwise, just go with what we already have.

It certainly cannot be done easily. I am not even sure it is at all possible with the current architecture, because this usual scaler would have to dictate the image dimensions to the output method...

For passing information to RENDER_Reset() in case you need to do so, I think it can be done with for example the usage of a global variable. But there may be other solutions too depending on the actual code.

I have done it via sdl.desktop.isperfect—a flag in the global structure that can be used for pixel-perfect scaling in other output types, too.

I am not quite familiar with the doublescan option. @joncampbell123 probably knows how this option works much better than me.

OK. It works as documented for me, doubling the vertical dimension of the source image. Whereas a 320x200 image can be upscaled by a factor of 3 to 960x600, with dobulescan=true it becomes 320x400 and would require a fractional vertical scale of 1.5. @joncampbell123 , can you please tell me whether doublescan is worth using by default, and whether you can turn it off if/when you accept my implementation of pixel perfect mode? Or should you prefer that I bypass doublescan programmatcially in pixel-perfect mode?

In any case, thank you so much for the pixel-perfect implementation for DOSBox-X.

I sure do. It is very strange that DOSBox and its forks should support so fancy image processing, including interpolative scaling, xBRZ, and even shaders, but fail to provide the most basic and obvious method of displaying source pixels as regular and sharp rectangles with proportions approximating the required pixel aspect ratio (PAR)...

Can somebody please post a lossless screenshot showing the sharp shader at work in a window whose dimensions are not multiples of the resolution of the emulated graphical mode? Shaders do not seem to work on my Windows XP...

ant-222 commented 3 years ago

I wrote:

Can somebody please post a lossless screenshot showing the sharp shader at work in a window whose dimensions are not multiples of the resolution of the emulated graphical mode? Shaders do not seem to work on my Windows XP...

Hmmm. glshader=sharp works on my PC. Although this shader is documented as providing pixel-perfect magnification, it looks rather blurry on my side with:

output   = opengl
glshader = sharp

See for yourselves:

I can tell it is neither pixel-perfect nor sharp even if you wake me up in the dead of night to ask it.

Banjo-Oz commented 3 years ago

@Banjo-Oz :

Wow! Huge thanks to ant-222 for this! A really important element of any fork for me. :) Excited to try it once it's available! Really, really appreciate this being ported to X.

Thank you. If you have Windows, then please try this build of DOSBox-X with pixel-perfect scaling. Neglect not to persuse dbx-winexe.txt first :-) I hereby invite every body with Windows XP or later to alpha-test my pre-built implementation of pixel-perfect scaling for DOSBox-X. If you have another OS or wish to build it from source for another reason, help yourself to my fork.

P.S.: can you please delete one of your duplicated messages above? P.P.S.: it's the same thing that happened with me while posting from an old browser: attempt to edit was perceived as a new message.

Sorry, the "edit" box for Github posts no longer loads for some (probably Google!) reason on Pale Moon (my browser), and trying to bypass that just posts a new message. I managed to fix the issue with "https://github.com/JustOff/github-wc-polyfill/releases" for now (in case anyone else is having the same issue).

Will give the build a test this weekend! Thanks again!

arromdee commented 3 years ago

Alternate possibility. Semi-pixel-perfect scaling: Pick an amount which scales the image by as much as possible such that one dimension is scaled by an exact amount, while the other dimension is scaled by an inexact amount that keeps the aspect ratio correct.

ant-222 commented 3 years ago

@arromdee, this issue is not about semi-pixel-perfect scaling. My personal opinion is that it is not much better than any other imperfect scaling: either it is perfect or it isn't :-) I will not consider implementing it until true pixel-perfect mode is ready. If you want to have this scaling algorithm in DOSBox-X, then I ask you to please create another issue for it (with a link from here).

dodleh commented 3 years ago

I would like to add my opinion, and hope it helps. I have seen many scaling algorithms and the biggest problem with them is that they cannot be applied universally. For instance, to start with the current implementation of Pixel-Perfect Scaling (as in Dosbox ECE), it is horrible for text. When you run Windows as guest, it is noticeable. The classic scalers such as hq2x, normal2x and others are very slow, a big issue when you need to emulate fast video as in the above case (in short, increases lag). The classic bi-linear filter is a mediocre solution. It works, everyone got used to it, and it is damn fast (hardware implementation). There is however, a small issue: the accuracy is left to the GPU driver.

I think that the best approach would be a contextual one, managed automatically, if desired. If the required resolution can be integer-scaled, so be it. If it is not integer, and the actual to required resolution ratio is small, attempt to use sharp bicubic scaling (reasonable quality/speed tradeoff). If the required resolution is very low compared with actual resolution, then use integer pixel scaling methods (so-called pixel-perfect). The goal is simple, reduce the graphics overhead. In low DOS resolutions such as 320x2xx the impact on emulation is minimal even if you process pixels by software, if the resolution required is your typical Windows host screen (until roughly 1024x768), use sharp bi-cubic, but not very sharp to avoid edge fringes. I know about lanczos, it is good, but it isn't quite fast, so I left it out of the discussion, albeit it is good for images and slightly better sometimes than bi-cubic.

Wengier commented 3 years ago

@dodleh Thanks for the suggestions! @ant-222 might want to take a look at it.

Meanwhile, the updated code with the pixel-perfect output (without DPI scaling) is now available in #2203 and an updated Windows binary is also available from:

Set "output=openglpp" or select it from the menu ("Video" => "Output" => "OpenGL perfect") to use it.

dodleh commented 3 years ago

I made an attempt at using openglpp, but I am sure I am missing something in the DosBox-X configuration (it crashes right before dosbox boots, probably when it sets the display context), or OpenGL 3.0 compatibility is required. I remember not incurring any issue with DosBox ECE (although I still use a classic videocard, the "screaming fast" Intel Series 4 Express GPU with a "dazzling" Intel Core 2 Duo P8600 at 2.40Ghz).

As a side note, I think that the TTF (TruetypeFont) output has some merit in that there were videocards that made use of a special scaling technique for the system font to display text-mode with no aliasing (NeoMagic256 on some IBM Thinkpads comes to mind). The result was limited in that 25 lines text mode screen ended up with narrow characters, but the potential is there. The only major issue seems to be in achieving (with the modern technique of TrueType), a reasonably pleasant output and good support for just about all special characters so that you can draw nice text-mode screens (Norton Commander comes to mind).

Wengier commented 3 years ago

@dodleh If you use Windows, can you try the following package (which defaults to output=openglpp) and see if it works on your system?

If not, you can try to run DOSBox-X with the following command-line and see if it works:

dosbox-x -defaultconf -set output=openglpp

If it does not work still, you can change output=openglpp to output=opengl and see if it helps. After all, "output=openglpp" and "output=opengl" should theoretically have the same requirements for OpenGL. If both do not work then it is likely a compatibility issue with your OpenGL version or so.

In any case, thanks a lot for your testing and feedback!

dodleh commented 3 years ago

Thank you for your suggestions but I am quite certain that the problem is due to a GPU OpenGL incompatibility on my side. Any configuration setting ends up with the same crashy result. For the moment i will see what can still be done (tested) with Direct3D.

On a side note, can you please tell me why pixel-perfect rendering cannot be done through a Direct2D/Direct3D pipeline? I expect that the whole screen is actually a single texture that can be scaled according to operations such as GL_NEAREST_MIPMAP_NEAREST with a similar corespondent in Direct3D (i do not know), and shown with a scale of 1:1 after the texture is being processed by the internal algorithm (cpu interpolated behind the scenes) then directly written to the surface (replacing the old one). Of course, my approach may be horrible in terms of performance, it is just what I can guess with my limited understanding of some graphics possibilities.

Wengier commented 3 years ago

@dodleh I think pixel-perfect rendering with Direct3D can technically be done (of course). But it will need be implemented as another output option then. For example, for the OpenGL outputs, there are already opengl, openglnb, and openglpp.

ant-222 commented 3 years ago

@dodleh

For instance, to start with the current implementation of Pixel-Perfect Scaling (as in Dosbox ECE), it is horrible for text.

Why? What is the exact problem with my original pixel-perfect scaling in DOSBox ECE? I used it with text all the time: command-line, Norton Commander, Turbo Pascal, VDE editor—and had no complaints.

The classic bi-linear filter is a mediocre solution. It works, everyone got used to it, and it is damn fast (hardware implementation). There is however, a small issue: the accuracy is left to the GPU driver.

The problem with bi-linear interpolation is that it is blurry, whereas true retro-maniacs want their pixels sharp :-)

I think that the best approach would be a contextual one, managed automatically, if desired. If the required resolution can be integer-scaled, so be it. If it is not integer, and the actual to required resolution ratio is small, attempt to use sharp bicubic scaling (reasonable quality/speed tradeoff). If the required resolution is very low compared with actual resolution, then use integer pixel scaling methods (so-called pixel-perfect).

Although it is technically possible, it requires some work. Why not set up individual DOSBox configuration for each game and program? That would be much easier that having DOSBox adjust on-the-fly.

The goal is simple, reduce the graphics overhead. In low DOS resolutions such as 320x2xx the impact on emulation is minimal even if you process pixels by software, if the resolution required is your typical Windows host screen (until roughly 1024x768), use sharp bi-cubic, but not very sharp to avoid edge fringes.

I don't think it will work, because the CPU load will depend mostly on the size of the host display, rather than the emulated display.

Bicubic and Lanczosh are both quite slow. The are good for photographic images, but not for low-res computer displays. If you really need them, consider a pixel shader.

ant-222 commented 3 years ago

@dodleh

Thank you for your suggestions but I am quite certain that the problem is due to a GPU OpenGL incompatibility on my side. Any configuration setting ends up with the same crashy result.

Does it crash with any OpenGL output or only with openglpp? If the latter,—there must be a bug in the pixel-perfect algorithm. Does openglnb work for you with the official DOSBox?

On a side note, can you please tell me why pixel-perfect rendering cannot be done through a Direct2D/Direct3D pipeline? I expect that the whole screen is actually a single texture that can be scaled according to operations such as GL_NEAREST_MIPMAP_NEAREST with a similar corespondent in Direct3D (i do not know), and shown with a scale of 1:1 after the texture is being processed by the internal algorithm (cpu interpolated behind the scenes) then directly written to the surface (replacing the old one). Of course, my approach may be horrible in terms of performance, it is just what I can guess with my limited understanding of some graphics possibilities.

That answers your question: OpenGL supports nearest-neighbor interpolation (opengnb in the original DOSBox), but I am not sure about Direct2D/3D. In any case, there is no direct3dnb output in DOSBox. Without such an option, we are limited to software, CPU-based upscaling, which is may be quite slow. I used to support this method as surfacepp in my original Pixel-perfect patch, and it was marginally tolerable. But with new high-resolution monitors, its performance will decrease proportionally to megapixels. Is it worth it?

dodleh commented 3 years ago

Why? What is the exact problem with my original pixel-perfect scaling in DOSBox ECE? I used it with text all the time: command-line, Norton Commander, Turbo Pascal, VDE editor—and had no complaints.

Correct to some extent. You see, in DOS EGA text modes (the above examples on 25 or 43 lines), since it is low resolution, scaling issues are not as obvious. On the other hand, if you try to scale a Windows 3.1 800x600 screen that contains a text window, you will immediately notice that fonts are not rendered as great on a 1440x900 or 1920x1080 desktop (with pixel perfect scaling it is impossible and near-pixel perfect scaling is more annoying than a bilinear-filtered text, except the overall inherent blur). Plus, if you want a decent look, you also need 4:3 aspect correction on the 16:10 or 16:9 screen so pp or npp is not good enough for this scenario. The problem with bi-linear interpolation is that it is blurry, whereas true retro-maniacs want their pixels sharp :-)

The problem is that Windows and GPU drivers implementation is even worse (less accurate) than it should be. I consider sharper pixel look to be better if scaling is accurate enough, if not, blurry is acceptable. I made a comparison of the output between Dosbox running on Linux and Windows, similar filters, and it is slightly worse quality-wise in Windows. Although it is technically possible, it requires some work. Why not set up individual DOSBox configuration for each game and program? That would be much easier that having DOSBox adjust on-the-fly.

Not quite. I know that some users do run DOSBox with their game and that is that. If you want a real feel of DOS, you need to actually work with that system so having versatility is key. Moreover, if you run Windows 3.x or 9x you also need DOS and multiple resolution support so a single filtering method (scaler) simple doesn't cut it as you do not have an infinite desktop display resolution. Plus, aspect ratio correction... I don't think it will work, because the CPU load will depend mostly on the size of the host display, rather than the emulated display.

Not quite. When you have to process 320x200x8 bit color it is much faster than 800x600x16 bit color so your algorithm, if it is CPU dependent, will run quicker, so less lag. Use more demanding scaling algorithms were it matters, on lower resolution. If scaling is bilinear it will look a lot worse the larger the screen. Bicubic and Lanczosh are both quite slow. The are good for photographic images, but not for low-res computer displays. If you really need them, consider a pixel shader.

Bicubic is much faster since it can be hardware accelerated, so it is quite different. Pixel Shaders, as far as I noticed, were quite slow on DosBox SVN-Daum and i do not remember this processing path to be that great (could be due to SDL, I don't know). I remember doing a Bilinear/Bicubic/Lanczos cross-comparison and even Bilinear was slow running as a pixel shader.

ant-222 commented 3 years ago

@dodleh

When you have to process 320x200x8 bit color it is much faster than 800x600x16 bit color so your algorithm, if it is CPU dependent, will run quicker, so less lag.

I tell you: the performance of my pixel-perfect algorithm is determined solely by the output resolution. If you don't believe me, you can test it yourself: here it is. It may be otherwise, however, with other algorithms, if the bottleneck is not output but internal calculations. But pixel-perfect scaling has no internal calculations.

Bicubic is much faster since it can be hardware accelerated, so it is quite different. Pixel Shaders, as far as I noticed, were quite slow on DosBox SVN-Daum and i do not remember this processing path to be that great (could be due to SDL, I don't know). I remember doing a Bilinear/Bicubic/Lanczos cross-comparison and even Bilinear was slow running as a pixel shader.

I don't understand that. Shaders should be the most efficient way for implementing a custom scaler. I suggest you give them a try again. GPU is much more efficient than CPU at such tasks. Shader implementations exist for both bicubic and Lanczos interpolation.

I have no specific objections to your other points, but we diverge from the topic of this issue.

dodleh commented 3 years ago

I understand the implementation in your case, in this context. The type of computations I remember for the scalers I used were more dependent on passes (time spent on each processing stage) and initial frame size. If Pixel-perfect scaling is agnostic in this sense, very well.

In theory, shaders should be fast, I agree with you. However, texture scalers using LINEAR_MIPMAP_LINEAR seem to always compute faster than some pixel-shader algorithm using the same linear scaler on a specific texture. I may be wrong, though. for specific newer cards or less obvious combinations. However, as far as I remember, additionally, different GPU implementations led to slight frame time variations based on calculus precision on AMD/NVidia hardware so not everything is as generic as it should.

To get back on the subject. What is the best way to implement a scaler for this use scenario, in your opinion: