Dead link

[markwoodhall]

Hi, sorry for creating an issue for this but I am interested in the Upscaled Blue Marble spectrum, I'd like to have a look at it but the link is dead, is there somewhere I can get it from?

Thanks

[0x09]

@markwoodhall the site is only accessible with ipv6 atm, I can just attach it here.

[markwoodhall]

Thank you so much. I realise this probably isn't the place but I'm trying to learn a bit more about detecting upscaling in absolute value spectrums. Is there a good place to read more?

I'm particularly interested in what I can learn about images like these.

What does the spectrum tell me in this instance? From an uninformed point of view, the diagonal from the top left looks odd.

Appreciate you may not want to answer this, and thanks for the image.

[0x09]

@markwoodhall if we mask out everything but this feature and invert the spectrum we can see that it's contributing to the many diagonal lines in the image, especially the part at the top left.

The effects of resizing really depend on the filter used and what post processing has been done, but one thing common to all conventional filters* is some mirroring of the spectrum veritcally/diagonally. For example, if I resize your image by half and then upscale again with imagemagick's default lanczos filter, the spectrum shows this very clearly. (This is boosted a bit with spec -G to make it easier to see.)

It isn't really the main focus but this article provides some explanation for why this is so starting about half way down: https://bartwronski.com/2021/02/15/bilinear-down-upsampling-pixel-grids-and-that-half-pixel-offset/

*A sinc filter with a high enough window wouldn't since all frequencies visible in the spectrogram that are higher than the original resolution would become zero.

[markwoodhall]

Thank you so much for replying. This is really great, and I appreciate you taking the time.

I'm very new to this but just came across a problem that triggered my interest.

I realise it's quite specific but are you aware of any methods that allow detection of upsampling using methods like TAA, e.g. as in Unreal Engine https://docs.unrealengine.com/en-US/RenderingAndGraphics/ScreenPercentage/index.html

Is there anything useful we can gain by generating spectrums for images upsampled in this way?

Fow what it's worth, I'm writing something to detect native framerate and resolution of games streamed by Google Stadia. Frame rate is really rather straightforward, but resolution is very difficult.

[0x09]

@markwoodhall interestingly resdet is able to detect upsampling in both of the example images on that page (upscaling with and without TAA) though it's not as visible in the spectrum with the latter. The fact that resdet even works with the TAA example is a bit surprising to me since according to that doc the engine is only partly rendering at lower resolution, with additional rendering and filtering happening after upscaling to create the final image. I expect results are going to vary significantly from scene to scene depending on how heavy this additional rendering is, but I can't say specifically what to look for as I don't have much experience with content like this. Probably the biggest challenge with TAA is the fact that this includes UI elements being rendered in at full resolution, which will introduce a lot of genuine high frequency info to the image, and make it difficult for algorithms like resdet to find matches or to visually separate these from the upscaled "pseduo" high frequency content. You'd definitely be better off trying to avoid scenes with UI where possible when doing this kind of detection.

[markwoodhall]

That's great, thanks for looking again!

Which of resdets current methods is best for detection of images produced with TAA?

Is there anywhere I can read more detail about how changing -r impacts detection?

Thanks again for being so helpful.

On Sun, 18 Apr 2021, 18:42 tab, @.***> wrote:

@markwoodhall https://github.com/markwoodhall interestingly resdet is able to detect upsampling in both of the example images on that page (upscaling with and without TAA) though it's not as visible in the spectrum with the latter. This is a bit surprising to me since according to that doc with TAA the engine is only partly rendering at lower resolution, with additional rendering and filtering happening after upscaling to create the final image. I expect results are going to vary significantly from scene to scene depending on how heavy this additional rendering is, but I can't say specifically what to look for as I don't have much experience with content like this. Probably the biggest challenge with TAA is the fact that this includes UI elements being rendered in at full resolution, which will introduce a lot of genuine high frequency info to the image, and make it difficult for algorithms like resdet to find matches or to visually separate these from the upscaled "pseduo" high frequency content. You'd definitely be better off trying to avoid scenes with UI where possible when doing this kind of detection.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/0x09/resdet/issues/11#issuecomment-822030164, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAG2B5IN2ODCZUAXH67JWU3TJMKXFANCNFSM425RLWVQ .

[0x09]

Is there anywhere I can read more detail about how changing -r impacts detection?

The library documentation here explains this parameter in a little more detail.

Range of coefficients to consider when looking for inversions. Lower values are faster, but may return many more misidentified results. The default is currently 12, with reasonable values between 8-32

resdet's methods do this slightly differently but they all just try to identify rows/columns in the spectrum with neighboring values that display this mirroring effect. This param increases the number of rows or columns around the resolution being guessed that are taken into consideration. Filters (both the actual resampling filters and regular image filters), compression, etc will destroy some of this and so requiring too many rows/columns to match will give overly pessimistic results. But not taking enough into account will give false positives due to chance, so there's some balance there. I'm not sure which method would give the best results for this so feel free to try different combinations.

[markwoodhall]

Thank you so much for taking the time to help. I'll close this now.