search

gnif / LookingGlass

An extremely low latency KVMFR (KVM FrameRelay) implementation for guests with VGA PCI Passthrough.
GNU General Public License v2.0
4.6k stars 254 forks source link

NvFBC host stops producing frames on modeset #912

Open Xyene opened 2 years ago

Xyene commented 2 years ago

I'm filing this for posterity as it's technically a bug, but it's pretty low-impact. This occurs with B5, but I suspect it's been around forever.

Guest desktop running at 2560x1440.

image

Host desktop running at 2560x1440.

xyene@lynwood:~$ swaymsg -t get_outputs
Output HDMI-A-1 'Goldstar Company Ltd LG TV SSCR 0x00000101' (focused)
  Current mode: 2560x1440 @ 119.998 Hz

Game (specifically, osu!) set to modeset to 3840x2160.

image

osu! config This goes in the .cfg file in `%userprofile%\AppData\Local\osu!`. ``` BeatmapDirectory = Songs VolumeUniversal = 30 VolumeEffect = 80 VolumeMusic = 80 AllowPublicInvites = 1 AutoChatHide = 1 AutomaticDownload = 1 AutomaticDownloadNoVideo = 0 BlockNonFriendPM = 0 Shaders = 0 BloomSoftening = 0 BossKeyFirstActivation = 1 ChatAudibleHighlight = 1 ChatChannels = #osu ChatFilter = 0 ChatHighlightName = 1 ChatMessageNotification = 1 ChatLastChannel = #osu ChatRemoveForeign = 0 ChatSortMode = Rank ComboBurst = 0 ComboFire = 0 ComboFireHeight = 3 ConfirmExit = 0 AutoSendNowPlaying = 1 CursorSize = 0.99 AutomaticCursorSizing = 0 DimLevel = 80 IHateHavingFun = 0 Display = 1 DisplayCityLocation = 0 DiscordRichPresence = 1 DistanceSpacingEnabled = 1 EditorTip = 0 VideoEditor = 0 EditorDim = 0 EditorGrid = 0 EditorDefaultSkin = 0 EditorSnakingSliders = 1 EditorHitAnimations = 0 EditorFollowPoints = 1 EditorStacking = 1 ForceSliderRendering = 0 FpsCounter = 1 GuideTips = 1 CursorRipple = 0 HighlightWords = HighResolution = 0 HitLighting = 0 IgnoreBarline = 0 IgnoreBeatmapSamples = 0 IgnoreBeatmapSkins = 0 IgnoreList = KeyOverlay = 0 Language = en LastPlayMode = Osu AllowNowPlayingHighlights = 0 LastVersion = b20211225 LastVersionPermissionsFailed = b20200831.2 LoadSubmittedThread = 1 LobbyPlayMode = -1 ShowInterface = 1 ShowInterfaceDuringRelax = 0 LobbyShowExistingOnly = 0 LobbyShowFriendsOnly = 0 LobbyShowFull = 0 LobbyShowInProgress = 1 LobbyShowPassworded = 1 LogPrivateMessages = 0 LowResolution = 0 ManiaSpeed = 13 UsePerBeatmapManiaSpeed = 0 ManiaSpeedBPMScale = 0 MenuTip = 12 MouseDisableButtons = 1 MouseDisableWheel = 0 MouseSpeed = 0.99 Offset = 0 ScoreMeterScale = 1 DistanceSpacing = 0.8 EditorBeatDivisor = 1 EditorGridSize = 32 EditorGridSizeDesign = 32 Height = 2160 Width = 3840 HeightFullscreen = 2160 CustomFrameLimit = 200 WidthFullscreen = 3840 MsnIntegration = 0 MyPcSucks = 0 NotifyFriends = 1 NotifySubmittedThread = 1 PopupDuringGameplay = 1 ProgressBarType = Pie RankType = Top RefreshRate = 60 OverrideRefreshRate = 0 ScaleMode = WidescreenConservative ScoreboardVisible = 1 ScoreMeter = Error ScreenshotId = 0 MenuSnow = 0 MenuTriangles = 1 SongSelectThumbnails = 1 ScreenshotFormat = Jpg ShowReplayComments = 0 ShowSpectators = 1 ShowStoryboard = 0 Skin = Default SkinSamples = 1 SkipTablet = 0 SnakingSliders = 0 Tablet = 0 UpdatePending = 0 UserFilter = All UseSkinCursor = 0 SeasonalBackgrounds = Always UseTaikoSkin = 0 Video = 0 Wiimote = 0 YahooIntegration = 0 ForceFrameFlush = 0 DetectPerformanceIssues = 1 Fullscreen = 1 MenuMusic = 0 MenuVoice = 0 MenuParallax = 0 RawInput = 1 AbsoluteToOsuWindow = 0 ConfineMouse = Fullscreen ShowMenuTips = 0 HiddenShowFirstApproach = 1 ComboColourSliderBall = 1 AlternativeChatFont = 0 Username = DisplayStarsMaximum = 10 DisplayStarsMinimum = 0 AudioDevice = AudioCompatibility = 0 SavePassword = 0 SaveUsername = 0 Password = TreeSortMode = Difficulty TreeSortMode2 = Difficulty Letterboxing = 0 LetterboxPositionX = 0 LetterboxPositionY = 0 FrameSync = VSync ShowUnicode = 0 PermanentSongInfo = 0 Ticker = 0 CompatibilityContext = 0 CanForceOptimusCompatibility = 0 keyOsuLeft = Z keyOsuRight = X keyOsuSmoke = C keyFruitsDash = LeftShift keyFruitsLeft = Left keyFruitsRight = Right keyTaikoInnerLeft = X keyTaikoInnerRight = C keyTaikoOuterLeft = Z keyTaikoOuterRight = V keyPause = Escape keySkip = Space keyToggleScoreboard = Tab keyToggleChat = F8 keyToggleExtendedChat = F9 keyScreenshot = F12 keyIncreaseAudioOffset = OemPlus keyDecreaseAudioOffset = OemMinus keyQuickRetry = OemTilde keyIncreaseSpeed = F4 keyDecreaseSpeed = F3 keyToggleFrameLimiter = F7 keyVolumeIncrease = Up keyVolumeDecrease = Down keyDisableMouseButtons = F10 keyBossKey = Insert keySelectTool = D1 keyNormalTool = D2 keySliderTool = D3 keySpinnerTool = D4 keyNewComboToggle = Q keyWhistleToggle = W keyFinishToggle = E keyClapToggle = R keyGridSnapToggle = T keyDistSnapToggle = Y keyNoteLockToggle = L keyNudgeLeft = J keyNudgeRight = K keyHelpToggle = H keyJumpToBegin = Z keyPlayFromBegin = X keyAudioPause = C keyJumpToEnd = V keyGridChange = G keyTimingSection = None keyInheritingSection = None keyRemoveSection = None keyEasy = Q keyNoFail = W keyHalfTime = E keyHardRock = A keySuddenDeath = S keyDoubleTime = D keyHidden = F keyFlashlight = G keyRelax = Z keyAutopilot = X keySpunOut = C keyAuto = V keyScoreV2 = B EditorVideo = 1 CredentialEndpoint = ```

With DXGI this produces the below output when running the game. It is the wrong size, of course.

image

We can see the modeset from the client logs.

[I]    740314432              main.c:637  | main_frameThread               | Format: FRAME_TYPE_BGRA 3840x2160 stride:3840 pitch:15360 rotation:0
[I]    791598146              main.c:637  | main_frameThread               | Format: FRAME_TYPE_BGRA 2560x1440 stride:2560 pitch:10240 rotation:0
[I]    834073818              main.c:637  | main_frameThread               | Format: FRAME_TYPE_BGRA 3840x2160 stride:3840 pitch:15360 rotation:0
[I]    870712028              main.c:637  | main_frameThread               | Format: FRAME_TYPE_BGRA 2560x1440 stride:2560 pitch:10240 rotation:0

However, when trying this with NvFBC, the host gets put in a bad state.

Host logs ``` [E] 450339731493 wrapper.cpp:331 | NvFBCToSysCapture | Unknown NVFBCRESULT failure -19 [E] 450339732077 app.c:784 | app_main | Capture interface reported a fatal error [I] 450339732696 mousehook.c:141 | threadProc | Mouse hook thread received quit request [I] 450339732697 mousehook.c:101 | updateThreadProc | Mouse hook update thread received quit request [I] 450339738939 force_compose.c:89 | threadProc | Force composition received quit request [I] 450340191402 app.c:840 | app_main | Host application exited [I] 450394345776 time.c:85 | windowsSetTimerResolution | System timer resolution: 500.0 μs [I] 450394346668 app.c:542 | app_main | Looking Glass Host (B4-566-g67d1073c0a+) [I] 450394347598 cpuinfo.c:36 | lgDebugCPU | CPU Model: AMD Ryzen 9 5950X 16-Core Processor [I] 450394348529 cpuinfo.c:37 | lgDebugCPU | CPU: 4 cores, 8 threads [I] 450394350503 ivshmem.c:131 | ivshmemInit | IVSHMEM 0* on bus 0x0, device 0x9, function 0x0 [I] 450394452353 app.c:559 | app_main | IVSHMEM Size : 128 MiB [I] 450394452809 app.c:560 | app_main | IVSHMEM Address : 0x2CA0000 [I] 450394453257 app.c:561 | app_main | Max Pointer Size : 1024 KiB [I] 450394453680 app.c:562 | app_main | KVMFR Version : 14 [I] 450394454307 app.c:623 | app_main | Max Frame Size : 62 MiB [I] 450394454682 app.c:648 | app_main | Trying : NVFBC (NVidia Frame Buffer Capture) [I] 450394462775 wrapper.cpp:90 | NvFBCInit | NvFBC SDK Version: 112 [I] 450395141092 nvfbc.c:241 | nvfbc_init | DiffMap block : 128x128 [I] 450395141462 nvfbc.c:242 | nvfbc_init | Cursor mode : decoupled [I] 450395166253 force_compose.c:81 | threadProc | Created window to force composition [I] 450395242340 app.c:673 | app_main | Using : NVFBC (NVidia Frame Buffer Capture) [I] 450395242737 app.c:674 | app_main | Capture Method : Synchronous [I] 9662260 time.c:85 | windowsSetTimerResolution | System timer resolution: 500.0 μs [I] 9662839 app.c:542 | app_main | Looking Glass Host (B4-566-g67d1073c0a+) [I] 9663227 cpuinfo.c:36 | lgDebugCPU | CPU Model: AMD Ryzen 9 5950X 16-Core Processor [I] 9663570 cpuinfo.c:37 | lgDebugCPU | CPU: 4 cores, 8 threads [I] 9664304 ivshmem.c:131 | ivshmemInit | IVSHMEM 0* on bus 0x0, device 0x9, function 0x0 [I] 9670035 app.c:559 | app_main | IVSHMEM Size : 128 MiB [I] 9670346 app.c:560 | app_main | IVSHMEM Address : 0x2950000 [I] 9670634 app.c:561 | app_main | Max Pointer Size : 1024 KiB [I] 9670939 app.c:562 | app_main | KVMFR Version : 14 [I] 9672233 app.c:623 | app_main | Max Frame Size : 62 MiB [I] 9672536 app.c:648 | app_main | Trying : NVFBC (NVidia Frame Buffer Capture) [I] 9679545 wrapper.cpp:90 | NvFBCInit | NvFBC SDK Version: 112 [I] 10347289 nvfbc.c:241 | nvfbc_init | DiffMap block : 128x128 [I] 10347621 nvfbc.c:242 | nvfbc_init | Cursor mode : decoupled [I] 10352868 force_compose.c:81 | threadProc | Created window to force composition [I] 10448501 app.c:673 | app_main | Using : NVFBC (NVidia Frame Buffer Capture) [I] 10448872 app.c:674 | app_main | Capture Method : Synchronous [I] 10449164 app.c:348 | captureStart | ==== [ Capture Start ] ==== [E] 146236643 wrapper.cpp:319 | NvFBCToSysCapture | Invalid parameter [E] 146236925 app.c:784 | app_main | Capture interface reported a fatal error [I] 146237282 mousehook.c:101 | updateThreadProc | Mouse hook update thread received quit request [I] 146237296 mousehook.c:141 | threadProc | Mouse hook thread received quit request [I] 146238160 force_compose.c:89 | threadProc | Force composition received quit request [I] 146249034 app.c:840 | app_main | Host application exited [I] 147373017 time.c:85 | windowsSetTimerResolution | System timer resolution: 500.0 μs [I] 147373384 app.c:542 | app_main | Looking Glass Host (B4-566-g67d1073c0a+) [I] 147373738 cpuinfo.c:36 | lgDebugCPU | CPU Model: AMD Ryzen 9 5950X 16-Core Processor [I] 147374092 cpuinfo.c:37 | lgDebugCPU | CPU: 4 cores, 8 threads [I] 147374763 ivshmem.c:131 | ivshmemInit | IVSHMEM 0* on bus 0x0, device 0x9, function 0x0 [I] 147380282 app.c:559 | app_main | IVSHMEM Size : 128 MiB [I] 147380589 app.c:560 | app_main | IVSHMEM Address : 0x2D40000 [I] 147380877 app.c:561 | app_main | Max Pointer Size : 1024 KiB [I] 147381176 app.c:562 | app_main | KVMFR Version : 14 [I] 147381606 app.c:623 | app_main | Max Frame Size : 62 MiB [I] 147381891 app.c:648 | app_main | Trying : NVFBC (NVidia Frame Buffer Capture) [I] 147386156 wrapper.cpp:90 | NvFBCInit | NvFBC SDK Version: 112 [I] 147753632 nvfbc.c:241 | nvfbc_init | DiffMap block : 128x128 [I] 147754307 nvfbc.c:242 | nvfbc_init | Cursor mode : decoupled [I] 147760952 force_compose.c:81 | threadProc | Created window to force composition [I] 147855941 app.c:673 | app_main | Using : NVFBC (NVidia Frame Buffer Capture) [I] 147856525 app.c:674 | app_main | Capture Method : Synchronous ```

Once in this state any one of the following is insufficient:

One must do both to get Looking Glass working again.

Some debug prints show that

https://github.com/gnif/LookingGlass/blob/b21d842f0effa44b03b1a8deb023d5416ad7d7fa/host/platform/Windows/capture/NVFBC/src/nvfbc.c#L331-L333

correctly returns 3840x2160 after the modeswitch, and that the error occurs upon the third frame. A dumb hardcoding of width/height in getDesktopSize to 2560x1440 gets things working, but cropped weirdly.

Looking Glass is otherwise able to run on this machine when the desktop resolution is set to 3840x2160, it's just the modeset that breaks it.

gnif commented 2 years ago

Please update to B5 and test using DXGI. If this resolves the issue then it's a bug in the deprecated NvFBC API which we can do nothing about.