search

RPCS3 / rpcs3

PlayStation 3 emulator and debugger
https://rpcs3.net/
GNU General Public License v2.0
15.44k stars 1.92k forks source link

RPCS3/PlayStation Home's videos not displaying properly, distorted but audio plays. #11751

Open PSHFigment opened 2 years ago

PSHFigment commented 2 years ago

Quick summary

RPCS3 doesn't support or incorrectly supports MP4 format of video codec's used in PlayStation Home.

Details

RPCS3 doesn't correctly display Home's video format. visuals distorted but audio plays fine. (PS3, display of video in Bowling Alley.) 6371df6edb4894de94281c6d31a59ee7 (RPCS3, display of video in Bowling Alley.) 7d76fa561550611fd71272af33790958 (Original Video from PlayStation Home content snapshot..) https://user-images.githubusercontent.com/74638626/162596647-f56fdc1c-d730-406c-9d45-5637caa88e83.MP4

Logs

https://cdn.discordapp.com/attachments/838553670831308820/962539682044997672/RPCS3.log

Specs

Processor Intel(R) Core(TM) i7-7700 CPU @ 3.60GHz 3.60 GHz Installed RAM 32.0 GB GPU NVIDIA GeForce RTX 2080 Ti System type 64-bit operating system, x64-based processor Edition Windows 10 Home Version 21H1 OS build 19043.1586 & Ryzen 5800H GeForce 3050ti 16GB Ram Windows 11

NagatoDEV commented 2 years ago

Information regarding PlayStation Home Supported Video Codec Formats:

Greetings RPCS3 developers, I hope this message finds you well. I'm one of the developers for the Destination Home project and I wanted to add some additional information/details that can possibly give insight on this issue.

To add to this post PlayStation Home supports the following media/video codecs.

Furthermore, it also supports the following video container formats:

I know that Home as well can play video bit rates up to the maximum allowed for each supported MPEG-4 profile level e.g., up to 8 Mbps for MPEG-4 & 14 Mbps for AVC/H.264. From research, I know there are some general guidelines that the Home client will accept for video content. Note this, is just what I've seen from various media content for scenes on Home.

Format Width Height FPS
MPEG-4 Part 2, Simple Profile 176 144 15 FPS
MPEG-4 Part 2, Simple Profile 352 288 aprox. 29 FPS
MPEG-4 Part 2, Advanced Simple Profile 352 288 aprox. 29 FPS
MPEG-4 Part 2, Advanced Simple Profile 640 480 aprox. 29 FPS
MPEG-4 Part 2, Advanced Simple Profile 720 576 aprox. 29 FPS
MPEG-4 Part 10 (AVC/H.264) 320 240 aprox. 29 FPS
MPEG-4 Part 10 (AVC/H.264) 640 480 aprox. 29 FPS
MPEG-4 Part 10 (AVC/H.264) 720 480 aprox. 29 FPS
MPEG-4 Part 10 (AVC/H.264) 1280 720 aprox. 29 FPS

PlayStation Home Supported Audio Codecs:

It seems on our end, that the audio from content is being played just fine. However, as a showcase from the image below you can clearly see the screen is distorted.

NPIA00005_screenshot_2022_04_10_00_33_22

In contrast on original hardware (PS3) you can see that in the same location the video content is playing as intended, for the end-user.

PlayStationAHome_Developer_HEN-QA_Launcher

The original video content displayed on PlayStation Home

https://user-images.githubusercontent.com/67494727/162604932-93afca60-27cf-4fe8-851a-591dc874b62b.mp4

This video (_NowPlaying_AnimalHouseChainSwing.mp4) has the following properties:

General

Video

Audio

ID: 2

RPCS3 Logs

To provide some documentation, here were the original logs from my current gameplay on RPCS3.

My Specs

Megamouse commented 2 years ago

Thanks for the detailed info. But please hide or remove the huge log and upload a file attachment instead

NagatoDEV commented 2 years ago

Thanks for the detailed info. But please hide or remove the huge log and upload a file attachment instead

Hey Megamouse, I've redacted the huge logfile from the original post & thank you in advance for looking into this issue. Down below here are the logs in a .txt file RPCS3_PlayStation_Home_Video_Bug (3).txt

Further investigation regarding PlayStation Home's video content:

NPIA00005_screenshot_2022_04_11_19_24_20

After further analysis, it seems that some video content on Home runs on RPCS3 as intended in certain spaces. For instance, in the Konami Penthouse public scene, the video is parsing information for the client. In turn, the end-user will be able to see the video along with its original audio. For documentation purposes here is a quick video of Konami_clothing_PSHOME_Profile3.mp4 running along with RPCS3 logs for your review.

https://user-images.githubusercontent.com/67494727/162849875-7c13586f-e635-4e16-a89c-17723d97f972.mp4

Not so sure why some video content is working without any issues in certain areas versus others, but hopefully this brief synopsis can help you out.

Konami_clothing_PSHOME_Profile3.mp4 Properties:

Here is the original Konami_clothing_PSHOME_Profile3.mp4 file that you can review as well.

https://user-images.githubusercontent.com/67494727/162850681-132f77c7-99fe-4057-a2ad-ab72efa1b8da.mp4

General:

Video:

Audio:

Other #1

Other #2:

My Specs:

Megamouse commented 2 years ago

please try with #11754

PSHFigment commented 2 years ago

Screen is completely black now instead of glitching with this client (audio works but no video.), this gets spammed into the log as well.

E SYS: 'cellVdecDecodeAu' failed with 0x80610101 : CELL_VDEC_ERROR_ARG, vdec=1, auInfo=1, upper=0, startAddr=0x42daf380 [1]

https://user-images.githubusercontent.com/74638626/163130556-cbe1e04c-e010-4e44-9531-f754325bf665.mp4

Megamouse commented 2 years ago

Please try again with the PR

Megamouse commented 2 years ago

Please try again with the latest version

NagatoDEV commented 2 years ago

Hey Megamouse, sorry for the delayed response I have been busy with my academic career, and just saw this comment. However, I just tested out the RPCS3 client version rpcs3-v0.0.21-13475-6a671554_win64 & ran PlayStation Home, and unfortunately the client still only displays audio & the actual video footage seems to be distorted/fuzzy.

https://user-images.githubusercontent.com/67494727/164791533-72e814e2-8e15-46e2-9cb0-11979f150bcc.mp4

Logs

As always, here are logs from this session.

Megamouse commented 2 years ago

how can i reproduce this on my own?

kd-11 commented 2 years ago

The newest error looks like output pitch is broken. Hmm, maybe it is worth trying enabling WCB and CPU blit to debug this. But yea, we need some kind of steps to reproduce this one ourselves.

Megamouse commented 2 years ago

It might still be a straightforward vdec pitch issue, similar to what i fixed for Uncharted 2.

kd-11 commented 2 years ago

Yea. It seems like the data is there, just incorrect scanning pitch.

NagatoDEV commented 2 years ago

Hey Megamouse, here is the offline client you can look at it to replicate this issue, Home has an MLA issue in this client. However, with Home's navigator, you can travel to Central Plaza. Just for documentation purposes, here is a quick guide I posted on YouTube on how to fully travel to Central Plaza for debugging.

Download

https://github.com/NagatoDEV/PlayStation-Home-Master-Archive/releases/tag/v1.00

illusion0001 commented 2 years ago

What happens if you LLE libvdec?

NagatoDEV commented 2 years ago

Hey, illusion0001 that seemed to actually work by activating the LLE libvdec.sprx option in RPCS3 settings to activate with the following client rpcs3-v0.0.21-13475-6a671554_win64

Images/Videos

https://user-images.githubusercontent.com/67494727/164844099-8b518598-5d81-451a-b64b-6dab0c59341f.mp4

NPIA00005_screenshot_2022_04_22_21_04_43

NPIA00005_screenshot_2022_04_22_21_10_25

illusion0001 commented 2 years ago

Thanks for confirming, only HLE issue now

Megamouse commented 2 years ago

It appears that the rendering and the outgoing data are correct. So the issue is caused earlier. This means that either the data offsets or pitch is wrongly set by us, or the frame is not decoded correctly or the frame data is wrong in the first place.

I think it's just us doing some derpy stuff with av. I managed to get a slightly different picture by using a different method of determining the incoming data offsets, but it seems I only made it worse xD

Megamouse commented 1 year ago

The fix wasn't correct and broke other games.