search

bucanero / pkgi-ps3

A PlayStation 3 package download tool
http://www.bucanero.com.ar/
MIT License
721 stars 91 forks source link

Cannot download / unpack XML #100

Open cinnamonmatexfce opened 2 years ago

cinnamonmatexfce commented 2 years ago

Is this a know behaviour with this NPEB02182? Shouldn't PS3 automatically handle PKG merging by the instructions provided via the XML file? The real package is not downloaded.

Here's what I found:

/dev_hdd0/vsh/game_pkg/80000001:
EP0101-NPEB02182_00-MGS4MAINGAME0000.pkg [1,94KiB]
ICON_FILE [1,20KiB]
d0.pdb [738B]
d1.pdb [792B]
f0.pdb [0B]
bucanero commented 2 years ago

The app doesn't support XML data files. It can only download direct .pkg links. See #34

cinnamonmatexfce commented 2 years ago

The app doesn't support XML data files. It can only download direct .pkg links. See #34

OK thank you. How actually TONY handles splitted multipart .pkg games? Is this a known procedure or it still needs to be discovered / reverse-engineered?

bucanero commented 2 years ago

The PS3 background download manager is probably in charge of handling that, but it hasn't been fully reversed so we don't really know.

My assumption is that the PS3 parses the XML, and adds all the parts to the background manager that downloads and merges everything as a single file.

A good test for someone with access to the store, would be to queue one of these big xml downloads, and once it's queued then just pause it. Next, with either HEN or CFW, grab all the tasks files (d0.pdb, d1.pdb, f0.pdb, etc) from the /dev_hdd0/vsh/game_pkg/ folder. With those files we could probably get a clear view of how it is handled and maybe it can be replicated on pkgi.

cinnamonmatexfce commented 2 years ago

I didn't fully understand all the story behind TONY shutting down PSN for PS3... Is nowadays possible to access PSN and purchase PSN games from it? I would glady try it with my account in my PS3. Only thing I should care is disabling syscalls+clear history? Or should I take extra precautions?

bucanero commented 2 years ago

They reverted that, so currently on latest firmware (4.88) you can access the store and even buy new content for the PS3. (but not from the store's web site, only from the store app on the PS3) I think they don't allow adding funds on the PS3 store app, so you need for example to add funds first, and then just do the purchase.

Removing syscalls + clear history should be enough, yes. If you want additional safety and you have patience, you could install a 4.88 dual-boot firmware (so it acts just like OFW), then after the purchase you can go back and re-install CFW (without any patch or jailbreak).

cinnamonmatexfce commented 2 years ago

The PS3 background download manager is probably in charge of handling that, but it hasn't been fully reversed so we don't really know.

My assumption is that the PS3 parses the XML, and adds all the parts to the background manager that downloads and merges everything as a single file.

A good test for someone with access to the store, would be to queue one of these big xml downloads, and once it's queued then just pause it. Next, with either HEN or CFW, grab all the tasks files (d0.pdb, d1.pdb, f0.pdb, etc) from the /dev_hdd0/vsh/game_pkg/ folder. With those files we could probably get a clear view of how it is handled and maybe it can be replicated on pkgi.

Please note this: while downloading from within the Store, if I choose to put the task in background, then it makes me wait about 4 minutes telling me preparation in progress. During these four minutes of waiting (note I'm still inside the PlayStation Store APP) there is no network activity (I can tell because leds of the router's ethernet port aren't constantly blinking) but a lot of hard disk activity is happening instead (I see harddisk's led of the PS3 steady blinking)!


So, what should I report to you?

bucanero commented 2 years ago

So, what should I report to you?

  • "Background Download" files (copied after the completion of the 4 minutes preparation in progress)? or
  • "Normal" files (ungracefully shutting down PS3 while downloading from within the PlayStation Store)?

Let's try first with the "background download" files generated after the 4 minutes preparation process. Once the Background download is ready, just pause the download, grab the pdb files and all the stuff in the task folder /dev_hdd0/vsh/....

Btw, I assume you'll also find a huge empty file that is a placeholder for the download. No need to share that file as I think it has nothing for this research.

We can start with those files as they should be small and easy to review, if that doesn't help we'll look into the other option (ungraceful shutdown, etc)

cinnamonmatexfce commented 2 years ago

So, what should I report to you?

  • "Background Download" files (copied after the completion of the 4 minutes preparation in progress)? or
  • "Normal" files (ungracefully shutting down PS3 while downloading from within the PlayStation Store)?

Let's try first with the "background download" files generated after the 4 minutes preparation process. Once the Background download is ready, just pause the download, grab the pdb files and all the stuff in the task folder /dev_hdd0/vsh/....

Inside the attached dev_hdd0--vsh--task.zip you'll find 00000001 and 00000002 folders from dev_hdd0/vsh/task/. I don't know why there's also the 00000001 folder. Instead, the 00000002 folder is the one where PS3 is actually downloading the PSN PKG game.


Btw, I assume you'll also find a huge empty file that is a placeholder for the download. No need to share that file as I think it has nothing for this research.

Yup, I confirm.

bucanero commented 2 years ago

I was reviewing the files and all the links are embedded in the pdb files, so my guess confirms that the background downloader downloads each link in order and stores everything in the big empty "placeholder" file.

When I have a chance, I'll try to replicate and generate such files manually and see if the ps3 actually downloads the parts.

cinnamonmatexfce commented 2 years ago

Hey @bucanero , any news / updates? 😉

Thanks

bucanero commented 2 years ago

Hey @bucanero , any news / updates? 😉

Thanks

no, I didn't had any time to work on PS3 homebrews or check this download issue. I have been focusing on PS4 stuff lately, but at some point I'll go back to ps3 stuff and take a look.

For the moment these kind of links should be downloaded on a PC, merged into a single file, and then installed as a regular .pkg

cinnamonmatexfce commented 2 years ago

No problem and thanks for your kind work, pal!! 😎💪

bucanero commented 1 year ago

Leaving the source XML of these task files, for future reference:

<hfs_manifest>
<file_name>EP0101-NPEB02182_00-MGS4MAINGAME0000_bg_1_a625a8e6ac40a4f6228733858b44bca3e8c003e9.pkg</file_name>
<file_size>28598265664</file_size>
<number_of_split_files>7</number_of_split_files>
<pieces file_size="4687500000" hash_value="0a727a8f7c0a1992770f90f0505c23aa816dea79" index="0" url="http://xxx/xxx_00.pkg"/>
<pieces file_size="4687500000" hash_value="cbd20df7687aee236792c4d913d9d67e5df766c6" index="1" url="http://xxx/xxx_01.pkg"/>
<pieces file_size="4687500000" hash_value="ef4c3fcb50ef29f44b882d03a07eab4f71595b31" index="2" url="http://xxx/xxx_02.pkg"/>
<pieces file_size="4687500000" hash_value="9a49afbb21d15a2d65547c43777c7c09e8142d4b" index="3" url="http://xxx/xxx_03.pkg"/>
<pieces file_size="4687500000" hash_value="58f67181ae14e566796d5aa55831c8131b78e40b" index="4" url="http://xxx/xxx_04.pkg"/>
<pieces file_size="4687500000" hash_value="95233845784a1d30fcf0572614ec4d8bbc402d64" index="5" url="http://xxx/xxx_05.pkg"/>
<pieces file_size="473265664" hash_value="8a27fa16663d4c6570b4a9909fdf911f689cceb1" index="6" url="http://xxx/xxx_06.pkg"/>
</hfs_manifest>