search

aldostools / webMAN-MOD

Extended services for PS3 console (web server, ftp server, netiso, ntfs, ps3mapi, etc.)
https://aldostools.github.io/webMAN-MOD/
GNU General Public License v3.0
1.3k stars 176 forks source link

ps3netsrv slow when 5k games #1125

Open Noro944 opened 3 days ago

Noro944 commented 3 days ago

Hi, i recently started to use ps3netsrv with latest webman mod on my ps3 ultra slim hen. Thank you very much for the tools. I use a cable directly connected from ps3 to pc. Speed is like 20mb/s

In ROMS folder i have in each folder 1k games ROMS\MEGADRIVE, ROMS\NES, ROMS\PSPISO, ROMS\SNES ROMS\PSXISO(550 games because bin.cue files are listed multiple times so 550*2 =1100, i think the limit is 1200 ) So around 4-5k games, knowing that only psp/psxiso games are heavy.Others are like some mb. I use prepiso to scan folders.

Multiple problems: So FIRST problem : the scan is really really slow. Is it possible to speed it up ? SECOND problem : as i said above, games in ROMS\PSXISO each bin/cue files are listed so for one game i have two/multiple icons. But PSXISO which is in main folder with GAMES, PS3ISO, etc the games are correctly displayed so even if in one game folder there are like multiple bin cue files the game is listed only one time. Thanks

aldostools commented 3 days ago

Thank you for your feedback. I will try to improve these issues if I get some free time this weekend.

aldostools commented 3 days ago

You can try this test build [LINK REMOVED]

It should fix the issue that allowed duplicated PSX games in /ROMS/PSXISO

I also added a new condition to only scan the ROMS paths listed in /dev_hdd0/tmp/wm_res/roms_paths.txt You have to edit the text file after installation so it only includes the roms folders that you need to scan. Check if the scanning time is improved.

Noro944 commented 3 days ago

Thanks, i will try it

Noro944 commented 3 days ago

So i have tested it, i edited roms_paths.txt. Launched the scan and it hanged at the end of scan. Tried multiple times, same result. Every time i have to restart the ps3. So something got wrong.

Something that i have noticed: games in folder ROMS\PSPISO are scanned very fastly (30 seconds for 800 games) but SNES,NES,MEGADRIVE which contain 1k games each are scanned super slowly like 5-10 minutes each folder. Is there a limit maybe 800 games in each folder that i have to put ?

aldostools commented 3 days ago

Thank you for your feedback. I will check why it's hanging... probably a segfault error due the optimization.

In regards to the difference in time between PSPISO and SNES/NES/MEGADRIVE ROMS, I don't know why it's taking 10X more time. The code executed is exactly the same. I will try to investigate.

Noro944 commented 3 days ago

I have reverted to the stable version and got another issue: i have uploaded ABC Monday Night Football (USA).png in /dev_hdd0/game/BLES80608/USRDIR/covers_retro/snes and i have the game in ROMS\SNES folder but after scan it didn't add the icon in game list. Is there something wrong i did ?

aldostools commented 3 days ago

webMAN MOD does not check the ROMS covers in multiMAN folder. The covers for ROMS should be included in the same folder of the ROMS with the same file name (without extension) + .png or .jpg

I have fixed the last version, so it doesn't freeze (at least it didn't on my system) webMAN_MOD_latest.zip

It has a few hacks to speed up the file scanning, but probably you won't see big improvements.

Noro944 commented 2 days ago

Thank for the tip for ROMS covers. It didn't hang after the scan, it does list now in ROMS\PSXISO only one game per folder 👍 However the scan is very slow for all the folders (even for PSXISO) except PSPISO, maybe because PSPISO contains .iso files ?

Noro944 commented 2 days ago

Can you tell me which format is the fastest to scan the games on external disk fat32 or exfat ? is external disk faster than ps3netsrv ? ( i don't think so)

aldostools commented 2 days ago

Can you test taking out the cue files from the folder PSXISO?

If it scans much faster, I could make wMM load the cue files from another folder when the BIN is mounted.

aldostools commented 2 days ago

Can you tell me which format is the fastest to scan the games on external disk fat32 or exfat ? is external disk faster than ps3netsrv ? ( i don't think so)

External disk FAT32 could be faster due it's reading the files using the native file system. But files cannot be larger than 4GB. Therefore most PS3ISO files need to be split as .iso.0.

External exFAT or NTFS could be faster than ps3netsrv, but I haven't measured the performance to confirm it.

Noro944 commented 1 day ago

Hi, I have like 1.5k psxiso games so i can't remove manually cue files from game folders. I did try fat32 drive and it's not faster. So i gave UP scanning rétro. I will only scan PS1,ps2,psp. And ps3 of course. So anther thing i have found that there IS a folder named "psx" in multiman rétro folder that contains covers for ps2,PS1,psp and i have a lot of them. I will try to empty covers folders in multiman to see if the scan IS faster. With your new version of webmanmod should i put covers in each separate folder in multiman ?

aldostools commented 1 day ago

Thank you for your feedback. The covers for ROMS now are also scanned in multiMAN^s covers_retro within the folder of the same name but in lowercase.

If you have ROMS/PSXISO, it will search the covers in covers_retro/psxiso. For SNES it will use snes.

Noro944 commented 18 hours ago

Hi, Just for information i tried the latest version 1.47.47 and with fat 32 drive at the scan ps3 hangs but the 1.47.46h works great.

aldostools commented 17 hours ago

Thank you for your feedback. I don't remember to have touched any code related to FAT32. I will investigate.

aldostools commented 2 hours ago

Hi, Just for information i tried the latest version 1.47.47 and with fat 32 drive at the scan ps3 hangs but the 1.47.46h works great.

I tried to scanning games stored on FAT32 but unfortunately I couldn't reproduce your issue.

Maybe your issue with 1.47.47 was a random freeze that could happen ocassionally. If you could find a pattern of how to reproduce the bug, it would help identify the bug.