[question] NTFS Compression support

[korn3r]

Hi. Is it possible to add support for ntfs compression in prepNTFS? I tried enabling ntfs compression on my external HDD for NBA2k14 iso and iso went from 24gigabytes down to 8 gigabytes*. But unfortunately prepNTFS cant find such iso on drive. Would be a nice feature to have if PS3 would be able to handle files that are compressed using NTFS native compression.

*- freed up space depends heabily on how much of empty space is there inside image. for example compressing Uncharted 2 and Ratchet and Clank All 4 one ISOs gave me additional ~700 megabytes total.

[aldostools]

The NTFS library doesn't seem to support NTFS compression. I have been trying to use that feature without success.

Or maybe I'm overlooking some option.

[korn3r]

The NTFS library doesn't seem to support NTFS compression. I have been trying to use that feature without success.

Or maybe I'm overlooking some option.

Which library is that? Does it have own github page?

[aldostools]

https://github.com/aldostools/webMAN-MOD/tree/master/_Projects_/libs/libntfs-master

This is the original repository: https://github.com/Estwald/libntfs_ext

[korn3r]

https://github.com/aldostools/webMAN-MOD/tree/master/_Projects_/libs/libntfs-master

This is the original repository: https://github.com/Estwald/libntfs_ext

thanks. i will try to compile that myself. weird that it doesnt support compression, since it has code for compression - compress.h and compress.c files.

also it says its based on "libntfs for Wii". sources in Estwald repo was updated 11 years ago, while sources on libntfs for Wii github where updated 5 years ago.... maybe i would be able to compile newer source code (probably not)

[aldostools]

Yes, it has the compress.c module, but for some reason that I haven't been able to make it work.

That's why I said that maybe I'm overlooking some option.

[korn3r]

Yes, it has the compress.c module, but for some reason that I haven't been able to make it work.

That's why I said that maybe I'm overlooking some option.

prepISO seems to be using libntfs_ext.a and not libntfs_prx.a do i need to recompile both .../libs/libntfs-master and the source from original Estwald repository? Which one of those is actually used when mounting games from external drive? libntfs_prx.a from webman or libntfs_ext.a from prepISO?

[aldostools]

libntfs_ext.a is the original NTFS library ported to PS3 by Estwald. It uses PSL1GHT SDK. libntfs_prx.a is a conversion of libntfs_ext for use on plugins that require to be compiled using the official PS3 SDK.

prepISO is PSL1GHT project. Therefore it requires libntfs_ext.a.

libntfs_prx.a is more limited due the memory restrictions on VSH. The number of pages in the cache is smaller (1 vs 128). I don't think it supports ext2/3/4 either, unlike libntfs_ext.

[aldostools]

libntfs_prx is used by webMAN MOD to scan the games in NTFS without require prepISO, browse files and allow FTP transfers to NTFS drive.

prepISO has more features than the prepNTFS in webMAN MOD:

• it supports exFAT, NTFS and ext2/3/4. webMAN's prepNTFS only supports NTFS.
• prepISO can convert packages, videos, music files & themes to fake ISOs. webMAN's prepNTFS can't.

[korn3r]

libntfs_prx is used by webMAN MOD to scan the games in NTFS without require prepISO, browse files and allow FTP transfers to NTFS drive.

prepISO has more features than the prepNTFS in webMAN MOD:

• it supports exFAT, NTFS and ext2/3/4. webMAN's prepNTFS only supports NTFS.
• prepISO can convert packages, videos, music files & themes to fake ISOs. webMAN's prepNTFS can't.

hm... thanks for answer, i suppose i can just remove prepiso and try to use internal one. this will at least remove requirement of installing psl1ght which i cant do for some reason. i did rebuild libntfs_prx.a and it became 60kb fatter than original one. i wonder why it became so fat...

[aldostools]

I recommend the external prepISO over the intenal prepNTFS. prepNTFS is sometimes buggy, while prepISO tends to work better. At least on my system.

[korn3r]

I recommend the external prepISO over the intenal prepNTFS. prepNTFS is sometimes buggy, while prepISO tends to work better. At least on my system.

I tried to build with USE_INTERNAL_NTFS_PLUGIN flag, and got this:

In file included from main.c:427:
include/mount/rawseciso.h: In function 'rawseciso_thread':
include/mount/rawseciso.h:1097: warning: implicit declaration of function 'fake_eject_event'
include/mount/rawseciso.h:1097: warning: nested extern declaration of 'fake_eject_event'
include/mount/rawseciso.h:1112: warning: implicit declaration of function 'fake_insert_event'
include/mount/rawseciso.h:1112: warning: nested extern declaration of 'fake_insert_event'
include/mount/rawseciso.h: In function 'rawseciso_stop_thread':
include/mount/rawseciso.h:1289: warning: unused variable 'exit_code'
In file included from main.c:450:
include/mount/eject_insert.h: At top level:
include/mount/eject_insert.h:38: error: static declaration of 'fake_insert_event' follows non-static declaration
include/mount/rawseciso.h:1112: error: previous implicit declaration of 'fake_insert_event' was here
include/mount/eject_insert.h:45: error: static declaration of 'fake_eject_event' follows non-static declaration
include/mount/rawseciso.h:1097: error: previous implicit declaration of 'fake_eject_event' was here
make: *** [objs/main.ppu.o] Error 1
Could Not Find C:\Users\u\Downloads\webMAN-MOD-master\webftp_server.elf
Could Not Find C:\Users\u\Downloads\webMAN-MOD-master\webftp_server.prx
Could Not Find C:\Users\u\Downloads\webMAN-MOD-master\webftp_server.sym

is it supposed to fail? not sure if i should open another ticket for that.

[aldostools]

No. I fixed the compilation errors in this commit. https://github.com/aldostools/webMAN-MOD/commit/c3071072361a1e7623fcda2b05a7d4f9b2404e55

I didn't notice the error because I don't compile the release versions with that flag enabled.

[korn3r]

No. I fixed the compilation errors in this commit. c307107

I didn't notice the error because I don't compile the release versions with that flag enabled.

there is also a typo here - https://github.com/aldostools/webMAN-MOD/blob/master/_Projects_/prepISO/source/main.c ";" is missing at the end of line 165, thus prepISO wont build:

c:/Users/u/Downloads/webMAN-MOD-master/_Projects_/prepISO/source/main.c: In function 'main':
c:/Users/u/Downloads/webMAN-MOD-master/_Projects_/prepISO/source/main.c:167:2: error: expected ';' before 'if'
c:/Users/u/Downloads/webMAN-MOD-master/_Projects_/prepISO/source/main.c:188:5: error: 'button' undeclared (first use in this function)
c:/Users/u/Downloads/webMAN-MOD-master/_Projects_/prepISO/source/main.c:188:5: note: each undeclared identifier is reported only once for each function it appears in
make[1]: *** [main.o] Error 1
make: *** [build] Error 2
Could Not Find C:\Users\u\Downloads\webMAN-MOD-master\_Projects_\prepISO\*.elf

Sorry for being annoying :)

[korn3r]

Also since i already started being annoying: build fails on nobd patch if ps3mapi is disabled (i probably already mentioned that when i was creating another issue many months ago)

include/www/../cmd/nobd.h: In function 'do_web_command':
include/www/../cmd/nobd.h:29: warning: implicit declaration of function 'ps3mapi_find_peek_poke_hexview'
include/www/../cmd/nobd.h:29: warning: nested extern declaration of 'ps3mapi_find_peek_poke_hexview'
  undefined reference to `ps3mapi_find_peek_poke_hexview'

[aldostools]

The issue should be fixed in the latest commit.

[korn3r]

About compression: is there even lz library on ps3 needed to work with ntfs compression? also accoding to microsoft documentation there is totally separate call used to handle compressed files. but thats just info in case you would some day want to dig that compression issue yourself.

And... to be even more annoying, i tried to rebuild thing from Projects folder. pkglaunch didnt build, other ones just give warnings.

pkglaunch: c:/Users/u/Downloads/webMAN-MOD-master/_Projects_/pkglaunch/main.c:7:19: fatal error: untar.h: No such file or directory

ps3mon

In file included from source/syscon.c:9:
./include/printf.h:27: warning: redundant redeclaration of 'sprintf'
./include/printf.h:28: warning: redundant redeclaration of 'snprintf'
./include/printf.h:29: warning: redundant redeclaration of 'printf'
c:/msys/1.0/cell_4.50/host-win32/ppu/bin/../../../target/ppu/include/stdio.h:143: warning: previous declaration of 'printf' was here
source/syscon.c: In function 'sys_sm_request_error_log':
source/syscon.c:14: warning: cast from pointer to integer of different size
source/syscon.c:14: warning: cast from pointer to integer of different size
source/syscon.c:14: warning: cast from pointer to integer of different size
source/syscon.c: In function 'dump_syscon_errors':
source/syscon.c:13: warning: inlining failed in call to 'sys_sm_request_error_log': --param max-inline-insns-single limit reached
source/syscon.c:133: warning: called from here

video_rec

main.c: In function 'video_rec_thread':
main.c:300: warning: assignment from incompatible pointer type
main.c:302: warning: pointer to a function used in arithmetic

wm_proxy

In file included from main.c:64:
mount.h: In function 'get_explore_interface':
mount.h:42: warning: cast from function call of type 'int' to non-matching type 'struct explore_plugin_interface *'

[korn3r]

Also build fails on disabling #define VIRTUAL_PAD

In file included from main.c:419:
include/init/vsh.h: In function 'start_xmb_player':
include/init/vsh.h:347: warning: implicit declaration of function 'parse_pad_command'
include/init/vsh.h:347: warning: nested extern declaration of 'parse_pad_command'
include/init/vsh.h:349: warning: implicit declaration of function 'press_accept_button'
include/init/vsh.h:349: warning: nested extern declaration of 'press_accept_button'
main.c: At top level:
include/init/compare.h:18: warning: '_islike' defined but not used
  undefined reference to `parse_pad_command'
  undefined reference to `press_accept_button'
  undefined reference to `parse_pad_command'
  undefined reference to `press_accept_button'
  undefined reference to `press_accept_button'
make: *** [webftp_server.sym] Error 1
Could Not Find C:\Users\u\Downloads\webMAN-MOD-master\webftp_server.elf
Could Not Find C:\Users\u\Downloads\webMAN-MOD-master\webftp_server.prx
Could Not Find C:\Users\u\Downloads\webMAN-MOD-master\webftp_server.sym

im not in extreme need of that thing working, just decided that is a right thing to report everything that doesnt work. if i annoyed you too much with this, you can just ignore this or say me to stop :)

UPD (added much much later): undefining #define PKG_LAUNCHER:

In file included from include/mount/mount_cobra.h:123,
                 from include/mount/mount.h:1642,
                 from main.c:473:
include/mount/mount_net.h: In function 'mount_thread':
include/mount/mount_net.h:147: warning: implicit declaration of function 'copy_rom_media'
include/mount/mount_net.h:147: warning: nested extern declaration of 'copy_rom_media'
include/mount/mount_net.h:250: error: 'map_title_id' undeclared (first use in this function)
include/mount/mount_net.h:250: error: (Each undeclared identifier is reported only once
include/mount/mount_net.h:250: error: for each function it appears in.)
In file included from include/www/www_client.h:566,
                 from main.c:480:
include/www/../cmd/pad_combo_play.h: In function 'do_web_command':
include/www/../cmd/pad_combo_play.h:111: error: 'map_title_id' undeclared (first use in this function)
make: *** [objs/main.ppu.o] Error 1
Could Not Find C:\Users\u\Downloads\webMAN-MOD-master\webftp_server.elf
Could Not Find C:\Users\u\Downloads\webMAN-MOD-master\webftp_server.prx
Could Not Find C:\Users\u\Downloads\webMAN-MOD-master\webftp_server.sym

[aldostools]

Well not all flags combinations are allowed or were tested. Anyway I'll try to fix this one.

[sunnyqeen]

Yes, it has the compress.c module, but for some reason that I haven't been able to make it work.

That's why I said that maybe I'm overlooking some option.

The problem may not only be the ntfs library, also how webman works. From my limit knowledge, when gaming, the rawsecios plugin is used for reading the iso files stored on the USB disk. It just reads the raw sectors of the disk, ntfs lib is not used, so any compressed format is not supported.

to make it work, you need to fix ntfs lib, and then write a new plugin that could do directly file reading with ntfs lib.
you could check netiso and rawseciso plugin source code to see how it works.

[aldostools]

Ah it makes sense... I think you're right. I still have to check if IRISMAN's File Manager is able to read compressed files, since it uses the ntfslib to access the file system. If that works, maybe adding the ntfslib_prx to rawseciso and accessing the ISO through the library instead of using the raw sectors could use the compression.

BTW I know very well how netiso and rawseiso work 😀

My major issue now is to find the time to implement that.