animetosho / ParPar

High performance PAR2 create client for NodeJS
190 stars 19 forks source link

ParPar removes empty nested directories #48

Closed Ravencentric closed 11 months ago

Ravencentric commented 11 months ago

If i par the following directory: parpar -s1400K -m1024M --threads 2 --slice-size-multiple=700K --max-input-slices=2000 -r1n -R -o "[EAC][170215] Porter Robinson & Madeon - SHELTER" "[EAC][170215] Porter Robinson & Madeon - SHELTER"

[EAC][170215] Porter Robinson & Madeon - SHELTER
├── [EAC][170215] Porter Robinson & Madeon - SHELTER
│   └── BDROM
│       └── PORTER_ROBINSON_MADEON_SHELTER
│           ├── BDMV
│           └── CERTIFICATE
├── [EAC][170215] Porter Robinson & Madeon - SHELTER.par2
├── [EAC][170215] Porter Robinson & Madeon - SHELTER.vol000+001.par2
├── [EAC][170215] Porter Robinson & Madeon - SHELTER.vol001+002.par2
├── [EAC][170215] Porter Robinson & Madeon - SHELTER.vol003+004.par2
├── [EAC][170215] Porter Robinson & Madeon - SHELTER.vol007+008.par2
├── [EAC][170215] Porter Robinson & Madeon - SHELTER.vol015+016.par2
├── [EAC][170215] Porter Robinson & Madeon - SHELTER.vol031+032.par2
└── [EAC][170215] Porter Robinson & Madeon - SHELTER.vol063+062.par2

then uploaded it with nyuu

nyuu.exe -C config-sample.json -o "[EAC][170215] Porter Robinson & Madeon - SHELTER.nzb" "[EAC][170215] Porter Robinson & Madeon - SHELTER"
[2023-08-14 00:46:30.200][INFO] Uploading 2815 article(s) from 32 file(s) totalling 1905.05 MiB
[2023-08-14 00:46:30.201][INFO] Reading file 00000.clpi...
[2023-08-14 00:46:30.205][INFO] Reading file 00001.clpi...
[2023-08-14 00:46:30.206][INFO] Reading file 00002.clpi...
[2023-08-14 00:46:30.206][INFO] Reading file 00003.clpi...
[2023-08-14 00:46:30.207][INFO] Reading file index.bdmv...
[2023-08-14 00:46:30.207][INFO] Reading file MovieObject.bdmv...
[2023-08-14 00:46:30.207][INFO] Reading file 00000.mpls...
[2023-08-14 00:46:30.207][INFO] Reading file 00001.mpls...
[2023-08-14 00:46:30.209][INFO] Reading file 00002.mpls...
[2023-08-14 00:46:30.209][INFO] Reading file 00000.clpi...
[2023-08-14 00:46:30.209][INFO] Reading file 00001.clpi...
[2023-08-14 00:46:30.210][INFO] Reading file 00002.clpi...
[2023-08-14 00:46:30.210][INFO] Reading file 00003.clpi...
[2023-08-14 00:46:31.238][INFO] Reading file index.bdmv...
[2023-08-14 00:46:31.239][INFO] Reading file MovieObject.bdmv...
[2023-08-14 00:46:31.249][INFO] Reading file 00000.mpls...
[2023-08-14 00:46:31.257][INFO] Reading file 00001.mpls...
[2023-08-14 00:46:31.261][INFO] Reading file 00002.mpls...
[2023-08-14 00:46:31.264][INFO] Reading file 00000.m2ts...
[2023-08-14 00:46:38.813][INFO] Reading file 00001.m2ts...
[2023-08-14 00:46:38.826][INFO] Reading file 00002.m2ts...
[2023-08-14 00:49:24.860][INFO] Reading file 00003.m2ts...
[2023-08-14 00:49:25.153][INFO] Reading file id.bdmv...
[2023-08-14 00:49:25.172][INFO] Reading file id.bdmv...
[2023-08-14 00:49:25.221][INFO] Reading file [EAC][170215] Porter Robinson & Madeon - SHELTER.par2...
[2023-08-14 00:49:25.334][INFO] Reading file [EAC][170215] Porter Robinson & Madeon - SHELTER.vol000+001.par2...
[2023-08-14 00:49:25.457][INFO] Reading file [EAC][170215] Porter Robinson & Madeon - SHELTER.vol001+002.par2...
[2023-08-14 00:49:25.848][INFO] Reading file [EAC][170215] Porter Robinson & Madeon - SHELTER.vol003+004.par2...
[2023-08-14 00:49:26.383][INFO] Reading file [EAC][170215] Porter Robinson & Madeon - SHELTER.vol007+008.par2...
[2023-08-14 00:49:27.277][INFO] Reading file [EAC][170215] Porter Robinson & Madeon - SHELTER.vol015+016.par2...
[2023-08-14 00:49:29.533][INFO] Reading file [EAC][170215] Porter Robinson & Madeon - SHELTER.vol031+032.par2...
[2023-08-14 00:49:33.743][INFO] Reading file [EAC][170215] Porter Robinson & Madeon - SHELTER.vol063+062.par2...
[2023-08-14 00:49:42.235][INFO] All file(s) read...
[2023-08-14 00:49:51.776][INFO] Finished uploading

and then download it with Sabnzbd

The resulting reconstruction looks like this

[EAC][170215] Porter Robinson & Madeon - SHELTER
├── BDMV
└── CERTIFICATE

I'm trying to avoid raring large files like BDMVs which is why I'm looking into the par reconstruction. The directory structure is important and needs to be kept 1:1 for it to work and maintain the cross-seed ability. The above probably becomes a bigger problem for multi discs with nested folders although I haven't done any extensive testing on it

animetosho commented 11 months ago

I presume you're using the patched version here?

Could you open the PAR2 in something like MultiPar to check that it has the correct structure? If the PAR2 is correct, you'll need to check that the downloader is acting correctly.

Note that many Usenet downloaders/tools don't like folder structures. Nyuu/ParPar should deal with them fine, but you may need to convince others to support it.

Ravencentric commented 11 months ago

image [[EAC][170215] Porter Robinson & Madeon - SHELTER.zip]

I don't exactly understand what I'm looking at.

I'm running sabnzbd 4.1.0-develop 80f34bd on Windows

Here's the NZB [EAC][170215] Porter Robinson & Madeon - SHELTER.nzb.zip if you want to test it on your end and see if I'm doing something wrong. (.zip because GitHub won't allow .nzb)

animetosho commented 11 months ago

Do the paths in the PAR2 look correct?

If you're looking to include parent directories in the path, you'll need to specify what you want via the --filepath-format/--filepath-base options.

Ravencentric commented 11 months ago

Do the paths in the PAR2 look correct?

Nope. Looking at an example file such as [EAC][170215] Porter Robinson & Madeon - SHELTER\BDROM\PORTER_ROBINSON_MADEON_SHELTER\BDMV\BACKUP\CLIPINF\00000.clpi where PAR2 had it as BDMV\BACKUP\CLIPINF\00000.clpi

You can see this in MultiPar gif I posed above where "rename" restores EAC][170215] Porter Robinson & Madeon - SHELTER\BDROM\PORTER_ROBINSON_MADEON_SHELTER\BDMV to BDMV\

--filepath-format

--filepath-format keep and --filepath-format outrel both make PAR2 store the required path and fix said issue. Thank you! It's now reconstructing it 1:1. So this means the default (common) was discarding them but I have a question as to what exactly is considered the "common" part of the path

animetosho commented 11 months ago

If we have the following three files:

All these files' paths start with 'a/b', which is considered the 'common' path, and would get stripped off.

Ravencentric commented 11 months ago

Okay that makes sense. Again, thank you very much!