NNedi3 not behaving well in some edge cases with kanji

[dscp46]

Hi devs,

I've recently discovered your project and I've been quite impressed with the results nnedi3 has been producing overall.

However, I've noticed a few corner cases on which it tends to perform under the competition, especially when there are horizontal features in some kanji, and in a specific case (refer to sample 2), some form of aliasing completely changed the character.

For reference, here is the command line used to generate the following samples:

NNEDI_WEIGHTS="/usr/share/nnedi3/nnedi3_weights.bin"
ffmpeg -i ../source/A1_t00.mkv -color_primaries smpte170m -color_trc smpte170m -colorspace smpte170m -vf yadif -c:v libx265 -preset veryslow -crf 16 -x265-params aq-mode=3:aq-strength=0.9:psy-rd=1.2:deblock=-1,-1,level=4.1:colorprim=smpte170m:transfer=smpte170m:colormatrix=smpte170m:limit-sao=1 -pix_fmt yuv420p10le -c:a flac -compression_level 12 ep01.mkv
ffmpeg -i ../source/A1_t00.mkv -color_primaries smpte170m -color_trc smpte170m -colorspace smpte170m -vf nnedi=weights=${NNEDI_WEIGHTS}:nns=n128:pscrn=new3 -c:v libx265 -preset veryslow -crf 16 -x265-params aq-mode=3:aq-strength=0.9:psy-rd=1.2:deblock=-1,-1,level=4.1:colorprim=smpte170m:transfer=smpte170m:colormatrix=smpte170m:limit-sao=1 -pix_fmt yuv420p10le -c:a flac -compression_level 12 ep01_nnedi.mkv

Sample 1

Sample 2

Should I try different settings, or might this need to re-generate the weights.bin file? How many samples would you need if applicable?

Kind regards

[dubhater]

I can't help you with that. I only copied the code and the weights from the original project: https://forum.doom9.org/showthread.php?t=147695 (thread by user "tritical" titled "nnedi2/nnedi3/eedi3").

[dscp46]

Thanks for the info, at least, I got a lead to follow (^^) (in our specific case, I've figured out while comparing odd and even fields that the content was a 3:2 pulldown, so I ended up doing IVTC. Sorry for the noise)