icegood
commented
1 month ago As i understand the issue is with x264 sublibrary. will try to upgtade it and let you know about result
Open icegood opened 1 month ago
icegood
commented
1 month ago As i understand the issue is with x264 sublibrary. will try to upgtade it and let you know about result
th0ma7
commented
1 month ago it seems the github mirror for x264 is no longer in sync with the official repository. I've switched back to https://code.videolan.org/videolan/x264 using latest commit.
Once test builds complete you should be able to test out the resulting ffmpeg packages at https://github.com/SynoCommunity/spksrc/pull/6177
th0ma7
commented
1 month ago Builds almost complete, you'll find a package for your arch here: https://github.com/SynoCommunity/spksrc/actions/runs/10021281364
icegood
commented
1 month ago Sorry, @th0ma7 . It will not help :( Already checked against debug versions of latest x264 and still old ffmpeg6. Obtained the same with the stacktrace:
#0 0x40688b26 in psy_3gpp_analyze_channel () at libavcodec/aacpsy.c:682
#1 0x40689738 in psy_3gpp_analyze () at libavcodec/aacpsy.c:855
#2 0x4067d372 in aac_encode_frame () at libavcodec/aacenc.c:991
#3 0x407b0d60 in ff_encode_encode_cb () from /home/ice/external/synology/analysis/exec/lib/libavcodec.so.60
#4 0x407b0fc8 in encode_receive_packet_internal () from /home/ice/external/synology/analysis/exec/lib/libavcodec.so.60
#5 0x407b11a8 in avcodec_send_frame () from /home/ice/external/synology/analysis/exec/lib/libavcodec.so.60
#6 0x4011a3a6 in encode_frame () at fftools/ffmpeg.c:910
#7 0x4011be5c in reap_filters () at fftools/ffmpeg.c:1052
#8 0x4011d404 in transcode () at fftools/ffmpeg.c:4039
#9 0x400fef24 in main () at fftools/ffmpeg.c:4220
> will continue investigation. Let's go to discord to and discuss build-related stuff....
icegood
commented
1 month ago The things getting worse. I debugged against eligible gdb in https://global.synologydownload.com/download/ToolChain/toolchain/7.1-42661/Marvell%20Armada%20370%20Linux%203.2.101/armada370-gcc445_glibc211_softfp_armada370-GPL.txz
however our packages are compiled against https://global.synologydownload.com/download/ToolChain/toolchain/7.1-42661/Marvell%20Armada%20370%20Linux%203.2.101/armada370-gcc850_glibc226_hard-GPL.txz and it doesn't have gdb.
So, stacktrace above is not true.
I wonder, why they miss gdb. Additionally will ask community to compile gdbserver for platform. It would be even better....
th0ma7
commented
1 month ago As mentionned here https://github.com/SynoCommunity/spksrc/issues/6188#issuecomment-2267103887 we do provide our own gdb as we faced this issue a few times and ended-up building our own. Hope this helps.
icegood
commented
3 weeks ago Ok, so I'm ready to explain what went wrong and how to fix it. In the attachment, you may find a 'buggy' session (not from the beginning) with a segfault. The code was compiled with everything for debugging (-fsanitize=address -fsanitize=undefined -fstack-protector-all -ggdb3 -g3) but all of the utilities/sanitizers don't help much.
It happens because right after step 8 (segfault) we have:
pc 0x4a5fe540
and the issue is that instruction under this address is not supposed to be executed at all. It is meaningless in a given context.
Everything was fine in step 6
=> 0x4a5fe1b0 <psy_3gpp_init+83052>: add.w r3, sp, #1744 @ 0x6d0
and accordingly to listing in step 10 (as a continuation for 6) instructions to execute nearby 0x4a5fe540
should be
0x4a5fe53e <psy_3gpp_init+83962>: add.w r3, sp, #1872 @ 0x750 0x4a5fe542 <psy_3gpp_init+83966>: subs r3, #60 @ 0x3c
Not i wonder, why the processor in the initial code was supposed to execute instructions on 2-byte-aligned addresses. And seems, it is the purpose of thumb
And i don't know, at which stage the program decides to align ip to 4byte boundary back
(we have suspicious
0x4a5fe4c6 <psy_3gpp_init+83842>: blx 0x4a810b38 <____addvsi3_from_thumb>)
or it happens after thread re-scheduling (from another thread that executes smth in non-thumb mode, e.g. under x264 library that was compiled w/o thumb). Whenever it happens, it shouldn't happen. I compiled the program w/o thumb in ffmpeg itself and no segfault anymore. Now I'm trying to find out a proper way to leave thumb mode for my cpu which supports it.
Action Items so far: 1) From all bunch of stuff I did locally I will probably apply only changes in the build system that allow easy and properly built debug packages.
2) Regarding the bug itself.... In my opinion, all CPU-depended stuff shouldn't be resolved on package level. If some package decides to do something about it, then it should be disabled in the configure level but everything we need should be applied in the toolchain level instead. Like (regarding thumb flag) already done in https://github.com/SynoCommunity/spksrc/blob/master/toolchain/syno-alpine4k-7.2/Makefile#L11 Of course, some options should still be present on the package level as well (like ignoring assembler units or adding precompiler flags), but that stuff should be minimized.
3) My error occured in many different places. Firstly under gbb it occurs in code with many fpu instructions nearby and then i realized that something wrong with ABI. But on that moment i only found inconsistency for fpu setting.
My proc has toolchain settings here while toolchain itself was compiled against:
[DEBUG] CT_ARCH_ARCH="armv7-a+mp+sec" [DEBUG] CT_ARCH_FPU="vfpv2"
Could someone raise a question to synology to recompile toolchain properly?
Should be
[DEBUG] CT_ARCH_ARCH="armv7-a+mp+sec+fp" [DEBUG] CT_ARCH_FPU="vfpv3-d16"
for my cpu:
Architecture: armv7l Byte Order: Little Endian CPU(s): 1 On-line CPU(s) list: 0 Vendor ID: Marvell Model name: PJ4/PJ4b Model: 1 Thread(s) per core: 0 Core(s) per socket: 0 Socket(s): 0 Stepping: 0x1 BogoMIPS: 1196.85 Flags: swp half thumb fastmult vfp edsp vfpv3 vfpv3d16 tls
,
Hadn't found issues regarding this ABI inconsistency thought. I expect only some optimizations in, probably, math units. However, somewhere found an inconsistency with neon support and it definitely must be removed.
Hope, it helps to someone if they decide to optimize them NASes
th0ma7
commented
3 weeks ago I wonder, your NAS DS414slim runs with armada370 which is no longer supported with DSM 7.2. So related to your third point I doubt Synology will ever rework this out, further as it runs a 3.2 kernel.
Still, currently your arch is being built with armv7 grouping, while clearly it is of a armv7l type from your CPU output. Two things comes to mind:
make arch-armada370-7.1)? or was it a generic armv7 (make arch-armv7-7.1)? If it was generic then I suggest you rebuild with the specific.ARMv7L_ARCHS definition, rebuilding and check if it works out?... further looking, there isn't much ARMv7 nor ARMv7l specific enablement under the cross/ffmpeg6/Makefile so impact may be somewhat limited. Although one thing in particular caught my attention, the --disable-vfp which may need to not be set, or changed to be -cpuflags vfpv3 (from https://www.ffmpeg.org/ffmpeg.html).
icegood
commented
3 weeks ago I usually build only via:
make V=1 arch-armada370-7.1
And the issue in is that the line
https://github.com/SynoCommunity/spksrc/blob/master/cross/ffmpeg6/Makefile#L216
twice buggy for me:
1) I don't have neon
2) I have thumb, but it is buggy for me due, probably, toolchain issues...
th0ma7
commented
3 weeks ago Have a look at line 216 of cross/ffmpeg6/Makefile. Play with those relatively to neon and vfp. Also much earlier there is another switch for asm. Maybe you'll be able to find a working configuration that we can create proper ifeq calls later on.
icegood
commented
3 weeks ago I already found a working configuration by eliminating -mthumb. And it works fine. Moreover, I narrowed issue to instructions only (it is neither related to some calls or thread scheduling). Please, find attachment: broken_context.txt In 0x40768c5a everything is fine and it somehow got to 0x40768c68 and failed there. Interesting fact, issue happens at 5041th occurrence of given code.
Next, from Thumb2 Standard:
Most Thumb 32-bit instructions cannot use the PC as a source or destination register. Instead, if a register
is specified as 0b1111 in an instruction encoding, the instruction is a special case instruction. If an
instruction definition does not specify otherwise, the instruction is UNPREDICTABLE if a register is specified
as 0b1111. See Usage of 0b1111 as a register specifier in 32-bit encodings on page 3-38 for more
information.And this is what happens. So , GCC from toolchain is buggy. And if before I hesitated to leave '-mthumb' on toolchain level or not, now I know: it MUST be disabled at all.
Is this a new Bug?
Package Name
ffmpeg6
Package Version
6.0.1-3
Device Model
DS414slim
Device Architecture
ARMv7
Firmware Version
Version: 7.1.1-42962 Update 6
What happened?
transformation fails with segmentation fault from mpg to mp4
Reproduction steps
OK to transform to avi. Output is:
NOT OK to transform to mp4. Output is:
Install Log
Service Log
Other Logs