Compilation on FreeBSD

[drphilth]

I've been trying to get this to compile on FreeBSD (I'm trying to get QTGMC working in vapoursynth in a freebsd jail).

It seems like the assembler code needs some changes to have it compile on freebsd - nasm has to make a.out code rather than elf code for freebsd.

I've tried to make the required changes, and the assembler compiles now, but the object files it produces aren't right and c++ can't create libmvtools.so - it errors out with:

mvtools@sha/const-a.o: file not recognized: File format not recognized
c++: error: linker command failed with exit code 1 (use -v to see invocation)

Here's what I've changed so far: https://github.com/dubhater/vapoursynth-mvtools/compare/master...drphilth:master

I was hoping someone who understands this a bit better might be able to take a look at why the object files are truncated / corrupted?

[dubhater]

What makes you think elf32/elf64 is wrong for BSD?

[drphilth]

Well, I tried it as is first with a minor change to the buildfile to get it to run (need to specify host_system == 'freebsd', just bsd doesn't work). It compiles, but causes vapoursynth to crash (with an illegal instruction error).

Then I read the nasm documentation (https://cs.nyu.edu/courses/fall02/V22.0201-001/nasm_doc_html/nasmdoc6.html) which states that aoutb is the correct output format for freebsd.

Then I had to edit the macro in x86inc.asm because "aoutb supports no special directives, no special symbols, and only the three standard section names .text, .data and .bss" - this was causing compile errors (segment name `.rodata align=32' not recognized). I think this might have been one change too far, because while the asm code now compiles, it seems that the object files it produces are incorrect.

[dubhater]

I'm still not convinced aoutb is the right format for BSD. I'm pretty sure x264 runs on BSDs and it only uses win32/64, macho32/64, and elf32/64 formats. If MVTools compiled when you added host_system == 'freebsd' then elf32/64 is probably correct.

Let's look at the illegal instruction error instead. Chances are it's just something in MVTools not working correctly because it's rarely tested on a BSD. To figure out what the problematic instruction is you should probably compile a debug build:

meson configure --buildtype=debug
ninja

and then run the command that crashes in gdb:

gdb --args your command here
run

and when it crashes get a backtrace and also disassemble the function:

backtrace
set disassebly-flavor intel
disassemble

If the function is long it will prompt you to press enter to print more of the disassembly. You need to keep pressing enter until you see something like "=>" on the left, which points out the illegal instruction. Then paste the whole output here.

[drphilth]

I've done some more reading, and you're right. FreeBSD switched over to ELF ages ago. Lesson learned: check the date of the docs I'm reading :)

Unfortunately for this particular issue, once I compiled a debug build the problem went away. So I'm going to close the bug for now, and I'll see what happens if I ever get the rest of vapoursynth to work properly.

[dubhater]

You can run a release build in gdb. The backtrace maybe won't be useful enough, but maybe it will be. I can still have a look, if you want.