Mac/Linux support (ongoing)

[petmac]

Hello,

I'm a Mac user- wait! don't close the issue yet!

As commented in previous issues, adding support for Linux/Mac/whatever is a huge chunk of work, especially the FS-UAE changes. But would it be OK if we took baby steps to get in that general direction?

Say, I could submit a PR to build Windows executables using GitHub Actions, then try to add more platforms once we're happy with that.

We could leave debugging off the table at first, and work towards it. Doesn't have to be all-or-nothing.

Thanks Peter

[BartmanAbyss]

If you're doing the work, sure...

[petmac]

I gave it a try last night, starting with the Linux build as I figured that's maybe easier than trying to get WSL working on GitHub Actions.

Hit a snag building GCC but I'll keep investigating.

https://github.com/petmac/vscode-amiga-debug/runs/7291936309

[BartmanAbyss]

that error is expected, see README.md. Basically, it tries to execute the new cross-compiler to get a specs file, but is missing the '.exe' extension

[petmac]

Thanks. Is that because I'm building on Linux but I've built a MinGW exe?

I'm not going to be able to run the exe on a Linux-hosted GitHub runner. Maybe I need to switch to building in WSL, or targeting Linux.

You're referring to the following I think, where you work around it and restart the build:

gcc/gcc-cross.exe -dumpspecs >gcc/specs
make all-gcc -j16

It would be nice if I can get to a place where none of the commands fail.

I was wondering what this bit is about:

sed 's/selftest # srcextra/# selftest srcextra/' gcc/Makefile >gcc/Makefile.tmp
mv gcc/Makefile.tmp gcc/Makefile

Is it just an optimisation to skip an unwanted part of building GCC? (selftest)

[BartmanAbyss]

Yes, if you're building on linux for linux, you shouldn't use MinGW. Somehow those selftests failed for me, so that line just comments them out.

[petmac]

I had to remove the step with sed, as something couldn't find the temp Makefile.

https://github.com/petmac/vscode-amiga-debug/runs/7310364546

sed 's/selftest # srcextra/# selftest srcextra/' gcc/Makefile >gcc/Makefile.tmp
mv gcc/Makefile.tmp gcc/Makefile
(snip)
gcc/Makefile.tmp: No such file or directory

Maybe the sed command didn't find the pattern searched for, and didn't print anything out to gcc/Makefile.tmp.

On the plus side, nothing complained about the selftests. So it might be worth retrying on WSL, maybe the selftests are now working.

[BartmanAbyss]

I think the selftests have the same problem as the spec file (not adding '.exe' extension). That shouldn't be an issue on a real linux, though.

[petmac]

Thanks. I got GCC building under Linux. I think I need to fiddle a bit with the strip command.

https://github.com/petmac/vscode-amiga-debug/actions/runs/2666640330

Next up, I'll look at elf2hunk.

Is porting Shrinkler required?

[BartmanAbyss]

probably low priority, but shouldn't be an issue..

[petmac]

I hope you don't mind, I submitted a PR for elf2hunk, to get it compiling for Linux and Mac.

https://github.com/BartmanAbyss/elf2hunk/pull/2

[BartmanAbyss]

sure. merged.

[petmac]

That's great, thanks.

I wonder if we can shorten the build time at all. Binutils+GDB takes about 11 minutes just for the build step, and GCC takes about 16 minutes.

Maybe we don't need to build all of GCC?

Before I move on to Windows/WSL or Mac, would it make sense to open a PR so you can see if you're happy with the overall approach so far?

[BartmanAbyss]

Do you think it's worth investing time into researching how to get the build times down? Usually I only build GCC & GDB once a year when a new version is released. Sure, open the PR.

[BartmanAbyss]

Hey, it seems your strip call isn't working for all executables. See https://github.com/BartmanAbyss/vscode-amiga-debug/pull/107#pullrequestreview-1046477440

[petmac]

Fixed in #108

[petmac]

I found, too late, that GCC's Makefile has an "install-strip" target, which I assume does a strip then install. So that might be the best way to do this.

https://gcc.gnu.org/install/finalinstall.html

[BartmanAbyss]

Nice. Also for gdb?

[petmac]

I made a couple of discoveries last night:

1. On Mac, you can build GCC with Clang. I had assumed you need GCC to build GCC, and I wasted hours trying various flags and alternative versions of programs, but you don't need it. I'll clean up my Mac branch and submit a PR soon.
2. The "upload artifacts" step strips files of their permissions, meaning executables no longer have the +x permission set. The solution seems to be to tar the files first, then upload. So I'll make a PR that fixes this for Linux, and it will fix it for Mac too.

[petmac]

Nice. Also for gdb?

Sorry, I missed this. Yes I think so. The m68k-amiga-elf-gdb executable is large, but it is stripped. I tried stripping the Mac one again locally and the size didn't change.

[petmac]

Thanks for merging the Mac toolchain PR.

I was thinking if building the Windows toolchain is a pain to do automatically, could you please continue to build it manually?

If it's OK, I would like to restructure the bin folder a bit, so the Windows tools can be in one folder, Mac in another, etc.

bin/
├── dh0/
├── symbols/
└── windows/ (renamed from opt)
    ├── bin/
    ├── lib/
    ├── libexec/
    ├── m68k-amiga-elf/
    ├── elf2hunk.exe (moved from bin to windows)
    ├── gnumake.exe (moved from bin to windows)
    ├── Shrinkler.exe (moved from bin to windows)
    └── winuae-gdb.exe (moved from bin to windows)
gcc-barto.patch (moved from bin to root)

Code and config in the extension that refer to the Windows tools or the opt folder would need to be updated too.

[BartmanAbyss]

I think we have to do this a bit differently. There‘s amiga include files in bin/opt/m68k-amiga-elf/sys-includes. We don‘t want them to be duplicated for every platform.

[petmac]

That's a good point. Are those files from GCC, or did you add them manually?

From the GCC docs it looks like there's a configure option (--exec-prefix) that might do what we want, but I'd need to try it first to see.

[BartmanAbyss]

Those are from the Amiga NDK and have been modified by me. They need to be found by #include <...> without needing to change any compiler settings.

[petmac]

I was able to use --exec-prefix to build a Mac toolchain that can almost build the template demo, after some hacks to the Makefile of course. It falls over while linking:

m68k-amiga-elf-gcc -g -MP -MMD -m68000 -Ofast -nostdlib -Wextra -Wno-unused-function -Wno-volatile-register-var -fomit-frame-pointer -fno-tree-loop-distribution -flto -fwhole-program -fno-exceptions -Wl,--emit-relocs,-Ttext=0,-Map=a.mingw.map  obj/main.o obj/gcc8_c_support.o obj/gcc8_a_support.o obj/depacker_doynax.o -o a.mingw.elf
/var/folders/02/yhrf693n3gn6fbx5r9t70b2m0000gn/T//cciOLwlX.s: Assembler messages:
/var/folders/02/yhrf693n3gn6fbx5r9t70b2m0000gn/T//cciOLwlX.s:19: Error: register save offset not a multiple of 4
/var/folders/02/yhrf693n3gn6fbx5r9t70b2m0000gn/T//cciOLwlX.s:20: Error: register save offset not a multiple of 4
/var/folders/02/yhrf693n3gn6fbx5r9t70b2m0000gn/T//cciOLwlX.s:21: Error: register save offset not a multiple of 4
/var/folders/02/yhrf693n3gn6fbx5r9t70b2m0000gn/T//cciOLwlX.s:22: Error: register save offset not a multiple of 4
/var/folders/02/yhrf693n3gn6fbx5r9t70b2m0000gn/T//cciOLwlX.s:23: Error: register save offset not a multiple of 4
/var/folders/02/yhrf693n3gn6fbx5r9t70b2m0000gn/T//cciOLwlX.s:24: Error: register save offset not a multiple of 4
lto-wrapper: fatal error: m68k-amiga-elf-gcc returned 1 exit status

Drat!

[petmac]

A temporary workaround is to disable debug info by using -g0 instead of -g.

[BartmanAbyss]

yeah, but that shouldn't be necessary, and will hinder debugging...

[petmac]

I agree.

I haven't looked too deeply but it looks like it's coming from cfi_add_CFA_offset in gas. The assembler is not happy when generating DWARF2 CFI information. It's parsing some of the CFI (call frame information) directives in the generated assembly. This directive appears to adjust the save offset for a register relative to the CFA (canonical frame address), and gas is expecting the offset to be aligned to 4 bytes and it's not.

It suggests to me that either the debug info is wrong, or is being handled differently on Windows vs Mac, or the code generation is wrong?

Any thoughts would be very welcome.

[BartmanAbyss]

Did all of the selftests pass? Anything to do with the specs file? Have you tried the Linux version? This should all be 68000 specific and not host specific... Strange..

[grahambates]

I've been having a go at porting the WinUAE changes to FS-UAE. I've made a fair bit of progress on this branch: https://github.com/grahambates/fs-uae/tree/barto

Main changes so far are:

• Making the Windows socket stuff cross-platform
• Getting the build to work with newer c++ standard
• Using FS-UAE equivalents for:
  • Getting screengrab data
  • Toggling warp and pause mode

I'll keep you updated on my progress.

I'm also looking at incorporating the additional features from @prb28's fork to allow it to be used with vscode-amiga-assembly and uae-dap. Benefits of this would be having a single remote debugger implementation and potentially opening up the possibility of implementing profiling there too. https://github.com/grahambates/fs-uae/tree/barto_prb

[grahambates]

Also possibly relevant: I have a working cross-platform version of Kingcon, which I see is used in the project template.

[grahambates]

Quick update on my progress with this:

I now have a build of FS-UAE which works with the debugger and generates profile data. Debug overlays seem to work fine too. It currently runs at a low frame rate with jittery audio when the debugger is active, but does at least work. I'll see what can be done to improve this. (fixed this)

The main blocker was that the current upstream FS-UAE is based on a pretty old version of WinUAE (4.2.1) which was missing some things we need for the profile data. I've now brought my fork up to date with the current version (4.9.1), which was a lot more effort than expected!

The biggest remaining issue now is that the non-Windows build of GCC still won't generate debug symbols as discussed above with the register save offset not a multiple of 4 error. I'll see if I can help with this.

We would also need to look at the best approach for distributing platform specific binaries. I've got something simple working in my fork for the moment where the binaries are just grouped in dirs by platform, but there's a lot of duplication. I'm wondering if it would be better to replicate the approach of vscode-amiga-assembly where platform specific binaries are downloaded on demand.

[petmac]

Thank you so much for this work, it sounds great!

[grahambates]

Good news on the GCC issue. I don't see the issue if I compile it locally on my Mac. I'll see if I can figure out why this is different to the Github Actions CI output, but I'm able to build and debug the example project now.

Still TODO:

• Try to get things working on Linux. There's currently a minor header issue stopping my FS-UAE fork from building.
• Figure out a nicer solution for handling platform specific binaries
• Come up with a cross-platform compatible Makefile for the project template
• Testing!
• Get things in decent shape for a PR

[lw64]

Come up with a cross-platform compatible Makefile for the project templat

I think it would be way easier to just use build system that is platform independent by default. For example meson.

[grahambates]

I think it would be way easier to just use build system that is platform independent by default. For example meson.

The existing Makefile only needed minimal changes in the end. Thanks for the suggestion though. I've never looked at Meson before and it looks interesting.

[lw64]

I tried to install the extension from your branch by packaging it with vsce package and installing it then from the menu, but when I want to init a project it shows this error message:

[amiga23]

@lw64 Seems this is depending on ms-vscode.cpptools plugin, which is not available in VSCodium and also not in the ArchLinux/Manjaro OSS Version :-(

[grahambates]

Seems this is depending on ms-vscode.cpptools plugin, which is not available in VSCodium and also not in the ArchLinux/Manjaro OSS Version :-(

I wonder if this could be a recommendation rather than a hard dependency. From what I understand you can prompt the user to install extensions in code rather than adding them to extensionDependencies in package.json.

[BartmanAbyss]

I guess it's only there so that all the CPP intellisense stuff works.

[amiga23]

When using this extension as alternative: https://open-vsx.org/extension/llvm-vs-code-extensions/vscode-clangd I managed to run the Amiga:Init command and compile the example code. But when trying to debug I get the message "Emulator exited with code/signal SIGSEGV before debugger could connect"

diff --git a/package.json b/package.json
index 30090f9..ba9219e 100644
--- a/package.json
+++ b/package.json
@@ -559,7 +559,7 @@
                "vscode-jsonrpc": "^6.0.0"
        },
        "extensionDependencies": [
-               "ms-vscode.cpptools"
+               "llvm-vs-code-extensions.vscode-clangd"
        ],
        "vsce": {
                "dependencies": false,

[BartmanAbyss]

try again with the latest commit. I just merged everything into here, should work now (at least it did on my Ubuntu 22)

[grahambates]

Yeah hopefully that issue should be resolved now. I'm going to do some more testing with different Linux distros.

[amiga23]

The lowercase shrinkler for darwin should be removed:

 ERROR  The following files have the same case insensitive path, which isn't supported by the VSIX format:
  - extension/bin/darwin/shrinkler
  - extension/bin/darwin/Shrinkler

[BartmanAbyss]

ah great you found it. I was getting strange modified changes...

[grahambates]

The lowercase shrinkler for darwin should be removed

Oh no, I'd tried to rename it to fix the case but it looks like we ended up with both versions!

[BartmanAbyss]

I think i fixed it on my linux box.

[amiga23]

Now FS-UAE opens for a second and then disappears.

Output:

{
    "base": "$gcc",
    "fileLocation": "absolute"
}

Error: the description can't be converted into a problem matcher:
{
    "base": "$gcc",
    "fileLocation": "absolute"
}

I do see object files and there are a.out and a.exe in the out folder.

btw: after Amiga:init I had to create the subfolder obj manually.

Maybe my issue is related the the OSS/vscodium version.

[BartmanAbyss]

btw: after Amiga:init I had to create the subfolder obj manually.

should be fixed in the next commit.

[BartmanAbyss]

New version 1.6.2 released with Mac/Linux support. Closing this issue. Please open new issues for any issues (hehe)

[lw64]

@lw64 Seems this is depending on ms-vscode.cpptools plugin, which is not available in VSCodium and also not in the ArchLinux/Manjaro OSS Version :-(

Shoudl I open a new issue for this?