search

ngscopeclient / scopehal-apps

ngscopeclient and other client applications for libscopehal.
https://www.ngscopeclient.org/
BSD 3-Clause "New" or "Revised" License
608 stars 100 forks source link

Packaging: ngscopeclient packaging and (eventually) CPack and the Windows and MacOS equivalent #743

Open Johnsel opened 1 month ago

Johnsel commented 1 month ago

Intro

We need to make our linux build outputs CPackable so we can generate distro-specific packages that pull in external dependencies automagically. Afterwards we need to fix Windows MSYS DLLs + installer We also need to do the same for MacOS, not sure how yet.

Linux distros:

Definitely:

Feedback on which distro of the three would be useful.

Maybe:

Definition of task/completion

Primary: ngscopeclient needs to be able to be built using the instructions from the documentation and a full build needs to result. We likely need to write out dependency lists per distro Secondary: We need to be able to run CPack to package this build and have dependency lists be part of the package meta-info.

Relevant other issues:

141

309

Maybe relevant:

497

611

bvernoux commented 1 month ago

An easy solution to include Windows DLL dependencies is to use my open source tool mingw-bundledlls at end of the build in the directory of ngscopeclient.exe

export MINGW_BUNDLEDLLS_SEARCH_PATH="C:\msys64\mingw64\bin;."
cd ./
./mingw-bundledlls.exe ./ngscopeclient.exe --copy

After that all DLL dependencies will be copied near ngscopeclient.exe and ngscopeclient could be launched from any Windows computer (like a portable executable) without requiring to have MSYS2/Mingw64 installed ...

Johnsel commented 1 month ago

Thanks for your input @bvernoux. I'm a little hesitant to pull in external dependencies. What is the least invasive way to use your tool? Export the search path and download the .exe and run it then delete again?

Ideas? Maybe you can release an .exe instead of a compressed file as well. This would be helpful for an automated process, since we can rely on github to host it and always be available. There used to be a link to a download on your personal website in the CMake which felt, well, kind of dirty, haha.

bvernoux commented 1 month ago

Solution1

on: push: pull_request: workflow_dispatch:

env: VULKAN_SDK_VERSION: 1.3.275.0

jobs: Windows: runs-on: windows-latest

defaults:
  run:
    shell: msys2 {0}

strategy:
  matrix:
    include:
      - { sys: mingw64, env: x86_64 }
      - { sys: ucrt64,  env: ucrt-x86_64 }
      - { sys: clang64, env: clang-x86_64 }

steps:
- uses: actions/checkout@v4
  with:
    submodules: recursive
    fetch-depth: 0

- uses: msys2/setup-msys2@v2
  with:
    update: true
    msystem: ${{matrix.sys}}
    install: >-
      git
      wget
    pacboy: >-
      cmake:p
      toolchain:p
      libsigc++:p
      cairomm:p
      yaml-cpp:p
      glfw:p
      catch:p
      vulkan-headers:p
      vulkan-loader:p
      shaderc:p
      glslang:p
      spirv-tools:p
      ffts:p

- name: Install mingw-bundledlls
  shell: msys2 {0}
  run: |
    wget https://github.com/bvernoux/mingw-bundledlls/releases/download/v0.2.4/mingw-bundledlls-0.2.4-1-x86_64.pkg.tar.zst
    pacman -U mingw-bundledlls-0.2.4-1-x86_64.pkg.tar.zst --noconfirm

- name: Build
  run: |
    mkdir build
    cd build
    cmake .. -GNinja
    ninja

- name: Bundle DLLs
  shell: msys2 {0}
  run: |
    export MINGW_BUNDLEDLLS_SEARCH_PATH="/mingw64/bin;."
    cd build
    mingw-bundledlls.exe ./ngscopeclient.exe --copy

- name: Test
  if: ${{ false }}
  shell: msys2 {0}
  run: |
    ninja test

- name: Upload Artifacts
  uses: actions/upload-artifact@v4
  with:
    name: glscopeclient-build-windows-${{ runner.os }}-${{ matrix.env }}-${{ github.job }}
    path: build

# - name: Upload Artifacts
#   uses: actions/upload-artifact@v4
#   with:
#     name: glscopeclient-windows-${{ runner.os }}-${{ github.job }}
#     path: msys2/*.zst

# - name: Upload Artifacts (ngscopeclient portable zip)
#   uses: actions/upload-artifact@v4
#   with:
#     name: ngscopeclient-windows-portable-${{ runner.os }}-${{ github.job }}
#     path: build/dist/ngscopeclient*.zip

# - name: Upload Artifacts (ngscopeclient MSI)
#   uses: actions/upload-artifact@v4
#   with:
#     name: ngscopeclient-${{ runner.os }}-${{ github.job }}.msi
#     path: build/dist/ngscopeclient*.msi

Solution2
- You can extract the exe from the zip from latest version https://github.com/bvernoux/mingw-bundledlls/releases and execute it ... (there is no any external dependencies)

Solution3
- You can just rebuild the exe from source it is very basic with a Makefile (you can even integrate the source code in the project the license is MIT License so fully compatible ...)
Johnsel commented 1 month ago

Hi @bvernoux,

Thanks for the extensive input. I see some valuable input indeed, hoewever the main question if it is possible to pull in a mingw-bundledlls.exe from your releases is not answered. Do you have any dependencies that would prevent this?

Thanks John

Johnsel commented 1 month ago

Solution1

name: Build-windows

on:
  push:
  pull_request:
  workflow_dispatch:

env:
  VULKAN_SDK_VERSION: 1.3.275.0

jobs:
  Windows:
    runs-on: windows-latest

    defaults:
      run:
        shell: msys2 {0}

    strategy:
      matrix:
        include:
          - { sys: mingw64, env: x86_64 }
          - { sys: ucrt64,  env: ucrt-x86_64 }
          - { sys: clang64, env: clang-x86_64 }

    steps:
    - uses: actions/checkout@v4
      with:
        submodules: recursive
        fetch-depth: 0

    - uses: msys2/setup-msys2@v2
      with:
        update: true
        msystem: ${{matrix.sys}}
        install: >-
          git
          wget
        pacboy: >-
          cmake:p
          toolchain:p
          libsigc++:p
          cairomm:p
          yaml-cpp:p
          glfw:p
          catch:p
          vulkan-headers:p
          vulkan-loader:p
          shaderc:p
          glslang:p
          spirv-tools:p
          ffts:p

    - name: Install mingw-bundledlls
      shell: msys2 {0}
      run: |
        wget https://github.com/bvernoux/mingw-bundledlls/releases/download/v0.2.4/mingw-bundledlls-0.2.4-1-x86_64.pkg.tar.zst
        pacman -U mingw-bundledlls-0.2.4-1-x86_64.pkg.tar.zst --noconfirm

    - name: Build
      run: |
        mkdir build
        cd build
        cmake .. -GNinja
        ninja

    - name: Bundle DLLs
      shell: msys2 {0}
      run: |
        export MINGW_BUNDLEDLLS_SEARCH_PATH="/mingw64/bin;."
        cd build
        mingw-bundledlls.exe ./ngscopeclient.exe --copy

    - name: Test
      if: ${{ false }}
      shell: msys2 {0}
      run: |
        ninja test

    - name: Upload Artifacts
      uses: actions/upload-artifact@v4
      with:
        name: glscopeclient-build-windows-${{ runner.os }}-${{ matrix.env }}-${{ github.job }}
        path: build

    # - name: Upload Artifacts
    #   uses: actions/upload-artifact@v4
    #   with:
    #     name: glscopeclient-windows-${{ runner.os }}-${{ github.job }}
    #     path: msys2/*.zst

    # - name: Upload Artifacts (ngscopeclient portable zip)
    #   uses: actions/upload-artifact@v4
    #   with:
    #     name: ngscopeclient-windows-portable-${{ runner.os }}-${{ github.job }}
    #     path: build/dist/ngscopeclient*.zip

    # - name: Upload Artifacts (ngscopeclient MSI)
    #   uses: actions/upload-artifact@v4
    #   with:
    #     name: ngscopeclient-${{ runner.os }}-${{ github.job }}.msi
    #     path: build/dist/ngscopeclient*.msi

Solution2

Solution3

  • You can just rebuild the exe from source it is very basic with a Makefile (you can even integrate the source code in the project the license is MIT License so fully compatible ...)

@d235j what do you think, is this acceptable. I'm nudging towards yes, even though I'd prefer just an .exe and worry about cross-platform building. Have not thought about this project for a while though so may be thinking up issues that don't exist