Change CI to create Debian Package to Release

[AndrewQuijano]

Your checklist for this pull request

• [ ] I've documented or updated the documentation of every API function and struct this PR changes.
• [X] I've added tests that prove my fix is effective or that my feature works (if possible)

This is updating the CI, so every release, this would be tested.

Detailed description

A new feature is made such that every time a new release is created, a Debian package is created and uploaded to releases.

Approach: Create a Debian folder, this contains everything needed to create a Debian package in the container. The container compiles and installs capstone, and creates the Debian package. The CI is updated to run the script which creates the container for upload release.

Also, I did slight tweaks as your release CI needed to be updated too. I updated the release process to not use end of life uploading. https://github.com/actions/upload-release-asset

Based on my hack in the Dockerfile, it seems to identify which files I need to copy in the Debian package

Note, I am copying (essentially should I copy /usr/lib/libcapstone.so too into the Debian Package?) libcapstone.a libcapstone.so libcapstone.so.6

Proof the Debian package is created

...

Test plan

See thread below, a script is created to check that certain packages/libraries are in the Debian Package before adding to release.

...

Closing issues

closes #2398 ...

[Rot127]

Note, I am copying (essentially should I copy /usr/lib/libcapstone.so too into the Debian Package?)

I think it should contain the static and shared lib file. You might need to build Capstone twice with cmake. Once for the static, once for creating the shared one.

[AndrewQuijano]

Current diff looks like this, I suspect I can ignore the /sys stuff and I suspect I can ignore the stuff under /usr/local/lib/cmake/

7526a7527,7553
> /usr/local/include/capstone/loongarch.h
> /usr/local/include/capstone/riscv.h
> /usr/local/include/capstone/x86.h
> /usr/local/include/capstone/xcore.h
> /usr/local/include/capstone/tricore.h
> /usr/local/include/capstone/platform.h
> /usr/local/include/capstone/mips.h
> /usr/local/include/capstone/hppa.h
> /usr/local/include/capstone/bpf.h
> /usr/local/include/capstone/sh.h
> /usr/local/include/capstone/wasm.h
> /usr/local/include/capstone/systemz.h
> /usr/local/include/capstone/ppc.h
> /usr/local/include/capstone/aarch64.h
> /usr/local/include/capstone/tms320c64x.h
> /usr/local/include/capstone/arm.h
> /usr/local/include/capstone/mos65xx.h
> /usr/local/include/capstone/cs_operand.h
> /usr/local/include/capstone/m68k.h
> /usr/local/include/capstone/systemz_compatibility.h
> /usr/local/include/capstone/alpha.h
> /usr/local/include/capstone/evm.h
> /usr/local/include/capstone/m680x.h
> /usr/local/include/capstone/arm64.h
> /usr/local/include/capstone/xtensa.h
> /usr/local/include/capstone/capstone.h
> /usr/local/include/capstone/sparc.h
7549a7577,7582
> /usr/local/lib/pkgconfig/capstone.pc
> /usr/local/lib/cmake/capstone/capstone-targets.cmake
> /usr/local/lib/cmake/capstone/capstone-targets-release.cmake
> /usr/local/lib/cmake/capstone/capstone-config-version.cmake
> /usr/local/lib/cmake/capstone/capstone-config.cmake
> /usr/local/lib/libcapstone.so.6.0
10138a10172
> /usr/local/bin/cstool
21455,21523d21488
< /sys/devices/virtual/net/lo/uevent
< /sys/devices/virtual/net/lo/carrier_changes
< /sys/devices/virtual/net/lo/testing
< /sys/devices/virtual/net/lo/carrier
< /sys/devices/virtual/net/lo/dev_id
< /sys/devices/virtual/net/lo/carrier_down_count
< /sys/devices/virtual/net/lo/proto_down
< /sys/devices/virtual/net/lo/address
< /sys/devices/virtual/net/lo/operstate
< /sys/devices/virtual/net/lo/phys_switch_id
< /sys/devices/virtual/net/lo/link_mode
< /sys/devices/virtual/net/lo/dormant
< /sys/devices/virtual/net/lo/statistics/tx_errors
< /sys/devices/virtual/net/lo/statistics/rx_length_errors
< /sys/devices/virtual/net/lo/statistics/rx_packets
< /sys/devices/virtual/net/lo/statistics/tx_carrier_errors
< /sys/devices/virtual/net/lo/statistics/tx_dropped
< /sys/devices/virtual/net/lo/statistics/rx_missed_errors
< /sys/devices/virtual/net/lo/statistics/rx_over_errors
< /sys/devices/virtual/net/lo/statistics/tx_aborted_errors
< /sys/devices/virtual/net/lo/statistics/rx_crc_errors
< /sys/devices/virtual/net/lo/statistics/rx_frame_errors
< /sys/devices/virtual/net/lo/statistics/rx_nohandler
< /sys/devices/virtual/net/lo/statistics/tx_fifo_errors
< /sys/devices/virtual/net/lo/statistics/multicast
< /sys/devices/virtual/net/lo/statistics/tx_packets
< /sys/devices/virtual/net/lo/statistics/tx_window_errors
< /sys/devices/virtual/net/lo/statistics/rx_bytes
< /sys/devices/virtual/net/lo/statistics/collisions
< /sys/devices/virtual/net/lo/statistics/rx_dropped
< /sys/devices/virtual/net/lo/statistics/tx_bytes
< /sys/devices/virtual/net/lo/statistics/tx_heartbeat_errors
< /sys/devices/virtual/net/lo/statistics/rx_fifo_errors
< /sys/devices/virtual/net/lo/statistics/rx_errors
< /sys/devices/virtual/net/lo/statistics/tx_compressed
< /sys/devices/virtual/net/lo/statistics/rx_compressed
< /sys/devices/virtual/net/lo/mtu
< /sys/devices/virtual/net/lo/gro_flush_timeout
< /sys/devices/virtual/net/lo/carrier_up_count
< /sys/devices/virtual/net/lo/ifalias
< /sys/devices/virtual/net/lo/speed
< /sys/devices/virtual/net/lo/netdev_group
< /sys/devices/virtual/net/lo/napi_defer_hard_irqs
< /sys/devices/virtual/net/lo/phys_port_name
< /sys/devices/virtual/net/lo/ifindex
< /sys/devices/virtual/net/lo/broadcast
< /sys/devices/virtual/net/lo/type
< /sys/devices/virtual/net/lo/dev_port
< /sys/devices/virtual/net/lo/phys_port_id
< /sys/devices/virtual/net/lo/queues/tx-0/tx_maxrate
< /sys/devices/virtual/net/lo/queues/tx-0/xps_cpus
< /sys/devices/virtual/net/lo/queues/tx-0/byte_queue_limits/limit_min
< /sys/devices/virtual/net/lo/queues/tx-0/byte_queue_limits/limit
< /sys/devices/virtual/net/lo/queues/tx-0/byte_queue_limits/limit_max
< /sys/devices/virtual/net/lo/queues/tx-0/byte_queue_limits/hold_time
< /sys/devices/virtual/net/lo/queues/tx-0/byte_queue_limits/inflight
< /sys/devices/virtual/net/lo/queues/tx-0/tx_timeout
< /sys/devices/virtual/net/lo/queues/tx-0/xps_rxqs
< /sys/devices/virtual/net/lo/queues/tx-0/traffic_class
< /sys/devices/virtual/net/lo/queues/rx-0/rps_flow_cnt
< /sys/devices/virtual/net/lo/queues/rx-0/rps_cpus
< /sys/devices/virtual/net/lo/name_assign_type
< /sys/devices/virtual/net/lo/duplex
< /sys/devices/virtual/net/lo/addr_assign_type
< /sys/devices/virtual/net/lo/addr_len
< /sys/devices/virtual/net/lo/threaded
< /sys/devices/virtual/net/lo/tx_queue_len
< /sys/devices/virtual/net/lo/iflink
< /sys/devices/virtual/net/lo/flags
21592a21558,21626
> /sys/devices/virtual/net/lo/uevent
> /sys/devices/virtual/net/lo/carrier_changes
> /sys/devices/virtual/net/lo/testing
> /sys/devices/virtual/net/lo/carrier
> /sys/devices/virtual/net/lo/dev_id
> /sys/devices/virtual/net/lo/carrier_down_count
> /sys/devices/virtual/net/lo/proto_down
> /sys/devices/virtual/net/lo/address
> /sys/devices/virtual/net/lo/operstate
> /sys/devices/virtual/net/lo/phys_switch_id
> /sys/devices/virtual/net/lo/link_mode
> /sys/devices/virtual/net/lo/dormant
> /sys/devices/virtual/net/lo/statistics/tx_errors
> /sys/devices/virtual/net/lo/statistics/rx_length_errors
> /sys/devices/virtual/net/lo/statistics/rx_packets
> /sys/devices/virtual/net/lo/statistics/tx_carrier_errors
> /sys/devices/virtual/net/lo/statistics/tx_dropped
> /sys/devices/virtual/net/lo/statistics/rx_missed_errors
> /sys/devices/virtual/net/lo/statistics/rx_over_errors
> /sys/devices/virtual/net/lo/statistics/tx_aborted_errors
> /sys/devices/virtual/net/lo/statistics/rx_crc_errors
> /sys/devices/virtual/net/lo/statistics/rx_frame_errors
> /sys/devices/virtual/net/lo/statistics/rx_nohandler
> /sys/devices/virtual/net/lo/statistics/tx_fifo_errors
> /sys/devices/virtual/net/lo/statistics/multicast
> /sys/devices/virtual/net/lo/statistics/tx_packets
> /sys/devices/virtual/net/lo/statistics/tx_window_errors
> /sys/devices/virtual/net/lo/statistics/rx_bytes
> /sys/devices/virtual/net/lo/statistics/collisions
> /sys/devices/virtual/net/lo/statistics/rx_dropped
> /sys/devices/virtual/net/lo/statistics/tx_bytes
> /sys/devices/virtual/net/lo/statistics/tx_heartbeat_errors
> /sys/devices/virtual/net/lo/statistics/rx_fifo_errors
> /sys/devices/virtual/net/lo/statistics/rx_errors
> /sys/devices/virtual/net/lo/statistics/tx_compressed
> /sys/devices/virtual/net/lo/statistics/rx_compressed
> /sys/devices/virtual/net/lo/mtu
> /sys/devices/virtual/net/lo/gro_flush_timeout
> /sys/devices/virtual/net/lo/carrier_up_count
> /sys/devices/virtual/net/lo/ifalias
> /sys/devices/virtual/net/lo/speed
> /sys/devices/virtual/net/lo/netdev_group
> /sys/devices/virtual/net/lo/napi_defer_hard_irqs
> /sys/devices/virtual/net/lo/phys_port_name
> /sys/devices/virtual/net/lo/ifindex
> /sys/devices/virtual/net/lo/broadcast
> /sys/devices/virtual/net/lo/type
> /sys/devices/virtual/net/lo/dev_port
> /sys/devices/virtual/net/lo/phys_port_id
> /sys/devices/virtual/net/lo/queues/tx-0/tx_maxrate
> /sys/devices/virtual/net/lo/queues/tx-0/xps_cpus
> /sys/devices/virtual/net/lo/queues/tx-0/byte_queue_limits/limit_min
> /sys/devices/virtual/net/lo/queues/tx-0/byte_queue_limits/limit
> /sys/devices/virtual/net/lo/queues/tx-0/byte_queue_limits/limit_max
> /sys/devices/virtual/net/lo/queues/tx-0/byte_queue_limits/hold_time
> /sys/devices/virtual/net/lo/queues/tx-0/byte_queue_limits/inflight
> /sys/devices/virtual/net/lo/queues/tx-0/tx_timeout
> /sys/devices/virtual/net/lo/queues/tx-0/xps_rxqs
> /sys/devices/virtual/net/lo/queues/tx-0/traffic_class
> /sys/devices/virtual/net/lo/queues/rx-0/rps_flow_cnt
> /sys/devices/virtual/net/lo/queues/rx-0/rps_cpus
> /sys/devices/virtual/net/lo/name_assign_type
> /sys/devices/virtual/net/lo/duplex
> /sys/devices/virtual/net/lo/addr_assign_type
> /sys/devices/virtual/net/lo/addr_len
> /sys/devices/virtual/net/lo/threaded
> /sys/devices/virtual/net/lo/tx_queue_len
> /sys/devices/virtual/net/lo/iflink
> /sys/devices/virtual/net/lo/flags
29901c29935
< /before-install.txt
---
> /after-install.txt
30087a30122
> /capstone/build/install_manifest.txt
32467a32503
> /before-install.txt

[AndrewQuijano]

Looks like it is working, going to push changes shortly

[AndrewQuijano]

Oh and @Rot127, assuming all this works as intended, could you make a capstone 5.0.3 Debian package or a 5.0.4 Debian package? The tool I use PANDA, for now, seems to only support version 5. The goal is to replace this with a one line to download the Debian package and install it.

Essentially, I do need a libcapstone.so.5 for the current capstone version https://github.com/panda-re/panda/blob/dev/panda/scripts/install_ubuntu.sh#L109-L118

https://github.com/panda-re/panda/blob/dev/Dockerfile#L37-L41

[AndrewQuijano]

Actually, I released I can test this on my own fork. Just be careful, you can only create tags say 5.0.4. If you have letters, think the debian part will break, just as a heads up. Works on 1.5.8 but not say v1.5.5

https://github.com/AndrewQuijano/capstone/releases/tag/1.5.8 https://github.com/AndrewQuijano/capstone/releases/tag/v1.5.5

[Rot127]

Thanks for all the work!

Just be careful, you can only create tags say 5.0.4. If you have letters, think the debian part will break, just as a heads up. Works on 1.5.8 but not say v1.5.5

This is very good to know. Can you may lint the version string given to the script? So for example:

• Remove letters from debian
• Or don't build for debian if other letters than a v prefix is given (e.g. in the case of 6.0.0a for Alpha v6)?

We just need a way to have this checked, handled and documented in code. Because otherwise it will absolutely be forgotten at some point.

[Rot127]

could you make a capstone 5.0.3 Debian package or a 5.0.4 Debian

Yes, we can release a new v5 version. Was on the list anyways. Just waiting for the TriCore bug to be resolved.

[AndrewQuijano]

Hello, I just got your changes in, seems like libcapstone.a is no longer appearing, not sure how to fix this

> /usr/local/include/capstone/loongarch.h
> /usr/local/include/capstone/riscv.h
> /usr/local/include/capstone/x86.h
> /usr/local/include/capstone/xcore.h
> /usr/local/include/capstone/tricore.h
> /usr/local/include/capstone/platform.h
> /usr/local/include/capstone/mips.h
> /usr/local/include/capstone/hppa.h
> /usr/local/include/capstone/bpf.h
> /usr/local/include/capstone/sh.h
> /usr/local/include/capstone/wasm.h
> /usr/local/include/capstone/systemz.h
> /usr/local/include/capstone/ppc.h
> /usr/local/include/capstone/aarch64.h
> /usr/local/include/capstone/tms320c64x.h
> /usr/local/include/capstone/arm.h
> /usr/local/include/capstone/mos65xx.h
> /usr/local/include/capstone/cs_operand.h
> /usr/local/include/capstone/m68k.h
> /usr/local/include/capstone/systemz_compatibility.h
> /usr/local/include/capstone/alpha.h
> /usr/local/include/capstone/evm.h
> /usr/local/include/capstone/m680x.h
> /usr/local/include/capstone/arm64.h
> /usr/local/include/capstone/xtensa.h
> /usr/local/include/capstone/capstone.h
> /usr/local/include/capstone/sparc.h
7549a7577,7581
> /usr/local/lib/pkgconfig/capstone.pc
> /usr/local/lib/cmake/capstone/capstone-targets.cmake
> /usr/local/lib/cmake/capstone/capstone-targets-release.cmake
> /usr/local/lib/cmake/capstone/capstone-config-version.cmake
> /usr/local/lib/cmake/capstone/capstone-config.cmake
10138a10171
> /usr/local/bin/cstool

[Rot127]

Oups, forgot to update the install targets:

diff --git a/CMakeLists.txt b/CMakeLists.txt
index 345740c9..c8c10880 100644
--- a/CMakeLists.txt
+++ b/CMakeLists.txt
@@ -878,7 +885,13 @@ if(CAPSTONE_INSTALL)
         DESTINATION ${CAPSTONE_CMAKE_CONFIG_INSTALL_DIR}
     )

-    install(TARGETS capstone
+    if(BUILD_SHARED_LIBS)
+        set(LIB_INSTALL_TARGETS capstone_static capstone_shared)
+    else()
+        set(LIB_INSTALL_TARGETS capstone_static)
+    endif()
+
+    install(TARGETS ${LIB_INSTALL_TARGETS}
             EXPORT capstone-targets
             RUNTIME DESTINATION ${CMAKE_INSTALL_BINDIR}
             LIBRARY DESTINATION ${CMAKE_INSTALL_LIBDIR}

[AndrewQuijano]

OK, I just need to fix capstone.pc version and removing arch and I am done, but in the meantime please test what I got so far

[AndrewQuijano]

Hm, this is tricky

[Rot127]

The workflow fails in my test release: https://github.com/Rot127/capstone/actions/runs/11461606430/job/31890993552

[Rot127]

Numbers seem fine. I think it is because it passes ./debian/capstone.deb to the check script. But the working dir is already ./debian/

[AndrewQuijano]

yeah just patched it

[AndrewQuijano]

Oh I forgot to pull out debian package because I was debugging the library issue. This should work end to end now

[Rot127]

I need to go. Will take a look at it tomorrow again. Thanks for fixing it so quickly.

[AndrewQuijano]

No worries! Thanks for your fast responses! Really look forward to updating PANDA with the package. Also QQ where did you find buster-slim is EOL? Figure I might as well update PANDA's Debian package thing too, but figure I want to to document it in the PR

[AndrewQuijano]

Looks like this works!

https://github.com/AndrewQuijano/capstone/actions/runs/11462174350/job/31892948396

https://github.com/AndrewQuijano/capstone/releases/tag/v1.5.9

[Rot127]

where did you find buster-slim is EOL?

Debian Buster is EOL since two years. Hence we don't have to build on it. This is also likely the reason you could not install cmake via apt.

[Rot127]

@AndrewQuijano Check https://github.com/AndrewQuijano/capstone/pull/1 please

[AndrewQuijano]

Clear up usage of static libraries.

• Python bindings only use the dynamic lib. But built and copied the static ones sometimes nonetheless.
• Add toggles to build only static, static/dyn or only dynamic.

Install cmake via apt. cmake >3.12 was not present in Debian Buster.

[AndrewQuijano]

@Rot127 just accepted your PR

[AndrewQuijano]

Thanks for your help with this! Just curious, when would you expect v 5.0.4 of capstone Debian Package be ready by? Figure I can start working on the PR for PANDA to use the debian package. I'm wondering if there might come a day you might have an apt repository?

https://earthly.dev/blog/creating-and-hosting-your-own-deb-packages-and-apt-repo/

[Rot127]

Nice! Lgtm. I fix the failing test tomorrow. @kabeor Please review this one.

[Rot127]

I'm wondering if there might come a day you might have an apt repository?

It is not planned. We need maintainers for this. And we are already too busy with all the other stuff to do it on our own.

when would you expect v 5.0.4 of capstone Debian Package be ready by?

https://github.com/capstone-engine/capstone/pull/2523 is the PR which has to be done. I review it tomorrow as well. Then we need to merge it into v5. Depending how much we have to change I guess 1-6 weeks. billow and me are pretty busy these days. So hard to say.

[AndrewQuijano]

btw I noticed it still says request changes blocking the merge: https://github.com/capstone-engine/capstone/pull/2521#pullrequestreview-2384823865

1. Do you still want me to remove the 'archs' from the pkconfig?
2. Safe to say the capstone debian package worked without needing ldconfig?

[AndrewQuijano]

@Rot127 Just made your suggested changes, let me know about if the ldconfig works/the package installation works as intended. Look forward to merging the PR!

[AndrewQuijano]

@Rot127, let me know when this PR is to be merged.

I made one last change, and provided more context in https://github.com/capstone-engine/capstone/issues/2398. I changed the package name from 'capstone.deb' to 'libcapstone-dev.deb' so it matches the name of the apt-package. Just to confirm, this debian package essentially would be both libcapstone5.deb and libcapstone-dev.deb package from apt? I suspect the answer is yes given that both library file is added, and the /usr/local/include is populated with headers.

And please confirm the package works as expected. Essentially, PANDA would have required both the capstone library, and does rely on capstone for development. https://www.capstone-engine.org/documentation.html