build: add ability to make test deb packages

[garlick]

Problem: flux-core and flux-sched can make test deb packages for quick test deployments but flux-pmix cannot.

Borrow the build infrastructure for from flux-core and adapt for this project.

Tested with the debian 11 default libpmix-dev package (4.0.0-4.1). The debian/rules file includes a workaround for the broken pmix.pc in that package.

$ flux run -o verbose=2 -q batch --label-io -o pmi=pmix -N2 flux pmi -v --method=pmix barrier 
flux-job: ƒTXjUTSsWoZ started                                          00:00:02
2.760s: flux-shell[1]: DEBUG: Loading /etc/flux/shell/initrc.lua
2.761s: flux-shell[1]: TRACE: Successfully loaded flux.shell module
2.761s: flux-shell[1]: TRACE: trying to load /etc/flux/shell/initrc.lua
2.766s: flux-shell[1]: DEBUG: pmix: server is enabled
2.766s: flux-shell[1]: TRACE: trying to load /etc/flux/shell/lua.d/intel_mpi.lua
2.766s: flux-shell[1]: TRACE: trying to load /etc/flux/shell/lua.d/openmpi.lua
2.776s: flux-shell[1]: DEBUG: pmix: jobid = 197054800230088704
2.776s: flux-shell[1]: DEBUG: pmix: shell_rank = 1
2.776s: flux-shell[1]: DEBUG: pmix: local_nprocs = 1
2.776s: flux-shell[1]: DEBUG: pmix: total_nprocs = 2
2.826s: flux-shell[1]: DEBUG: pmix: local_peers = 1
2.826s: flux-shell[1]: DEBUG: pmix: node_map = picl3,picl4
2.827s: flux-shell[1]: DEBUG: pmix: proc_map = 0;1
2.828s: flux-shell[1]: DEBUG: 1: task 1 on cores 0-3
2.763s: flux-shell[0]: DEBUG: Loading /etc/flux/shell/initrc.lua
2.764s: flux-shell[0]: TRACE: Successfully loaded flux.shell module
2.764s: flux-shell[0]: TRACE: trying to load /etc/flux/shell/initrc.lua
2.769s: flux-shell[0]: DEBUG: pmix: server is enabled
2.769s: flux-shell[0]: TRACE: trying to load /etc/flux/shell/lua.d/intel_mpi.lua
2.770s: flux-shell[0]: TRACE: trying to load /etc/flux/shell/lua.d/openmpi.lua
2.838s: flux-shell[0]: DEBUG: output: batch timeout = 0.500s
2.851s: flux-shell[0]: DEBUG: pmix: jobid = 197054800230088704
2.851s: flux-shell[0]: DEBUG: pmix: shell_rank = 0
2.852s: flux-shell[0]: DEBUG: pmix: local_nprocs = 1
2.852s: flux-shell[0]: DEBUG: pmix: total_nprocs = 2
2.852s: flux-shell[0]: DEBUG: pmix: server outsourced to 4.0.0rc3
2.913s: flux-shell[0]: DEBUG: pmix: local_peers = 0
2.914s: flux-shell[0]: DEBUG: pmix: node_map = picl3,picl4
2.915s: flux-shell[0]: DEBUG: pmix: proc_map = 0;1
2.917s: flux-shell[0]: DEBUG: 0: task_count=2 slot_count=2 cores_per_slot=1 slots_per_node=1
2.917s: flux-shell[0]: DEBUG: 0: task 0 on cores 0-3
2.986s: flux-shell[1]: TRACE: pmix: pmix server fence_upcall {"procs":[{"nspace":"ƒTXjUTSsWoZ","rank":-2}],"info":[{"key":"pmix.collect","flags":0,"value":{"type":1,"data":true}}],"data":"AgICAQ==","cbfunc":137579362112,"cbdata":137640361184}
2.987s: flux-shell[1]: TRACE: pmix: starting pmix exchange 0: size 4
3.011s: flux-shell[0]: TRACE: pmix: pmix server fence_upcall {"procs":[{"nspace":"ƒTXjUTSsWoZ","rank":-2}],"info":[{"key":"pmix.collect","flags":0,"value":{"type":1,"data":true}}],"data":"AgICAQ==","cbfunc":138446157632,"cbdata":138512776416}
3.011s: flux-shell[0]: TRACE: pmix: starting pmix exchange 0: size 4
3.019s: flux-shell[0]: TRACE: pmix: completed pmix exchange 0: size 8 SUCCESS
3.020s: flux-shell[0]: TRACE: pmix: pmix server fence_upcall {"procs":[{"nspace":"ƒTXjUTSsWoZ","rank":-2}],"info":[{"key":"pmix.collect","flags":0,"value":{"type":1,"data":true}}],"data":"AgICAQ==","cbfunc":138446157632,"cbdata":138512776416}
3.020s: flux-shell[0]: TRACE: pmix: starting pmix exchange 1: size 4
3.018s: flux-shell[1]: TRACE: pmix: completed pmix exchange 0: size 8 SUCCESS
3.019s: flux-shell[1]: TRACE: pmix: pmix server fence_upcall {"procs":[{"nspace":"ƒTXjUTSsWoZ","rank":-2}],"info":[{"key":"pmix.collect","flags":0,"value":{"type":1,"data":true}}],"data":"AgICAQ==","cbfunc":137579362112,"cbdata":137640361184}
3.019s: flux-shell[1]: TRACE: pmix: starting pmix exchange 1: size 4
3.022s: flux-shell[1]: TRACE: pmix: completed pmix exchange 1: size 8 SUCCESS
3.023s: flux-shell[0]: TRACE: pmix: completed pmix exchange 1: size 8 SUCCESS
3.030s: flux-shell[0]: DEBUG: task 0 complete status=0
3.029s: flux-shell[1]: DEBUG: task 1 complete status=0
3.042s: flux-shell[1]: DEBUG: exit 0
0: pmix: initialize: rank=0 size=2 name=ƒTXjUTSsWoZ: success
1: pmix: initialize: rank=1 size=2 name=ƒTXjUTSsWoZ: success
0: pmix: barrier: success
1: pmix: barrier: success
0: pmix: barrier: success
0: ƒTXjUTSsWoZ: completed pmi barrier on 2 tasks in 0.004s.
1: pmix: barrier: success
0: pmix: finalize: success
1: pmix: finalize: success
3.055s: flux-shell[0]: DEBUG: exit 0

[grondo]

I gave this a spin and I think debian/compat is missing:

 grondo@pi3:~/git/flux-pmix$ make deb
debbuild: Running make dist
debbuild: Building package from latest dist tarball
debbuild: Unpacking flux-pmix-0.3.0-7-ga521f7a.tar.gz
debbuild: Creating debian directory and files
debbuild: Creating debian/changelog
debbuild: Running debian-buildpackage -b
dpkg-buildpackage: info: source package flux-pmix
dpkg-buildpackage: info: source version 0.3.0-7-ga521f7a
dpkg-buildpackage: info: source distribution UNRELEASED
dpkg-buildpackage: info: source changed by Mark A. Grondona <mark.grondona@gmail.com>
dpkg-buildpackage: info: host architecture arm64
 dpkg-source --before-build .
 fakeroot debian/rules clean
make[1]: Entering directory '/home/grondo/git/flux-pmix/debbuild/flux-pmix-0.3.0-7-ga521f7a'
dh clean
dh: error: Please specify the compatibility level in debian/compat
make[1]: *** [debian/rules:16: clean] Error 255

[grondo]

Once I did

echo 10 >debian/compat

Then it works for me. :+1:

[garlick]

Oops! Thanks, good catch! Pushed a fix.

[codecov[bot]]

Codecov Report

Merging #87 (14bf9e1) into main (28e9b88) will not change coverage. The diff coverage is n/a.

:exclamation: Current head 14bf9e1 differs from pull request most recent head bea20e2. Consider uploading reports for the commit bea20e2 to get more accurate results

@@           Coverage Diff           @@
##             main      #87   +/-   ##
=======================================
  Coverage   79.01%   79.01%           
=======================================
  Files          12       12           
  Lines        1382     1382           
=======================================
  Hits         1092     1092           
  Misses        290      290           

:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more

[garlick]

Thanks! This was largely your work, so thank you.