search

samtools / htslib

C library for high-throughput sequencing data formats
Other
806 stars 446 forks source link

The htscodecs `version.h` is not built unless its .git is present; no instructions to prepare a working tarball #1846

Open adamnovak opened 5 days ago

adamnovak commented 5 days ago

The htslib build process needs to build htscodecs/htscodecs/version.h, since it isn't shipped in the htscodecs repository. This is accomplished here:

https://github.com/samtools/htslib/blob/2ff207bd16bb3094843395450aa48936dd774ecb/Makefile#L578-L591

But if htscodecs doesn't have a .git folder (for example, because we're building from a tarball and not a source checkout), this doesn't do anything and the file is not created.

So if you want to prepare a buildable tarball of htslib, you need to make sure that make htscodecs/htscodecs/version.h has been run before any .git folders are removed.

For vg, we ship tarballs that include a vendored-in htslib. But it turns out our tarballs don't actually build, since we don't perform that step as part of our tarball packaging process.

So I went looking for the official instructions for preparing htslib to be tarballed, and I couldn't find them. The Makefile provides a make distdir, but that doesn't seem to include htscodecs at all, nor does it include os/lzma_stub.h which the build seems to require, so I'm not sure that it is actually in use. The actual htslib release tarballs include these, so they must not use (just) make distdir.

If tarballing up everything in the Git tree except the .git folders doesn't produce a directory tree that can build, there should be instructions in the project documentation on how to prepare the tree to be tarballed, or how to export a working tarball through the build system somehow.

I can by trial and error discover that I need a make htscodecs/htscodecs/version.h, but how will I know when new required generated files are added to the project?

daviesrob commented 5 days ago

Our release tarballs are built using infrastructure in the https://github.com/samtools/c-maint repository. The htscodecs/htscodecs/version.h file gets made here, although the tarball building process does more than this as it also strips out bits of htscodecs that are not needed, builds the htslib configure script, and adds other files used by configure.

It looks like the make distdir target was supposed to do what you want, but may have stopped working after htscodecs was added. We'll take a look.