Open jdblischak opened 11 months ago
Per discussion, let's check CMAKE_INSTALL_LIBDIR
, it is possible that the DLLs are being installed in lib
where they can't be found (no rpath on windows, they need to go in bin/
.
Quick update. I confirmed that the conda env installs both tiledbvcf.exe
and tiledbvcf.dll
into Library\bin
, which explains why it works. However, it's unclear to me how this is configured. The conda recipe build script doesn't set any extra flags to control this. I also checked the compiler feedstock, which defines various CMake env vars. It doesn't define CMAKE_INSTALL_LIBDIR
. The most promising one I found was CMAKE_PROGRAM_PATH
.
Regardless, I pushed a commit to try CMAKE_INSTALL_LIBDIR
rem confirming conda installation setup
mamba create -n win-vcf -c tiledb tiledbvcf-py
mamba activate win-vcf
dir %CONDA_PREFIX%\Library\bin\tile*
rem Volume in drive C has no label.
rem Volume Serial Number is 8038-C77D
rem
rem Directory of C:\Users\john\mambaforge\envs\win-vcf\Library\bin
rem
rem 11/03/2023 09:51 AM 15,554,048 tiledb.dll
rem 11/10/2023 08:21 PM 1,328,640 tiledbvcf.dll
rem 11/10/2023 08:21 PM 1,752,576 tiledbvcf.exe
rem 11/10/2023 08:21 PM 2,159,104 tiledbvcf4test.dll
rem 4 File(s) 20,794,368 bytes
rem 0 Dir(s) 27,726,221,312 bytes free
tiledbvcf version
rem TileDB-VCF version
rem TileDB version 2.17.4
rem htslib version 1.15.1
Setting CMAKE_INSTALL_LIBDIR
had no effect. Furthermore, I realized that the .dll
was already being installed in bin
alongside the .exe
. From the build log:
-- Install configuration: "Release"
-- Installing: D:/a/TileDB-VCF/TileDB-VCF/install/lib/hts-3.lib
-- Installing: D:/a/TileDB-VCF/TileDB-VCF/install/lib/spdlog.lib
-- Installing: D:/a/TileDB-VCF/TileDB-VCF/install/bin/tiledbvcf.exe
-- Installing: D:/a/TileDB-VCF/TileDB-VCF/install/lib/tiledbvcf.lib
-- Installing: D:/a/TileDB-VCF/TileDB-VCF/install/bin/tiledbvcf.dll
-- Installing: D:/a/TileDB-VCF/TileDB-VCF/install/include/tiledbvcf/tiledbvcf.h
-- Installing: D:/a/TileDB-VCF/TileDB-VCF/install/include/tiledbvcf/tiledbvcf_enum.h
-- Installing: D:/a/TileDB-VCF/TileDB-VCF/install/include/tiledbvcf/arrow.h
-- Installing: D:/a/TileDB-VCF/TileDB-VCF/install/include/tiledbvcf/tiledbvcf_export.h
-- Installing: D:/a/TileDB-VCF/TileDB-VCF/install/lib/tiledbvcf4test.lib
-- Installing: D:/a/TileDB-VCF/TileDB-VCF/install/bin/tiledbvcf4test.dll
-- Up-to-date: D:/a/TileDB-VCF/TileDB-VCF/install/lib/hts-3.lib
Thus I thought maybe I need to set some env var, but confusingly, locally I can run tiledbvcf
by providing the path even when the conda env isn't activated. In other words, I don't see the difference between the example below and what I am trying to accomplish in the CI job
(base) C:\Users\john>mambaforge\envs\win-vcf\Library\bin\tiledbvcf version
rem TileDB-VCF version
rem TileDB version 2.17.4
rem htslib version 1.15.1
I uploaded the installation directory (contains both libtiledb and libtiledbvcf) as an artifact. It can be downloaded by scrolling to the bottom of the latest CI build
Note that I also run dumpbin
in the pipeline, eg here
This nightly build for Windows is a work-in-progress. The existing GitHub Actions pipeline to test TileDB-VCF against release libtiledb on Windows relies exclusively on conda (ie everything is built inside of an activated conda environment). In contrast, for this build against dev libtiledb, I have tried to use conda as little as possible (currently only for installing the Python dependencies, but I'm open to trying to replace these with PyPI packages). Unfortunately this is causing problems. Code that runs fine inside a conda env doesn't work in a more minimal setup. I'd appreciate any advice.
Problem 1: Unable to run
tiledbvcf.exe version
I install
tiledbvcf.exe
into a local directory that is not on thePATH
. Thus I provide the full path when trying to execute it. This does not work, and I've tried various things (with and without.exe
, absolute versus relative path). This approach of installing into a local directory works fine on Linux/macOS. Is this even possible on Windows?Problem 2: Python client unable to find
tiledbvcf.dll
at runtimeI can build the Python client against the locally built libtiledbvcf using
--libtiledbvcf
. However, it immediately fails at runtime because it can't findtiledbvcf.dll
(or one of its dependencies)I've already added the installation location to
PATH
. Maybe the issue is thattiledbvcf.dll
can't findtiledb.dll
?Problem 3: What tests to run for
libtiledbvf
?The existing Windows build only runs the tests for the Python client. Which
libtiledbvcf
tests are feasible to run on Windows?The unit tests,
tiledb_vcf_unit
, require building an executable and then running it. But from the example oftiledbvcf.exe version
above, I don't think I can simply executetiledb_vcf_unit
like I do for Linux/macOS.The script
run-cli-tests.sh
requires bcftools to be installed, and I don't think it's worth the effort to build bcftools on Windows.