varunsh-xilinx commented 1 year ago

Summary of Changes

Use vcpkg to install dependencies

Motivation

Using a package manager allows for building the server beyond just in a container. It provides a path to easily build on different operating systems as well as switching between dynamic and static libraries.

Implementation

vcpkg is integrated in manifest mode. The Docker build process now installs the dependencies in the development container with vcpkg in /opt/vcpkg. This makes the dev container self-sufficient (at least until you change the features or triplet).

vcpkg is also added as a submodule so you can build it outside docker using that as well. I tested it on Windows and most of the dependencies were successfully installed (ffmpeg hangs though there are some issues about this on vcpkg (29390 and others).

While vcpkg offers a path to building on Windows and statically, the inference server is not currently designed to work that way. It will only build for shared libraries on Linux until the build limitations are removed.

Notes

I initially tried vcpkg and then switch to trying conan. The recent conan 2.0 release has broken many of the packages and I was unable to resolve some of the issues with using specific versions of libraries that conflicted with the versions specified in the dependencies. It led to version conflicts and using override or force didn't work for my case.

I came back to vcpkg after the failed conan experiment and implemented the VAI libraries in vcpkg overlays (along with patches to fix them for building). One problem that came about near the end was that the CI was using the different containers (with different things installed) to automatically configure the CMake build with different things to enable. With vcpkg, the container doesn't have these components. In addition, you need to specify the manifest features before invoking project() and using find_package() to find out what exists. I explored using CMake presets to enable this functionality but this fell apart because every build command would then need to specify a preset and using user-specific presets and default presets becomes confusing and difficult to maintain.

Resolving AKS was another whole mess. AKS, by default, was invoking CMake independently for each kernel. When the VAI dependencies were moved to vcpkg, these individual CMake builds don't specify the toolchain so they don't find these dependencies. Instead, I redid the whole build for AKS kernels to be natively included as a subdirectory rather than being built separately. This way, the toolchain correctly gets propagated to the kernels.