Move inclusion of -lnvidia-ml to base library

[janciesko]

Allows to inherit this dependency in downstream projects. Might help to avoid having users run into undefined references when linking.