search

dotnet / runtime

.NET is a cross-platform runtime for cloud, mobile, desktop, and IoT apps.
https://docs.microsoft.com/dotnet/core/
MIT License
15.26k stars 4.73k forks source link

The new nethost static library is not published in the daily builds #1154

Open Brar opened 4 years ago

Brar commented 4 years ago

I'm trying to use the new nethost static library that was introduced via #296. Since I have yet to figure out how to use NuGet packages like Microsoft.NETCore.DotNetAppHost (or Microsoft.NETCore.App.Host.[win-x64|linux-x64] ) from native C builds I tried to download the daily releases (https://dotnetcli.blob.core.windows.net/dotnet/Runtime/master/dotnet-nethost-latest-win-x64.zip or https://dotnetcli.blob.core.windows.net/dotnet/Runtime/master/dotnet-nethost-latest-linux-x64.tar.gz) where I found out that the static libraries don't get published.

Is this an intentional decision or an oversight?

If it is an intentional decision, this might be a documentation issue. How am I supposed to consume the NuGet packages and are there daily builds of them? I've already read https://docs.microsoft.com/en-us/dotnet/core/tutorials/netcore-hosting and https://github.com/dotnet/runtime/blob/master/docs/design/features/native-hosting.md and also tried https://github.com/dotnet/samples/tree/master/core/hosting/HostWithHostFxr but all of them don't show an elegant or recommended way of using them. The samples actually show a way which is described as "relatively complicated" and recommend CMake for native builds which, to my knowledge, can't consume NuGet packages.

am11 commented 4 years ago

The switch-to-dotnet/runtime is still in progress for daily SDK builds: https://github.com/dotnet/core-sdk/pull/5924, which include runtime bits. Currently it is pointing to Nov 14 revision: https://github.com/dotnet/core-setup/commit/c77948d92a2f950140f09384f057cb893ec3955a.

dotnet-nethost-latest-linux-x64.tar.gz is also pointing to the same old commit:

CDN_URL="https://dotnetcli.azureedge.net/dotnet/Runtime/master/dotnet-nethost-latest-linux-x64.tar.gz"
curl -sSL $CDN_URL | tar xvz | strings libnethost.so | grep "@(#)"

@(#)Version 5.0.19.56401 @Commit: c77948d92a2f950140f09384f057cb893ec3955a
Brar commented 4 years ago

Thank you @am11. So if I understand you correctly the library will appear at the location linked from README.md as the switch-to-dotnet/runtime progresses. That solves my primary question, so I'm closing this.

Brar commented 4 years ago

I looks like I've prematurely closed this.

CDN_URL="https://dotnetcli.azureedge.net/dotnet/Runtime/master/dotnet-nethost-latest-linux-x64.tar.gz"
curl -sSL $CDN_URL | tar xvz | strings libnethost.so | grep "@(#)"
strings: 'libnethost.so': No such file

Is this a CDN issue?

am11 commented 4 years ago

I think just a broken pipe, this should work:

curl -sSL $CDN_URL | tar xvz && strings libnethost.so | grep "@(#)"
Brar commented 4 years ago

I think just a broken pipe, this should work:

You're right but where is libnethost.a?

curl -sSL $CDN_URL | tar xvz && strings libnethost.a | grep "@(#)"

am11 commented 4 years ago

where is libnethost.a?

Daily builds hasn't been published to blob storage in a while (latest build is from Nov 14 commit), and make target for libnethost.a was added 18 days ago.

library will appear at the location linked from README.md as the switch-to-dotnet/runtime progresses.

Yes, if we build the product locally, we can find libnethost.a in tar.gz.

git clone https://github.com/dotnet/runtime dotnet-runtime
cd dotnet-runtime

./build.sh -c release
./build.sh --subset PkgProj /p:Configuration=Release

tar xvzf ./artifacts/packages/Release/Shipping/dotnet-nethost-5.0.0-dev.19627.1-linux-x64.tar.gz
strings libnethost.a | grep "@(#)"

@dagood, could you share some insights when can we expect daily runtime builds from new repo?

dagood commented 4 years ago

We have official builds since Dec 21, check out https://github.com/dotnet/runtime/issues/98 for the latest status.

jeffschwMSFT commented 4 years ago

cc @elinor-fung

am11 commented 4 years ago

@dagood, ah thanks! So currently it is under a different subdomain:

https://dotnetfeed.blob.core.windows.net/dotnet-core/assets/core-setup/Runtime/5.0.0-alpha.1.19618.2/dotnet-nethost-5.0.0-alpha.1.19618.2-linux-x64.tar.gz

s/dotnetcli/dotnetfeed

@Brar, this one contains static library: .a and its -win-x64.zip counterpart has .lib.

Is it because we are waiting for dotnet/toolset and dotnet/core-sdk update to get unblocked https://github.com/dotnet/runtime/issues/98#issuecomment-571361769, before it will appear under dotnetcli subdomain (which is fronted by the CDN)? otherwise we can update the README.md table by replacing https://dotnetcli with https://dotnetfeed.

dagood commented 4 years ago

It looks like the location was broken with that build number, the file is intended to be available on dotnetcli same as always--here's a link from a newer build:

am11 commented 4 years ago

Could the version-less master tarbar with latest-linux-x64 be also updated? problem is that this one has old binaries: https://dotnetcli.azureedge.net/dotnet/Runtime/master/dotnet-nethost-latest-linux-x64.tar.gz

dagood commented 4 years ago

@am11 unfortunately no, the infra that was doing that is very brittle vs. Arcade breaking changes, so we decided to wait for https://github.com/dotnet/arcade/issues/3963 instead of bringing the old infra online in the new dotnet/runtime repo.

AaronRobinsonMSFT commented 4 years ago

This is now being published, but the libnethost.lib is incomplete due to source refactoring in https://github.com/dotnet/runtime/pull/2123. In order to link against libnethost.lib, nethost.h needs to be updated to remove __declspec(dllimport) on Windows, and the libhostmisc.lb needs to be consumed since that has a bunch of the implementation. I can submit a PR for the header file issue, but I am struggling to figure out how to make this work and get the benefit of https://github.com/dotnet/runtime/pull/2123 without doing some postprocessing of the libs.

Suggestions are welcome - sorry for hijacking this issue, but it is technically no longer fully published in a consumable way.

/cc @vitek-karas @jeffschwMSFT

vitek-karas commented 4 years ago

Thanks @AaronRobinsonMSFT for bringing this up. @swaroop-sridhar @VSadov can you please look into that - it's kind of related to the host build system (single-file and all)? We should obviously add a test...

AaronRobinsonMSFT commented 4 years ago

@vitek-karas @swaroop-sridhar @VSadov It might be easiest to revert #2123. I can do that if it would help.

jkotas commented 4 years ago

the libnethost.lib is incomplete due to source refactoring in #2123

My vote would be to revert #2123 . Building a couple of .cpp files twice is just fine.

AaronRobinsonMSFT commented 4 years ago

@vitek-karas @swaroop-sridhar @VSadov If you all concur with @jkotas and it doesn't impact your other single file work, I will do that revert.

swaroop-sridhar commented 4 years ago

I think it is fine to revert https://github.com/dotnet/runtime/pull/2123. The static host PR will need to merge appropriately, but @VSadov is looking at making changes to the file-moves in the PR anyways. So I think it'll be OK to adapt to the change. CC:@VSadov

AaronRobinsonMSFT commented 4 years ago

le sigh... Reverting this change is going to be basically impossible. Too many things have built on top of it. I will revert the conceptual change rather the actual commit.

swaroop-sridhar commented 4 years ago

Thanks @AaronRobinsonMSFT This may make some components smaller. For example, libhostcommon has things like info.cpp and header.cpp, which hostfxr and hostpolicy need, but nethost, comhost etc don't need.

VSadov commented 4 years ago

My current iteration of single file work no longer moves files under corehost. Projects will need to be adjusted though if this fix changes where files are.

AaronRobinsonMSFT commented 4 years ago

@VSadov I am trying to leave files where they are. However, the CMake files for corehost need a lot of attention. They are in an unacceptable state and make reasoning about this so much harder than they should be for such a simple set of build requirements like corehost.

VSadov commented 4 years ago

Most of the single-file changes are in CMake projects. The change is mostly about hostfxr and policy though, so depending what changes it might not be that hard to merge on top.

agocke commented 4 years ago

@AaronRobinsonMSFT As far as I can tell, this issue is currently adding tracking a test to make sure that the nethost is buildable as a static library?

AaronRobinsonMSFT commented 4 years ago

@agocke Yes that is true. It is now published and used, but a test would be prudent.

Brar commented 4 years ago

@agocke Yes that is true. It is not published and used, but a test would be prudent.

I'd say that a static nethost would be the most convenient way to actually use it (for outside projects) if it eventually gets published in a reliable way.

Just my 2ct.

AaronRobinsonMSFT commented 4 years ago

@Brar My mistake. I meant to say "it is now". I fixed this issue in #35431. Adding testing for the scenario is all that remains.