Enable referencing proto files in NuGet packages

alexvanboxel commented 5 years ago

A common pattern in other languages is that your proto files are not part of your project, especially if you are a client but ofter also when you are a server.

Certainly in enterprises that have their stack build on gRPC. What you see is that the proto's are managed in a common mono repo for all contracts (were talking mono-repo for the proto files only here). What you then often see is that contract repo has a language specific distribution step. This do not create client libraries, they just publish the proto files in a package format idiomatic to the language (an example is Java: they package the proto files in a jar package and publish them to a maven repo, in general a private one of the company).

What I like to see is that the .NET implementation would be able to reference proto in NuGet that it can fetch from a NuGet repository. That will make it easier for enterprises to adapt gRPC in .NET. They only need to add the NuGet distribution step in one place and a project just need to references NuGet as an external dependency.

Let me also stress that this is distributing the proto files and not client libraries. You want to avoid distribution of clients that are build with an older version of proto as the one you are using in your project.

And example how this works in Java (using gradle):

dependencies {
    compile("com.google.protobuf:protobuf-java:3.6.1")
    compile("io.grpc:grpc-protobuf:1.18.0")
    protobuf('io.anemos:protobeam-options-proto:0.0.3-SNAPSHOT')
}

The 3th dependency points to an artifact just containing proto, gradle will make sure that client/server stubs are generate with the same version of gRPC and protobuf as the project is.

JamesNK commented 5 years ago

Your Java example, is it like a csproj PackageReference + Protobuf reference of the contents rolled into one?

JamesNK commented 5 years ago

I think you could do this today:

Package a nupkg with the proto files as content
Reference the NuGet package in your client/server, which would then include the content files in your project
Add a <Protobuf> line to your csproj that references the proto files

JunTaoLuo commented 5 years ago

Considerations:

How to pass the options to the compiler, for example the option to use more efficient APIs for serialization.

antonioortizpola commented 5 years ago

@JamesNK Could you provide an example? I am trying using my protos from other project in my solution (the idea is latter use nuget), but I can not get it to work.

The idea I think is simple, I have a protos project with the proto files in a folder called "Protos". The files are marked as Content.

Then I add that project as reference to mi server project, i can see the references of the files

And I added the Protobuf line

    <ItemGroup>
        <Protobuf Include="Protos\**\*.proto" GrpcServices="Server" AdditionalImportDirs="Protos\" />
        <Content Include="@(Protobuf)" LinkBase="Protos" />
    </ItemGroup>

But the greet.proto is not detected, what am I missing?

stueeey commented 5 years ago

Technically you could make a targets file which enumerated the nuget references and included any .proto files in a well known folder within the packages i.e. 'proto'. might actually be pretty useful if the grpc package did this

JamesYangLim commented 5 years ago

Import proto files from nuget packages work under this csproj settings with grpc.

csproj

<Project Sdk="Microsoft.NET.Sdk">

  <PropertyGroup>
    <TargetFramework>netstandard2.0</TargetFramework>
  </PropertyGroup>

  <ItemGroup>
    <PackageReference Include="Google.Protobuf" Version="3.6.1" />
    <PackageReference Include="Google.Protobuf.Tools" Version="3.6.1" />
    <PackageReference Include="Grpc" Version="1.22.0" />
    <PackageReference Include="Grpc.Core" Version="1.22.0" />
    <PackageReference Include="Grpc.Tools" Version="1.22.0">
      <PrivateAssets>all</PrivateAssets>
      <IncludeAssets>runtime; build; native; contentfiles; analyzers; buildtransitive</IncludeAssets>
    </PackageReference>
    <PackageReference Include="NugetPackage1" Version="1.0.0" />
    <PackageReference Include="NugetPackage2" Version="1.0.1" />
  </ItemGroup>

  <ItemGroup>
    <NugetPkg1Path Include="$(NuGetPackageRoot)nugetpackage1\1.0.0" />
    <NugetPkg2Path Include="$(NuGetPackageRoot)nugetpackage1\1.0.1" />
    <Protobuf Include="*.proto" AdditionalImportDirs="@(NugetPkg1Path );@(NugetPkg2Path )" />
  </ItemGroup>

</Project>

proto

syntax = "proto3";
import "Pkg1.proto";
import "Pkg2.proto";
package test;
option csharp_namespace = "ProtoTest";

message M_Test
{
    pkg1.M_Pkg1 p1 = 1;
    pkg2.M_Pkg2 p2 = 2;
}

However, I could not workout how to get the version of the nuget package that I want. Instead, I have to hard code in it.

Any thoughts?

stueeey commented 5 years ago

This should do everything you want:

<Target Name="IncludeNugetProtoFiles" BeforeTargets="PreBuildEvent">
    <ItemGroup>
        <_ProtoNugetSearchPaths Include="$(NuGetPackageRoot)%(PackageReference.Identity)\%(PackageReference.Version)\Proto"/>
        <_ProtoNugetFilesFound Include="%(_ProtoNugetSearchPaths.FullPath)\*.Proto"/>
        <Protobuf Remove="@(_ProtoNugetFilesFound)"/>
        <Protobuf Include="@(_ProtoNugetFilesFound)"/>
    </ItemGroup>
    <PropertyGroup>
        <_FoundProtoFiles Condition="'@(_ProtoNugetFilesFound->Count())' &gt; 1">True</_FoundProtoFiles>
    </PropertyGroup>
    <Message Text="Searched for proto files in:" Importance="Normal" />
    <Message Text="&#009;%(_ProtoNugetSearchPaths.FullPath)" Importance="Normal"/>
    <Message Text="Found @(_ProtoNugetFilesFound->Count()) packaged protos:" Importance="Normal" Condition="'$(_FoundProtoFiles)' == 'True'"/>
    <Message Text="&#009;%(_ProtoNugetFilesFound.Identity)" Importance="Normal" Condition="'$(_FoundProtoFiles)' == 'True'"/>
</Target>

This will only work with PackageReference (the new format) but i think we should just accept this as a limitation as the old packages.config way is deprecated anyway and would be a lot harder to implement.

Output:

I'd suggest changing the importance of the first 2 messages printing the search paths to low once you are done testing but leave the others at normal.

Let me know if you need any more help, I've done a lot of MSBuild magic lately

stueeey commented 5 years ago

Actually it might be worth renaming _ProtoNugetSearchPaths to ProtoSearchPaths because then it can be easily added to like so:

<ItemGroup>
    <ProtoSearchPaths Include="SomePath1;SomePath2"/>
</ItemGroup>

JamesYangLim commented 5 years ago

Thanks @stueeey , that helps a lot. However, I am having another issue which I couldn't reference the protobuf type in C# code, error says the type (proto message type) could not be found.

csproj:

  <Target Name="PreBuild" BeforeTargets="PreBuildEvent">
    <ItemGroup>
      <Protobuf Include="*.proto;" />
    </ItemGroup>
  </Target>

proto:

syntax = "proto3";
package x;
option csharp_namespace = "ProtoTest";

message M_X
{
    double v = 1;
}

Build succeeded, however in C# code:

If I removed the target,

csproj:

  <ItemGroup>
    <Protobuf Include="*.proto;" />
  </ItemGroup>

Build succeeded and in C# code:

Seems like the target element trigger msbuild to do something different, not exactly sure what happen behind the scene of msbuild. But this simple test proof that having under target block does not work for grpc.

ghost commented 5 years ago

@stueeey Did you put that snippet into the csproj of the project that contains the proto file or the one that wants to consume it? I actually need what @JamesYangLim was doing with the AdditionalImportDirs so you can reference the proto file from the nuget package in a proto file in your project. Your snippet does not solve that, does it?

llawall commented 5 years ago

Using GeneratePathProperty on the nuget packge

<PackageReference Include="My.BaseProtos" Version="1.0.0" GeneratePathProperty="true" />

and adding that path as an protobuf include

<Protobuf Include="$(PkgMy_BaseProtos)/*.proto" ... />

seems to solve the problem here?

JamesYangLim commented 5 years ago

Hi @llawall, thank you for helping. Your approach works very well and very simple. I have the following example that works.

  <ItemGroup>
    <PackageReference Include="Google.Protobuf" Version="3.6.1" />
    <PackageReference Include="Google.Protobuf.Tools" Version="3.6.1" />
    <PackageReference Include="Grpc" Version="1.22.0" />
    <PackageReference Include="Grpc.Core" Version="1.22.0" />
    <PackageReference Include="Grpc.Tools" Version="1.22.0">
      <PrivateAssets>all</PrivateAssets>
      <IncludeAssets>runtime; build; native; contentfiles; analyzers; buildtransitive</IncludeAssets>
    </PackageReference>
    <PackageReference Include="my.package.1" Version="2.3.1" GeneratePathProperty="true"/>
    <PackageReference Include="my.package.2" Version="1.3.4" GeneratePathProperty="true"/>
  </ItemGroup>

  <ItemGroup>
    <Protobuf Include="*.proto" AdditionalImportDirs="$(Pkgmy_package_1)\proto;$(Pkgmy_package_2)\proto" />
  </ItemGroup>

stueeey commented 5 years ago

I had some time today and I've managed to get this working with no changes to the nuget references with the following:

<PropertyGroup>
    <Proto_DefaultAccess>Public</Proto_DefaultAccess>
    <Proto_DefaultClientBaseType>ClientBase</Proto_DefaultClientBaseType>
</PropertyGroup>

<Target Name="_includeNugetProtoFiles" BeforeTargets="_Protobuf_SelectFiles">
    <ItemGroup>
        <_ServerProtoNugetSearchPaths Include="$(NuGetPackageRoot)%(PackageReference.Identity)\%(PackageReference.Version)\Protobuf\Grpc\Server" Condition="Exists('$(NuGetPackageRoot)%(PackageReference.Identity)\%(PackageReference.Version)\Protobuf\Grpc\Server')" />
        <_ServerProtoNugetFilesFound Include="%(_ServerProtoNugetSearchPaths.FullPath)\*.proto" />

        <_ClientProtoNugetSearchPaths Include="$(NuGetPackageRoot)%(PackageReference.Identity)\%(PackageReference.Version)\Protobuf\Grpc\Client" Condition="Exists('$(NuGetPackageRoot)%(PackageReference.Identity)\%(PackageReference.Version)\Protobuf\Grpc\Client')" />
        <_ClientProtoNugetFilesFound Include="%(_ClientProtoNugetSearchPaths.FullPath)\*.proto" />

        <_BothProtoNugetSearchPaths Include="$(NuGetPackageRoot)%(PackageReference.Identity)\%(PackageReference.Version)\Protobuf\Grpc" Condition="Exists('$(NuGetPackageRoot)%(PackageReference.Identity)\%(PackageReference.Version)\Protobuf\Grpc')" />
        <_BothProtoNugetFilesFound Include="%(_BothProtoNugetSearchPaths.FullPath)\*.proto" />

        <_ProtoNugetSearchPaths Include="@(_BothProtoNugetSearchPaths)" />
        <_ProtoNugetSearchPaths Include="@(_ServerProtoNugetSearchPaths)" />
        <_ProtoNugetSearchPaths Include="@(_ClientProtoNugetSearchPaths)" />

        <Protobuf Remove="@(_ServerProtoNugetFilesFound)" />
        <Protobuf Include="@(_ServerProtoNugetFilesFound)" GrpcServices="Server" ClientBaseType="$(Proto_DefaultAccess)" Access="$(Proto_DefaultAccess)" OutputDir="$(Protobuf_OutputPath)\PackagedProtos" />

        <Protobuf Remove="@(_ClientProtoNugetFilesFound)" />
        <Protobuf Include="@(_ClientProtoNugetFilesFound)" GrpcServices="Client" ClientBaseType="$(Proto_DefaultAccess)" Access="$(Proto_DefaultAccess)" OutputDir="$(Protobuf_OutputPath)\PackagedProtos"/>

        <Protobuf Remove="@(_BothProtoNugetFilesFound)" />
        <Protobuf Include="@(_BothProtoNugetFilesFound)" GrpcServices="Both" ClientBaseType="$(Proto_DefaultAccess)" Access="$(Proto_DefaultAccess)" OutputDir="$(Protobuf_OutputPath)\PackagedProtos"/>
    </ItemGroup>
    <PropertyGroup>
        <_NugetProtoSearchPathsFound Condition="'@(_BothProtoNugetSearchPaths-&gt;Count())' &gt;= 1">True</_NugetProtoSearchPathsFound>
        <_FoundClientProtoFiles Condition="'@(_ClientProtoNugetFilesFound-&gt;Count())' &gt;= 1">True</_FoundClientProtoFiles>
        <_FoundServerProtoFiles Condition="'@(_ServerProtoNugetFilesFound-&gt;Count())' &gt;= 1">True</_FoundServerProtoFiles>
        <_FoundBothProtoFiles Condition="'@(_BothProtoNugetFilesFound-&gt;Count())' &gt;= 1">True</_FoundBothProtoFiles>
        <_FoundAnyPackagedProtoFiles Condition="'$(_FoundClientProtoFiles)' == 'True' OR '$(_FoundServerProtoFiles)' == 'True' OR '$(_FoundBothProtoFiles)' == 'True'">True</_FoundAnyPackagedProtoFiles>
    </PropertyGroup>
    <Message Text="Including packaged protos with the following settings:" Condition="'$(_FoundAnyPackagedProtoFiles)' == 'True'"/>
    <Message Text=" Proto_DefaultAccess: $(Proto_DefaultAccess)" Condition="'$(_FoundAnyPackagedProtoFiles)' == 'True'"/>
    <Message Text=" Proto_DefaultClientBaseType: $(Proto_DefaultClientBaseType)" Condition="'$(_FoundAnyPackagedProtoFiles)' == 'True'"/>
    <Message Text="Searched for proto files in:" Importance="Normal" Condition="'$(_NugetProtoSearchPathsFound)' == 'True'" />
    <Message Text=" %(_ProtoNugetSearchPaths.FullPath)" Importance="Normal" Condition="'$(_NugetProtoSearchPathsFound)' == 'True'" />
    <Message Text="Found @(_ClientProtoNugetFilesFound-&gt;Count()) packaged client proto(s):" Importance="Normal" Condition="'$(_FoundClientProtoFiles)' == 'True'" />
    <Message Text=" %(_ClientProtoNugetFilesFound.Identity)" Importance="Normal" Condition="'$(_FoundClientProtoFiles)' == 'True'" />
    <Message Text="Found @(_ServerProtoNugetFilesFound-&gt;Count()) packaged server proto(s):" Importance="Normal" Condition="'$(_FoundServerProtoFiles)' == 'True'" />
    <Message Text=" %(_ServerProtoNugetFilesFound.Identity)" Importance="Normal" Condition="'$(_FoundServerProtoFiles)' == 'True'" />
    <Message Text="Found @(_BothProtoNugetFilesFound-&gt;Count()) packaged client &amp; server proto(s):" Importance="Normal" Condition="'$(_FoundBothProtoFiles)' == 'True'" />
    <Message Text=" %(_BothProtoNugetFilesFound.Identity)" Importance="Normal" Condition="'$(_FoundBothProtoFiles)' == 'True'" />
</Target>

This will pick up anything under <NugetPackage>\protobuf\grpc and generate both client and server <NugetPackage>\protobuf\grpc\client and generate a client <NugetPackage>\protobuf\grpc\server and generate a server

Works like a dream:

Planning to add a property to the properties page of proto files to make it automatically include the proto in this folder structure when packing the project

I'll make a PR into the main GRPC repo - this probably belongs in the GRPC.Tools nuget package

ghost commented 5 years ago

If you want to ship .proto files in a nuget package for reuse in .proto files in your own project you can do this:

<ItemGroup>
  <PackageReference Include="MyCompany.ProtoBuf" Version="1.0" GeneratePathProperty="true" />
</ItemGroup>

<ItemGroup>
  <Protobuf Include="Grpc/*.proto" GrpcServices="Both" CompileOutputs="true" AdditionalImportDirs="$(PkgMyCompany_ProtoBuf)/content/Proto" ProtoRoot="Grpc" />
</ItemGroup>

The key here are the GeneratePathProperty and AdditonalImportDirs attributes.

In the MyCompany.ProtoBuf package you need to have something like this:

<ItemGroup>
    <Content Include="Proto\**\*.proto">
      <CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory>
    </Content>
</ItemGroup>

to get the proto files inlcuded.

dameeks commented 5 years ago

I wasn't able to get the AdditionalImportDirs to work. It did not work with a non-nuget proto file either (e.g. the proto was located at c:\temp). I ended up doing this:

  <ItemGroup>
    <Protobuf Include="$(PkgMyCompany_ProtoBuf)\Content\Proto\*.proto" GrpcServices="server" CompileOutputs="true" />
  </ItemGroup>
  <ItemGroup>
    <Protobuf Include="gRPC\**\*.proto" GrpcServices="server" CompileOutputs="true" />
  </ItemGroup>

ghost commented 5 years ago

@dameeks: What does this give you? Don't you get the generated client/server already by just referencing the package. Why do you need to generate it again in the consuming project?

dameeks commented 5 years ago

@roederja2 My NuGet package contains a single proto file. Adding a reference to the NuGet package adds the proto file to my csproj, but it does build a gRPC client/server. The only way I got it to build was to add a second ItemGroup. I also tried using

<Protobuf update="$(PkgMyCompany_ProtoBuf)\Content\Proto\*.proto" >

within the same ItemGroup as the include but that did not work either.

dmitrydvm commented 5 years ago

<ItemGroup>
    <Protobuf Include="*.proto;" />
</ItemGroup>

by @JamesYangLim works fine for me.

ipzKellyR commented 4 years ago

This is very similar to an issue I'm trying to solve. I can't use protos imported into other protos that are linked through a class library.

If I use the AdditionalImportDirs flag as seen it will compile, but then when you actually try to reference the something.proto messages in the org.service.proto messages then it says file not found.

<Protobuf Include="..\Auth0Service.Grpc\Protos\org.service.proto" GrpcServices="Client" AdditionalImportDirs="Protos">         
  <Link>Protos\org.service.proto</Link>
</Protobuf>
<Protobuf Include="..\Auth0Service.Grpc\Protos\something.proto" GrpcServices="Client" ProtoCompile="true" Access="Public" >
  <Link>Protos\something.proto</Link>
**</Protobuf>**

RichyP7 commented 4 years ago

Reading the intro more exact would have saved us a lot of time.

Let me also stress that this is distributing the proto files and not client libraries. You want to avoid distribution of clients that are build with an older version of proto as the one you are using in your project.

@roederja2 ´s advice for shipping protos works perfectly. Even we found a way to work around the issue, there should be a solution (like a flag or something similar) to easily import the proto files from a consumed nuget. Maybe the packaging of proto files in a standardized way would help.

stueeey commented 4 years ago

@RichyP7 I've done exactly this, they didn't approve it :/ https://github.com/grpc/grpc/pull/20071

neilwashere commented 4 years ago

Thought I'd weigh in as I'm trying to do exactly this: package proto's in nuget package for redistribution such that models can be used in other projects requiring the proto definitions. Specifically sharing 'model' definitions that will be used to build other services. I can package and distribute but inclusion is simply not working using the above methods. dotnet build will setup content links - I can even see them relative to local proto files in the IDE - but will fail with missing files.

I can include packaged protos locally only if I manually setup a symlink to the packaged proto files. Maybe this is specific to Linux (I don't have a window machine to test on) and therefore a separate issue altogether.

neilwashere commented 4 years ago

I have a solution that I can live with for now but it is by no means ideal. It also assumes running in a *nix env which is fine for my needs. Maybe it can help another soul hitting the same issue (MSbuild links are not honoured when compiling local protos using imports from pkg)

<ItemGroup>
    <PackageReference Include="My.Protos" Version="1.0.0" GeneratePathProperty="true" />
</ItemGroup>
<ItemGroup>
     <Protobuf Include="protos/**/*.proto" />
</ItemGroup>
<PropertyGroup>
    <PreBuildEvent>
        ln -sf $(PkgMy_Protos)/content/Some/Protos $(MSBuildProjectDirectory)/protos/foo
    </PreBuildEvent>
</PropertyGroup>

stueeey commented 4 years ago

@neilwashere if that is all you want to do you can just do this:

<Protobuf Include="$(PkgMy_Protos)/content/Some/Protos/**/*.proto"/>

Includes are cumulative so you can leave your existing Protobuf include where it is if you have local proto files too.

My PR which hasn't gone in would allow you to just put your proto files in /protobuf/grpc in your nuget package and they would be automatically found and included without any MSBuild witchery

RichyP7 commented 4 years ago

@RichyP7 I've done exactly this, they didn't approve it :/ grpc/grpc#20071

I really think this would be an huge improvement but maybe it doesn´t fit the current modular .net approach. Maybe a plugin for nuget would be the right approach like in java. maven protocol buffer

neilwashere commented 4 years ago

@stueeey thanks for the reply. Unfortunately in my case, where I need to reference the raw packaged protos in order for my local ones to build, MSBuild just says no. Without an explicit symlink the compiler errors with: File not found and my_local_proto.proto(5,1): import "my/external/referenced/protofile.proto" was not found or had errors :feelsgood: No amount of adjusting the import paths for reference seems to work.

stueeey commented 4 years ago

@neilwashere I can probably give you some msbuild magic for that when I'm home

emanuel-v-r commented 4 years ago

How can it be done when referencing directly another project that contains the proto file? I have this in the project that contains the file:

<ItemGroup>
     <Content Include="Protos\**\*.proto">
               <CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory>
      </Content>
</ItemGroup>

And then I have this in the project that is referencing the project containing the file:

<ItemGroup>
    <Protobuf Include="**/*.proto;"  GrpcServices="Client" />
</ItemGroup>

Still, seems to not get detected.

tverboon commented 4 years ago

This is how I do it. I create a client NuGet package per client. This is what goes in the Client project:

<ItemGroup>
  <Protobuf Include="..\Server.Project\Protos\*.proto" GrpcServices="Client" AdditionalImportDirs="..\Server.Project\" />
</ItemGroup>

emanuel-v-r commented 4 years ago

This is how I do it. I create a client NuGet package per client. This is what goes in the Client project:
<ItemGroup>
  <Protobuf Include="..\Server.Project\Protos\*.proto" GrpcServices="Client" AdditionalImportDirs="..\Server.Project\" />
</ItemGroup>

Ended up with that, thanks!

ChiefInnovator commented 4 years ago

I just found this discussion. I wish I would have found this discussion sooner. We just spent the last few weeks implementing the exact same solution which was to share protos as content via a NuGet package.

My team has mixed feelings on this. The concern they have is that we are checking in the protos twice or more. Once in the main repo for the protos and then again for any consuming project. Personally, I am not worried about this. There are bigger problems to solve, hence my next point.

Where this falls down is when you want to share common protos across multiple services. This wouldn't be an issue if a consumer only talked to one service. We have situations where one consumer can talk to multiple services. After all, we are in a world of microservices. Also, we do not cross the streams. Our clients are generated in separate NuGet packages and by extension, separate assemblies. One per service!

Some languages this may not be a problem, but in .NET, it is a big deal. The challenge is that common types generated from the common protos are seen in two different assemblies.

This should be doable. After all, we do not have to have the protos for the wellknown protos (ex. timestamp) from Google. Those are determined by the gRPC tooling and you are allowed to reference them. Why can't we do the same with our own wellknown (i.e. common protos)?

If I am missing any information, please educate me.

Reference to the issue I just created as well as my current path to solving this problem which is different, but similar. https://github.com/grpc/grpc-dotnet/issues/988

stueeey commented 4 years ago

Hey @JamesNK @jtattermusch , what needs to happen here? It seems like there is demand for this, what needs to happen for the pr to be accepted?

BetimBeja commented 3 years ago

Hello, if you need an alternative way to share .proto files and other artefacts using NuGet packages take a look here https://github.com/albanian-xrm/greeter.shared and here https://www.linkedin.com/pulse/source-only-nuget-packages-albanianxrm

Liversage commented 2 years ago

I have created a demo project based on @stueeey's comment. To me this is a great way to solve the problem, but unfortunately Visual Studio 2022 and Visual Studio Code (OmniSharp) will display red squiggles in the editor leading to a very bad editing experience. I have provided feedback to Visual Studio but unfortunately I seem to be the only one suffering from this.

BetimBeja commented 2 years ago

@Liversage have you tried my way in the previous comment?

Liversage commented 2 years ago

@BetimBeja No, the shared project (.shproj) seems like an old and almost undocumented feature of Visual Studio. Does it work with dotnet command line tools? Will it continue to work in the future after I no longer work on my current project? I guess I should try it out but I haven't invested the time and effort yet.

BetimBeja commented 2 years ago

You actually don't need to use the Shared Project... you can configure the .targets and .props with the necessary "imports" and they will get automatically included in the consuming project.

Liversage commented 2 years ago

I have created a demo project based on @stueeey's comment. To me this is a great way to solve the problem, but unfortunately Visual Studio 2022 and Visual Studio Code (OmniSharp) will display red squiggles in the editor leading to a very bad editing experience. I have provided feedback to Visual Studio but unfortunately I seem to be the only one suffering from this.

I have created a solution/work around that fixes the red squiggles in the editor. It adds a bit of friction to using .proto files from a NuGet package but still solves the fundamental problem. For those curious this solution is on a branch in my demo project.

runenilsenoe commented 2 years ago

@JamesNK we are trying to use the nuget packaging approach for building Rest APIs with the gRPC HTTP API concept. When referencing a nuget package with generated server code the application both builds and runs properly, but the services are not recognized and generated as controllers, swagger also cant find the server-code from nuget. We use a shared "contracts" mono-repo for the .proto files and code related to generation. Is there a good example of this workflow?

pophils commented 2 years ago

If you want to ship .proto files in a nuget package for reuse in .proto files in your own project you can do this:
<ItemGroup>
  <PackageReference Include="MyCompany.ProtoBuf" Version="1.0" GeneratePathProperty="true" />
</ItemGroup>

<ItemGroup>
  <Protobuf Include="Grpc/*.proto" GrpcServices="Both" CompileOutputs="true" AdditionalImportDirs="$(PkgMyCompany_ProtoBuf)/content/Proto" ProtoRoot="Grpc" />
</ItemGroup>
The key here are the GeneratePathProperty and AdditonalImportDirs attributes.

In the MyCompany.ProtoBuf package you need to have something like this:
<ItemGroup>
    <Content Include="Proto\**\*.proto">
      <CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory>
    </Content>
</ItemGroup>
to get the proto files inlcuded.

This was helpful although the build failed due to an import statement in one of the proto files . The below fix the issue for me:

 <ItemGroup> 
    <PackageReference Include="MyCompany.ProtoBuf" Version="1.0" GeneratePathProperty="True" /> 
  </ItemGroup>

<ItemGroup>
<Protobuf Include="$(PkgMyCompany_ProtoBuf)\Content\Proto\**\*.proto" GrpcServices="Both"
              ProtoRoot="$(PkgMyCompany_ProtoBuf)\Content\Proto"/>

AntonSmolkov commented 1 year ago

Looks like @JamesNK have figured out how to properly make nugets with proto-files here - https://github.com/dotnet/aspnetcore/pull/44999. Hope it will be documented as a common way to do it.

fmg-lydonchandra commented 1 year ago

Hi @llawall, thank you for helping. Your approach works very well and very simple. I have the following example that works.

  <ItemGroup>
    <PackageReference Include="Google.Protobuf" Version="3.6.1" />
    <PackageReference Include="Google.Protobuf.Tools" Version="3.6.1" />
    <PackageReference Include="Grpc" Version="1.22.0" />
    <PackageReference Include="Grpc.Core" Version="1.22.0" />
    <PackageReference Include="Grpc.Tools" Version="1.22.0">
      <PrivateAssets>all</PrivateAssets>
      <IncludeAssets>runtime; build; native; contentfiles; analyzers; buildtransitive</IncludeAssets>
    </PackageReference>
    <PackageReference Include="my.package.1" Version="2.3.1" GeneratePathProperty="true"/>
    <PackageReference Include="my.package.2" Version="1.3.4" GeneratePathProperty="true"/>
  </ItemGroup>

  <ItemGroup>
    <Protobuf Include="*.proto" AdditionalImportDirs="$(Pkgmy_package_1)\proto;$(Pkgmy_package_2)\proto" />
  </ItemGroup>

Thanks! This below works for our specific project.

     <Protobuf 
            Include="$(Pkgmy_package_1)\content\protos\**\*.proto" 
            ProtoRoot="$(Pkgmy_package_1)\content\"
            AdditionalImportDirs="$(Pkgmy_package_1)\content\;"
    />

karpikpl commented 1 year ago

Does this solution also works with ProjectReference ? or it requires nuget publish?

fmg-lydonchandra commented 1 year ago

This requires nuget publish. Does NOT work with ProjectReference as far as I know.

karpikpl commented 1 year ago

This requires nuget publish. Does NOT work with ProjectReference as far as I know.

thanks. I've ended up referencing protos by path in the same solution. I still think doing it via ProjectReference would be neater.

tonydnewell commented 12 months ago

Grpc.Tools BUILD-INTEGRATION.md has now been updated with how to share proto files.

grpc / grpc-dotnet

Enable referencing proto files in NuGet packages #183