hyperium / tonic

A native gRPC client & server implementation with async/await support.
https://docs.rs/tonic
MIT License
10.01k stars 1.02k forks source link

Import common proto in each crate? #1386

Open frederikhors opened 1 year ago

frederikhors commented 1 year ago

Version

tonic v0.9.2
  tonic v0.9.2 (*)
  tonic-build v0.9.2
  tonic v0.9.2 (*)
  tonic-web v0.9.2
tonic v0.9.2 (*)

Platform

Win 10 64

Description

Hi people! Thanks for your amazing work!

I have this dir structure:

- Cargo.toml (workspace)
- src
  - common
    - Cargo.toml
    - src
      - common.proto

  - crates
    - players
      - Cargo.toml
      - build.rs
      - src
        - proto
          - players.proto (I would like to import src/common/src/common.proto here)

    - teams
      - Cargo.toml
      - build.rs
      - src
        - proto
          - teams.proto (I would like to import src/common/src/common.proto here)

    - others...

as you can see I have many crates under crates dir and I have a common.proto which I would like to import in my crates.

I'm using a build.rs for each crate because I cannot (right?) use one for the entire Cargo workspace (I tried: Cargo doesn't start build.rs in workspace root dir).

So I'm trying to import common.proto in file like this:

syntax = "proto3";

package players;

import "google/protobuf/timestamp.proto";

import "src/common/src/common.proto";

and in my VSCode this is working.

But when I start build.rs throws with:

error: failed to run custom build command for `players v0.1.0`

Caused by:
  process didn't exit successfully: `C:\prj\target\debug\build\orders-b13065cbe3fac50f\build-script-build` (exit code: 1)
  --- stdout
  cargo:rerun-if-changed=src/proto/players.proto
  cargo:rerun-if-changed=src/proto

  --- stderr
  Error: Custom { kind: Other, error: "protoc failed: src/common/src/common.proto: File not found.\r\nplayers.proto:6:1: Import \"src/common/src/common.proto\" was not found or had errors.\r\nplayers.proto:29:3: \"common.Pagination\" is not defined.\r\nplayers.proto:49:9: \"common.Page\" is not defined.\r\n" }
[Finished running. Exit status: 0]

My build.rs is:

fn main() -> Result<(), Box<dyn std::error::Error>> {
    tonic_build::configure()
        .out_dir("src/proto")
        .compile(
            &["src/proto/players.proto"],
            &["src/proto"],
        )?;

    Ok(())
}

How can I fix this?

mamaart commented 11 months ago

I have a similar issue except everything is in the same crate.

I would like to be able to import a protofile into another.

Compiling and accessing the members of each protocol sepparately works fine, but they cannot access eachother:

Error[E0433]: failed to resolve: could not find `shared` in the crate root
  --> /home/martin/dk-slack/website/target/debug/build/eventsservice-b3f9e3601151fe7f/out/user.rs:90:43
   |
90 |             tonic::Response<super::super::shared::EventList>,
   |                                           ^^^^^^ could not find `shared` in the crate root

error[E0433]: failed to resolve: could not find `shared` in the crate root
   --> /home/martin/dk-slack/website/target/debug/build/userservice-622161de9a99e87b/out/registration.rs:203:60
    |
203 |             request: impl tonic::IntoRequest<super::super::shared::UserInformation>,
    |                                                            ^^^^^^ could not find `shared` in the crate root
error[E0283]: type annotations needed for `ProstCodec<(), U>`
   --> /home/martin/dk-slack/website/target/debug/build/eventsservice-b3f9e3601151fe7f/out/user.rs:102:17
    |
102 |             let codec = tonic::codec::ProstCodec::default();
    |                 ^^^^^
...
106 |             self.inner.unary(req, path, codec).await
    |                        ----- type must be known at this point
    |
    = note: cannot satisfy `_: Message`
    = help: the following types implement trait `Message`:
              bool
              i32
              i64
              u32
              u64
              f32
              f64
              EventList
            and 55 others
    = note: required for `ProstCodec<(), _>` to implement `Codec`
note: required by a bound in `tonic::client::Grpc::<T>::unary`
   --> /home/martin/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tonic-0.10.2/src/client/grpc.rs:215:12
    |
205 |     pub async fn unary<M1, M2, C>(
    |                  ----- required by a bound in this associated function
...
215 |         C: Codec<Encode = M1, Decode = M2>,
    |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ required by this bound in `Grpc::<T>::unary`
help: consider giving `codec` an explicit type, where the type for type parameter `U` is specified
    |
102 |             let codec: ProstCodec<(), U> = tonic::codec::ProstCodec::default();
    |                      +++++++++++++++++++

Some errors have detailed explanations: E0283, E0433.
For more information about an error, try `rustc --explain E0283`.
For more information about this error, try `rustc --explain E0433`.
error: could not compile `eventsservice` (lib) due to 2 previous errors
.
├── eventsservice
│   ├── build.rs
│   ├── Cargo.toml
│   └── src
│       └── lib.rs
└── protos
      └── events_service
              ├── shared.proto
              └── user.proto
// ../protos/events_service/user.proto
syntax = "proto3";

import "google/protobuf/empty.proto";
import "shared.proto";

package user;

option go_package = "github.com/dk-slack/protos/events_service/golang/user";

service Events {
  rpc List ( google.protobuf.Empty ) returns ( shared.EventList );
}
// ../protos/events_service/shared.proto
syntax = "proto3";

package shared;

option go_package = "github.com/dk-slack/protos/events_service/golang/shared";

message EventList {
  repeated shared.Event events = 1;
}

message Event {
  enum EventType {
    WEEKLYMEETING = 0;
    WATERLINE = 1;
    HIGHLINE = 2;
    MEETING = 3;
    WORKSHOP = 4;
    AUDIT = 5;
    RIGGING = 6;
  }

  message Location {
    message Coordinates {
      float lng = 1;
      float lat = 2;
    }
    string name = 1;
    Coordinates coordinates = 2;
  }

  string title = 1;
  Location location = 2;
  EventType event_type = 3;
  uint64 time = 4;
}
// build.rs
fn main() -> Result<(), Box<dyn std::error::Error>> {
    tonic_build::configure().build_server(false).compile(
        &[
            "../protos/events_service/user.proto",
            "../protos/events_service/shared.proto",
        ],
        &["../protos/events_service"],
    )?;
    Ok(())
}
// src/lib.rs
mod events {
    tonic::include_proto!("user");
}

mod proto_shared {
    tonic::include_proto!("shared");
}

use proto_shared::event::EventType as ET;

use tonic::transport::Channel;

pub use shared::errors::UserEventError;

#[derive(Clone)]
pub struct UserEventsClient {
    cli: events::events_client::EventsClient<Channel>,
}

impl UserEventsClient {
    pub async fn new() -> Result<Self, UserEventError> {
        Ok(Self {
            cli: events::events_client::EventsClient::connect("http://localhost:50057")
                .await
                .map_err(|_| UserEventError::ConnectionError)?,
        })
    }
}

I had similar issues in golang when importing a package because I was using relative package names imports and go doesn't allow that in modules. So I was forced to specify a fully qualified package name in the option go_package and make a library that my other go modules could then import the grpc functionality from.

haylinmoore commented 10 months ago

Has anybody figured this out? I am currently trying to compile the openconfig gnmi GRPC proto files which import each other and cannot get tonic_build to work

mamaart commented 10 months ago

I figured out in my case the problem was my own fault.

I think I should have called this one "shared" and not "proto_shared"

mod proto_shared {
    tonic::include_proto!("shared");
}

I just had to rename it and another thing I was importing which was also called shared :-)

It was actually really clear in the error message. the other proto implementation was looking for super::super::shared. but I called it something else so it could not find it.