bufbuild / protobuf-es

Protocol Buffers for ECMAScript. The only JavaScript Protobuf library that is fully-compliant with Protobuf conformance tests.
Apache License 2.0
1.15k stars 68 forks source link

Generated typescript is importing a non existent file #457

Closed juanpmarin closed 1 year ago

juanpmarin commented 1 year ago

Hi! I'm using buf to generate typescript code for my connect service.

import "google/type/money.proto"; I'm using that import in my proto file, and the generated typescript file is trying to import this file: import { Money } from "../../../../google/type/money_pb.js"; which does not exist

version: v1
plugins:
  - plugin: buf.build/protocolbuffers/go
    out: gen/go
    opt: paths=source_relative
  - plugin: buf.build/bufbuild/connect-go
    out: gen/go
    opt: paths=source_relative
  - plugin: buf.build/bufbuild/validate-go
    out: gen/go
    opt: paths=source_relative
  - plugin: buf.build/bufbuild/es
    out: gen/ts
    opt: target=ts
  - plugin: buf.build/bufbuild/connect-es
    out: gen/ts
    opt: target=ts

That's my buf gen config

timostamm commented 1 year ago

Hey Juan,

I suppose you have a dependency on buf.build/googleapis/googleapis in your buf.yaml? If you run buf generate, only the code for your module is generated, not the code for your dependencies.

In Go, you don't notice this, because money.proto has the following file option:

option go_package = "google.golang.org/genproto/googleapis/type/money;money";

When protoc-gen-go generates a proto file that imports money.proto, the import is redirected to this Go package, which is a precompiled version of googleapis.

There is no file option for a JS package, and there is no precompiled version of googleapis for protobuf-es either, so protoc-gen-es generates an import to a local file, just like any other regular import. The only exception are the well-known types, which are ship with every protobuf compiler.

To generate a module and dependencies, you can use buf generate --include-imports. The buf CLI currently does not have an option to specify this behaviour per plugin, so you will have to split your buf.gen.yaml into two files, one for Go, one for TypeScript, and generate the latter with the --include-imports flag.

See also the related report https://github.com/bufbuild/connect-es/issues/262, and https://github.com/bufbuild/buf/issues/1407, which was opened to improve the behavior.

I'm going to close this issue because I don't see any reasonable way for protobuf-es to improve the experience, but I'll raise it again with the team. Thank you for opening the issue!

juanpmarin commented 1 year ago

@timostamm is there a way to whitelist only some imports to be generated?

timostamm commented 1 year ago

There are several options to generate select parts, but in general, I think that it is desirable to avoid maintaining filters, and keep the generate commands as simple as possible.

juanpmarin commented 1 year ago

@timostamm thanks!

Palmik commented 1 year ago

Would it make sense to have an option, in the protobuf-es / connect-es plugin, to specify "module path prefix". This could be either directory, or a package. If you specify a package, e.g. "@acme/protobuf", then the generated code would contain import ... from '@acme/protobuf/foo.proto' instead of e.g. import ... from '../../acme/protobuf/foo.proto.

This would allow users to split modules into npm packages.