Support for rust, *and other languages*, backends

[arthurfiorette]

Describe the feature you'd like to request

I saw that the turborepo's go code supports multiple backends, and, in this moment, there's only backends for javascript.

Enabling turborepo to be programming language agnostic would be a really good, and i'm sure that turborepo with it's unique features can really exceed other solutions out there.

Describe the solution you'd like

I would like that turborepo starts supporting other languages, and if i could choose, i would start with Rust.

I know that turborepo depends a lot at package.json and it's scripts. And i also know, like rust, that there's some languages without this "workflows" built-in support. For rust, probably it'll be used cargo-make or Makefile directly, but that is a point to consider before coding.

Describe alternatives you've considered

Open for suggestions...

[samchouse]

For now you can add a package.json to the project and add scripts.

[jsoneaday]

Interesting idea but ironically in order to do that it would probably mean that turborepo would have to build their own config system beyond yarn workspaces and npm. Since in order to support any package/config system (cargo, go, gradle, etc) they would have to not be married to a particular one.

[jennysharps]

This would be very exciting as a feature

[nasso]

Interesting idea but ironically in order to do that it would probably mean that turborepo would have to build their own config system beyond yarn workspaces and npm. Since in order to support any package/config system (cargo, go, gradle, etc) they would have to not be married to a particular one.

or... turbo needs to understand the various different kinds of manifests (maybe through plug-ins/add-ons?)

at the end of the day, turbo just needs to build a dependency graph, and Cargo.toml files already give all the information necessary to do that. as long as turbo can understand the [dependencies] table, it can build the dependency graph, and can do its job.

for scripts, in the case of Cargo, the manifest supports having extra settings for external tools, which could be used to reimplement a kind-of scripts table containing the different scripts to implement:

[package]
name = "super_cool_wasm_package"
version = "0.1.0"
edition = "2021"

[dependencies]
# ...

[package.metadata.turbo.scripts]
build = "wasm-pack build ..."
lint = "cargo clippy"
format = "cargo fmt"
test = "cargo nextest run"

it will also probably need a way to express dependencies between packages written in different languages. that could also probably be encoded in the manifest (package.json, Cargo.toml, etc..) in custom sections:

{
  "name": "super-cool-npm-package",
  "scripts": {
    // ...
  },
  "dependencies": {
    "other-cool-npm-package": "workspace:*",
    // ...
  },
  "turboDependencies": {
    "super_cool_wasm_package": "*",
    // (could also be an array since we probably never want to specify a version)
  },
}

this is just what i came up with in 2 minutes to illustrate my point, but it's probably not ideal and there's probably a better/cleaner solution. but i think introducing Yet Another Config File is not very desirable, especially if we can just extend the "native" ones to give turbo all the information it needs that the original format doesn't provide.

[alexbechmann]

Would it make sense to allow for a turbo-project.json file in a sub directory instead of a package.json for non-JS projects?

{
  "name": "my-python-project",
  "scripts": {
    "build": "poetry run build"
  },
  "dependencies": {
    "another-turbo-project": "*"
  }
}

Then you can support any language or script dir etc

[nasso]

the problem with this is that now you potentially have to create turbo-project.json files everywhere, duplicating information that might be present in another manifest already (e.g. a Cargo.toml), which makes it easier for them to get out of sync for example

+ what would happen if you have both a package.json and a turbo-project.json? does turbo-project.json take precedence? are the two "merged"? do they declare 2 distinct packages?

---

i think i like the idea of using extensions/plug-ins/add-ons to support more languages or package managers. rather than manually writing a turbo-project.json.

the extension/plug-in/add-on for the corresponding language would translate the "native" manifest (e.g. package.json, Cargo.toml, xmake.lua, my-own-package-manifest.json, etc.) into whatever a general-purpose turbo-project.json would contain, eliminating the risk of them getting out of sync and reducing the maintenance burden

[vilkinsons]

We're heavy Rust/TS users currently utilizing Turborepo, but considering switching to https://please.build/ for improved cross-language support.

@jaredpalmer As (IIRC) Vercel itself is a dual Rust/TS shop, is there any prospect of first-class support arriving in the future?

[jtlapp]

This feature is probably important for future-proofing Turborepo. Wasm is enabling any language to run in the browser, and while JS/TS may be universally dominant at the moment, this is probably not going to remain the case.

[jsoneaday]

This feature is probably important for future-proofing Turborepo. Wasm is enabling any language to run in the browser, and while JS/TS may be universally dominant at the moment, this is probably not going to remain the case.

Brilliant synopsis. I totally agree, but still think js will remain the most popular for webdev

[jtlapp]

Brilliant synopsis. I totally agree, but still think js will remain the most popular for webdev

Yeah, by "universally" I meant across all use cases. JS will remain dominant, but I imagine some languages will come to dominate certain use cases. Maybe one language will come to be preferred for animation, another for AI, etc.

[SimonBiggs]

For those coming here via Google, the Nx monorepo tool supports both rust and python via plugins:

https://nx.dev/plugins/registry https://github.com/cammisuli/monodon/tree/main/packages/rust https://github.com/lucasvieirasilva/nx-plugins/tree/main/packages/nx-python

[arjendevos]

Just add a package.json works but not when pruning for docker. Currently figuring out how to build it for go, but it seems that there could be some improvements made.

[balanza]

Possibly, a low-hanging fruit would be to make Turbo compatible with make.

make is the less opinionated task runner, and can fit any stack. If Turbo could handle a logic like this:

# pseudo
execute ($command)
  if `package.json` exists
    npm  run $command
  else if `Makefile` exists
    make $command
  else 
    # skip

Any other stack could be easily adapted to be a workspace.

[nasso]

Possibly, a low-hanging fruit would be to make Turbo compatible with make.

make is the less opinionated task runner, and can fit any stack. If Turbo could handle a logic like this:

# pseudo
execute ($command)
  if `package.json` exists
    npm  run $command
  else if `Makefile` exists
    make $command
  else 
    # skip

Any other stack could be easily adapted to be a workspace.

im afraid supporting make like that doesn't provide much value over just defining scripts in a dummy package.json...

[balanza]

im afraid supporting make like that doesn't provide much value over just defining scripts in a dummy package.json...

Not much, but package.json brings some other semantics that can be annoying (it's a "manifest" file as well as a dependency descriptor).

[alexbechmann]

I like the Makefile idea as it doesn’t feel right to add a package.json to a python project for example.

[nasso]

i've actually put a package.json in rust crates to model their dependency relationship and allow turbo to run scripts on them haha

it works but i agree that its ugly

[zmzlois]

i've actually put a package.json in rust crates to model their dependency relationship and allow turbo to run scripts on them haha

it works but i agree that its ugly

honestly at this point whenever I need a makefile I will just use a package.json

[thenbe]

I'll mention task since I haven't seen it mentioned here and I suspect those following this thread may find it interesting or useful as it is a modern, language-agnostic build tool for any repo or monorepo.

Instead of adding a dummy package.jsons, I'd love to have the ability to use turbo with a Taskfile as it offers a superior dev experience. Not only does a Taskfile supports comments(!), but it has a bunch of other features and is in active development. The tool itself is written in go but is designed to be used in any repo (it's language agnostic).

IMO, the "ideal" tool would be one that had turbo's remote caching and supported taskfiles for defining tasks and modeling dependencies.

[nasso]

IMO, the \"ideal\" tool would be one that had turbo's remote caching and supported taskfiles for defining tasks and modeling dependencies.

task looks like a nice tool but for loops in yaml make me uneasy 🥲

imo one of the best things about turbo is that beyond the turbo config itself, it Just Works with existing package.json files

maybe what we want is some sort of "adapter" mechanism that allows us to write adapters for different languages, build tools etc...

at the end of the day turbo only needs a way to build a dependency tree and run scripts/commands on individual packages