mit-pdos / noria

Fast web applications through dynamic, partially-stateful dataflow
Apache License 2.0
4.98k stars 242 forks source link

Trouble Embedding noria_server #112

Closed zack-emmert closed 5 years ago

zack-emmert commented 5 years ago

I am currently trying to embed noria_server into a testing project, and I can't seem to figure out what needs to go into Cargo.toml to pull in the noria_server code. Attempting to only include the noria crate in Cargo.toml throws an error stating that the noria_server crate cannot be found, while trying to pull it from GitHub directly throws a 404 error, presumably because noria_server lives in a subfolder of the noria repo. Advise on this would be much appreciated.

fintelia commented 5 years ago

I think your issue is related to this: https://github.com/rust-lang/cargo/issues/6211

Try using a more recent rust nightly and/or include cargo-features = ["edition"] in your Cargo.toml.

zack-emmert commented 5 years ago

Unfortunately, that doesn't seem to be the issue. I am on 1.31.0-nightly (updated Oct 28), and this is the build output:

--> src/main.rs:2:1
  |
2 | extern crate noria_server;
  | ^^^^^^^^^^^^^^^^^^^^^^^^^^ can't find crate

Output remains the same with or without cargo-features = ["edition"]

ms705 commented 5 years ago

What does your test project do? If your code plans to interact with a Noria installation (i.e., you're trying to write a test application that uses Noria as a backend), you'll want to use the noria crate (which provides the API bindings), not noria_server.

You would only need to embed noria_server if you are trying to extend our server implementation itself (i.e., wrap a crate around the backend itself).

zack-emmert commented 5 years ago

I'm trying to write an application that uses Noria as a storage system, but it's client-only so I want to bundle the storage backend with the application. In a nutshell, the frontend and backend would be part of the same application.

ms705 commented 5 years ago

I see! In that case, you do indeed need noria_server, and you want to use build_local on a ControllerBuilder to generate the necessary ControllerHandle. As an example, see the TPC-W benchmarker.

Without more context, it's hard to tell why importing noria_server fails; can you provide your Cargo.toml?

I suspect a Cargo.toml entry like this one in noria-benchmarks should work for you.

zack-emmert commented 5 years ago

The problem is that I'm not sure what should go in my Cargo.toml. The file from noria-benchmarks you linked to requires that I download and keep updated my own copy of noria-server. Can I outsource this task to Cargo?

jonhoo commented 5 years ago

Let's see.. This works fine for me:

$ cat Cargo.toml
...
[dependencies]
noria = { git = "https://github.com/mit-pdos/noria.git" }
$ cat src/main.rs
extern crate noria;
...

As does this:

$ cat Cargo.toml
...
[dependencies]
noria = { git = "https://github.com/mit-pdos/noria.git" }
noria-server = { git = "https://github.com/mit-pdos/noria.git" }
$ cat src/main.rs
extern crate noria;
extern crate noria_server;
...

Note that we currently always build Noria with a particular nightly (2018-10-01), so you may need to add an override for that to work:

$ rustup override set nightly-2018-10-01
ms705 commented 5 years ago

In particular, it looks like the latest nightly crashes with a compiler bug while trying to compile one of Noria's dependencies (hyper), so the override is needed.

jonhoo commented 5 years ago

https://github.com/rust-lang/rust/issues/55376

zack-emmert commented 5 years ago

So I set the dependencies accordingly, and it compiles just fine on the latest nightly.