Example uses unavailable function compat

[flip111]

In the example here https://docs.rs/bolt-client/0.10.1/bolt_client/index.html There is this line

let stream = BufStream::new(stream).compat();

Where BufStream is imported with

use tokio::io::BufStream;

When i go to the documentation of this type https://docs.rs/tokio/1.15.0/tokio/io/struct.BufStream.html i don't see the method compat

---

Additionally i have a question. I would like to build up the stream and client and then store the client variable in another struct to make it available throughout my application. I can't quite get the type of Client right because i don't know what to fill in for S in Client<S: AsyncRead + AsyncWrite + Unpin> i tried bolt_client::Client<BufStream<bolt_client::Stream>> but couldn't verify it due to compile failure of compat. Would appreciate feedback on this type.

[0xSiO]

Hi @flip111, thanks for your questions.

For the bolt_client example, the line

use tokio_util::compat::*;

pulls in some compatibility traits for wrapping tokio types in a futures-compatible type, using the compat() method. See here for more details.

As for what types you can use to create a Client, any type implementing AsyncRead and AsyncWrite will work. You can:

• use my Stream type (with the compatibility layer described above) which covers tokio-based TCP/TLS streams using rustls,
• use a compatible IO type from another library (maybe async-std if you're using that)
• build your own type (maybe you want native TLS instead of rustls).

For the above approaches, I recommend also using a buffer of some kind (e.g. BufStream if using tokio). Alternatively, you can skip most of the work by using one of the provided connection pool adapters (deadpool-bolt, bb8-bolt, mobc-bolt), which set up a tokio-based Client for you. See the associated connection pool documentation for each of those libraries for more info on how to use those. I would also recommend the connection pool approach if you plan on creating multiple connections to your database, as you would only need to share the pool between different parts of your application.

Hope this helps!

[flip111]

I'll number this reply and put a topic per paragraph for clarity.

---

1. Original error i ran into about the compat method and tried to fix it with using tokio_util

I added this now to cargo.toml: tokio-util = { version = "0.6.9", features = ["compat"] }, still the error remains the same:

error[E0599]: no method named `compat` found for struct `BufStream` in the current scope
  --> src/main.rs:37:39
   |
37 |   let stream = BufStream::new(stream).compat();
   |                                       ^^^^^^ method not found in `BufStream<bolt_client::Stream>`

---

2. the type of Client I found this code now https://github.com/0xSiO/bolt-rs/blob/c0c40dd345686b82e7f5e87f3e16c29b28507e1e/mobc-bolt/src/lib.rs#L53

Not sure if i should stay with bolt_client::Client<BufStream<bolt_client::Stream>> as i have now or also use Client<Compat<BufStream<Stream>>> like in the mobc code.

---

3. Choosing a Async Read/Write type

I'm using actix which also is using tokio, so i thought that the provided Stream type in bolt_client would be the best choice (purely from the fact that i recognized the word tokio). I have otherwise no idea what the tradeoffs are between the 3 alternatives.

---

4. Using a connection pool.

A connection pool sounds like a good idea. I'm not sure which pool to choose which works best with actix-web can you recommend one?

Below the descriptions of each pool type:

deadpool

Deadpool is a dead simple async pool for connections and objects of any type.

This crate provides two implementations:

Managed pool (deadpool::managed::Pool)
    Creates and recycles objects as needed
    Useful for database connection pools
    Enabled via the managed feature in your Cargo.toml

Unmanaged pool (deadpool::unmanaged::Pool)
    All objects either need to be created by the user and added to the pool manually. It is also possible to create a pool from an existing collection of objects.
    Enabled via the unmanaged feature in your Cargo.toml

bb8

A full-featured connection pool, designed for asynchronous connections (using tokio). Originally based on r2d2.

Opening a new database connection every time one is needed is both inefficient and can lead to resource exhaustion under high traffic conditions. A connection pool maintains a set of open connections to a database, handing them out for repeated use.

bb8 is agnostic to the connection type it is managing. Implementors of the ManageConnection trait provide the database-specific logic to create and check the health of connections.

mobc

A generic connection pool with async/await support.

Opening a new database connection every time one is needed is both inefficient and can lead to resource exhaustion under high traffic conditions. A connection pool maintains a set of open connections to a database, handing them out for repeated use.

mobc is agnostic to the connection type it is managing. Implementors of the Manager trait provide the database-specific logic to create and check the health of connections.

---

5. Documentation about compat method

using the compat() method. See here for more details.

I also found that page saw a lot of mentioning of the word "compat", didn't see any method named compat. I'm not sure how to browse the documentation.

The BufStream page also doesn't mention any implementations of the traits that tokio_util::compat provides.

That is obviously not a problem with the bolt_client library, but i would like to properly understand the compat method that is used in the example, which is tough when i don't find documentation. Also with google i don't find it (searched 30 min+).

---

6. Request for change of example code and documentation

It would be nice if the example on the front page of the documentation website for this library had just a little bit more information https://docs.rs/bolt-client/0.9.1/bolt_client/

Some packages are required with respective features:

• bolt_client with tokio-stream https://docs.rs/bolt-client/0.9.1/bolt_client/enum.Stream.html
• tokio_util with compat https://docs.rs/tokio-util/0.3.1/tokio_util/compat/index.html
• tokio with io-util https://docs.rs/tokio/0.2.2/tokio/io/struct.BufStream.html

Possibly some type annotations could be useful as well. For example instead of

let mut client = result.unwrap();

let mut client: Client<Compat<BufStream<Stream>>> = result.unwrap();

---

7. example projects

Is there any simple open source project online that uses the bolt_client library? Maybe just as small as the example code that i can just download and run.

[0xSiO]

If you want to set up your own stream like in the library example, you need to import the traits from tokio_util::compat - the compat() method for tokio types like BufStream should be automatically provided. The specific type for Client<S> in this case would be Client<Compat<BufStream<Stream>>>, which is what I use in all the connection pool crates. It's definitely a bit confusing, but you can blame all the ecosystem fragmentation around async IO as Rust was growing these last few years. Hopefully we can settle on an interface in std eventually, but I chose to support the traits from futures for now.

If you want to use a connection pool, you don't need to worry about tokio-util::compat. As for which connection pool to use, it's mostly up to personal preference - deadpool seems to be the most popular of the three right now, so I'd probably go with that. A managed pool sounds like it would take the least effort. Probably best to share it across your application's route handlers via the Data extractor.

I will definitely look into improving the documentation! Having this discussion was valuable for understanding how people are approaching and using this project. Perhaps I should add some examples in an examples folder.

One project I know of that uses bolt-client is warpgrapher - they're currently using an older version, but many of the same concepts still apply in the current version.

[flip111]

Hey @0xSiO thanks for all the help. I took a look at warpgrapher tried to reproduce with deadpool what they were doing with mobc. I'm getting pretty stuck, could you take a look at this example project i made? https://github.com/flip111/deadpool_bolt_example I figured i would get this working first before i tried to integrate it with actix web. Possibly later i don't need the block_on and i can run it from main directly with this special actix annotation. I point to the websocket code because it's most similar to my code. My plan was to use .data() as you suggested and then in the index method put the client in the MyWebSocket::new() method https://github.com/actix/examples/blob/master/websockets/websocket/src/main.rs#L21