Idomatic APIs #28

Open jessebraham opened 2 years ago

This issue was migrated from GitLab. The original issue can be found here: https://gitlab.com/susurrus/serialport-rs/-/issues/112

Opening this issue to discuss the possibility of moving towards more idiomatic Rust APIs in the future. The changes I'm suggesting would be significant API changes, so would require a major-version update to implement.

Concrete Types

In the Rust standard library, types like File and TcpStream are implemented as structs with platform-dependent internals. In serialport, the user has to decide whether to use platform dependent or platform agnostic types ahead of time when developing their application. This forces the developer into two possible paths which both have some noteworthy downsides.

Going the platform-agnostic route, you will be using Box<dyn SerialPort> as your port type. This causes an unnecessary dynamic dispatch and makes it very difficult if you later discover you do need some platform specific functionality. Since the correct implementation of SerialPort on a given platform is always known at compile time, there will never be a case where client code actually needs dynamic dispatch to work with both TTYPort and COMPort polymorphically. If you start out using this platform-agnostic setup and later find out you need to set some platform-specific setting, you have to change all code leading to the spot where the platform-specific setting is used into platform-specific code. That is, Box<TTYPort> is convertible to Box<dyn SerialPort>, but because SerialPort doesn't inherit from Any or provide similar downcast methods, there's no way to get back a TTYPort from dyn SerialPort if you only need a platform-specific method in one place.

Going the platform-specific route, you will be using either TTYPort or COMPort, depending on platform. Because these two types have different names, this makes it fairly inconvenient to write platform-agnostic code for parts of your application that don't depend on a particular platform. You either have to use a bunch of generics or write your own platform-specific aliases of #[cfg(unix)] type Serial = TTYPort;, etc.

This proposal is to follow the pattern of the standard library for the SerialPort type: make the main type provided by the crate a concrete struct with opaque internals that vary by platform. Platform specific methods can either be provided directly on that type conditional on the platform, or can be provided by a platform-specific extension trait, similar to MetadataExt.

Advantages

One concrete type used for both platform-specific and platform-agnostic operations.

Avoids unnecessary dynamic dispatch.
Clients can easily use platform-specific features on an as-needed basis without having to change types.

fn somewhere_down_in_my_app(port: &mut SerialPort) -> Result<()> {
// do some stuff...
// I was being platform agnostic, but now I need to do one platform-specific thing temporarily. NBD!
#[cfg(unix)] {
    use serialport::unix::SerialPortExt;
    port.set_exclusive(false)?;
}
// ...
Ok(())
}

Simpler type-name.

Disadvantages

Clients cannot provide their own implementations of SerialPort.

As I see it, the core value of this crate is in providing an implementation of serial ports for Rust across several major platforms, not in providing a generic interface to be used across many external implementations. If there are other useful implementations of the SerialPort trait, they could instead be added to this as additional platform-conditional implementations through pull-requests.

If there are clients that need to generically handle both the SerialPort type and their own custom types, but where they don't have a fully-new implementation of SerialPort worth adding to this crate, then I would suspect that they could likely cover much of their functionality through other traits, either existing ones like Read and Write or their own custom traits implemented just for the shared functionality that they need, just as one could implement custom traits for File.
Concrete types are harder to mock in tests.

I would suspect that a lot of client code that interacts with SerialPort but wants to test against a "mock" would be reasonably served by being generic over either Read, Write, or Read + Write, and then just using standard library types that implement those, such as Vec or Cursor.

That said, there is the possibility of needing to test interactions with actual serial-port features, such as setting baud rate, in an environment where you don't want to or can't open an actual port. So there is an actual case for having a trait that covers those methods. If that's a significant concernt, I would propose having a trait like SerialPortSettings which provides all the methods for changing baud rate, etc. For convenience in the common use case I might still have the same methods available directly on the concrete SerialPort struct (so they work without importing the trait), but having a settings trait available would allow for those kinds of tests, if you think that's a significant thing clients want to be able to do. I will note for example that TcpStream provides the methods for setting timeouts directly on the struct, and if users want to use that generically to check it in tests, they have to create their own trait for it.

Implemeting for Immutable References

While the Read and Write traits both take &mut self in arguments, many standard library types for working with files/sockets and similar objects implement Read and Write for both T and &T, thereby allowing both reads and writes even if one only has an immutable reference to the type.

I don't know if this would work safely for serial ports, since I don't know what the concurrency-safety of serial ports is across different operating systems. However, if it is possible to safely share the RawFd/RawHandle of a serial port across multiple threads, then I think it would be good to impl Read for &TTYPort etc., as a way to allow easier concurrent use. It's certainly possible to use try_clone/try_clone_native to get multiple handles to the same serial port to share one port in multiple threads, but if a single handle could be shared between threads, that might be more convenient in some cases. For example, you could then share a serial port in an Arc without having to make an OS-level copy of the handle every time you share it, or could share by simple reference-borrow in a case like crossbeam::scope.

Discussion/Implementation

The proposed change to concrete types in particular would be a very significant change in API, even compared to the kinds of changes you've made in previous major-version bumps, so such a change is probably not to be made lightly on a crate with a couple hundred downloads per day.

The suggestion to use a concrete type is made from the perspective of coming here from the standard library, but I don't really have great context for why the library was designed this way in the first place, so let me know if there's strong reasons for setting up the library this way.

If any of this does seem like a good idea, I would be able to to put together pull requests for it.

If you're interested in this, I can port my pull request from gitlab and resubmit it here.

Thank you for your offer!

I admittedly have only given the issue a quick read, but I think you make some good arguments! I figure now is as good a time as any to make API changes as well. I will need to read through the original comment thread again at some point.

I don't have a timeline for when I will get to it, but if you're willing to open a PR with your changes here I would encourage you to do so!

Very good suggestions.

If I'm understanding this correctly, I think that the final bullet item under your Disadvantages section is actually a 'feature' suggestion and should be moved out of the 'Disadvantages' section. Or maybe I'm misreading it?

The disadvantage is that if the Serialport type is a concrete (struct) type rather than a trait, it is harder to mock out in unit tests. I just then spend much more time describing ways to make it easier to work around that disadvantage.

On Thu, Jun 8, 2023, 4:05 PM JWR_FABNexus @.***> wrote:

Very good suggestions.

If I'm understanding this correctly, I think that the final bullet item under your Disadvantages section is actually a 'feature' suggestion and should be moved out of this section. Or maybe I'm misreading it?

— Reply to this email directly, view it on GitHub https://github.com/serialport/serialport-rs/issues/28#issuecomment-1583267067, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABKGF3JGOR3YB2DBHCTXDA3XKIV63ANCNFSM5NYUOBKA . You are receiving this because you commented.Message ID: @.***>

The disadvantage is that if the Serialport type is a concrete (struct) type rather than a trait, it is harder to mock out in unit tests. I just then spend much more time describing ways to make it easier to work around that disadvantage.

Does the serial port itself need to be a trait for mocking? I'm wondering to which extend mocking will be used and makes sense in unit tests. I've seen https://github.com/dbrgn/embedded-hal-mock where mocking happens with respect to traits of embedded-hal. This allows for example to mock I2C communication at the level of the I2C traits like i2c::Read, i2c::Write, and i2c::WriteRead but not the initialization of a certain I2C controller.

Wouldn't mocking io::Read and io::Write for a serial port be a Pareto-style solution for mocking general serial communication? Just as a first thought, mocking things like changing baud rate, parity, ... "in-flight" looks like where things get rough. In contrast, just mocking the read and write traits seems pretty straightforward to me. And if there is a really a good use case, we could still introduce for example a configuration trait for it.

That is esssentially what my proposal is -- make SerialPort a struct and let people mock read/write and optionally add a trait for the serialport controls, if needed. Note that because of how traits work in rust, anyone who really needs it could just write their own 'serialport controls' trait to test against and implement it for our SerialPort type, so I don't even think it's necessary to provide such a trait, but I wanted to describe the option in case other people thought it was important.

On Thu, Jun 8, 2023, 4:49 PM Christian Meusel @.***> wrote:

The disadvantage is that if the Serialport type is a concrete (struct) type rather than a trait, it is harder to mock out in unit tests. I just then spend much more time describing ways to make it easier to work around that disadvantage.

Does the serial port itself need to be a trait for mocking? I'm wondering to which extend mocking will be used and makes sense in unit tests. I've seen https://github.com/dbrgn/embedded-hal-mock where mocking happens with respect to traits of embedded-hal. This allows for example to mock I2C communication at the level of the I2C traits like i2c::Read, i2c::Write, and i2c::WriteRead but not the initialization of a certain I2C controller.

Wouldn't mocking io::Read and io::Write for a serial port be a Pareto-style solution for mocking general serial communication? Just as a first thought, mocking things like changing baud rate, parity, ... "in-flight" looks like where things get rough. In contrast, just mocking the read and write traits seems pretty straightforward to me. And if there is a really a good use case, we could still introduce for example a configuration trait for it.

— Reply to this email directly, view it on GitHub https://github.com/serialport/serialport-rs/issues/28#issuecomment-1583435440, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABKGF3PBAVG7KH4U2JTDZRDXKJCH5ANCNFSM5NYUOBKA . You are receiving this because you commented.Message ID: @.***>

Per 'sirhcel's comment above, with regards to this crate's unit test having the ability to operate without physical serial ports (using mocked hardware), I believe a key question to ask is how beneficial this approach can realistically be. Writing code that in-fact copies data streams around in memory (rather than actually transferring over a physical port) is quite difficult to accurately and precisely implement in a way that comes close to modeling real hardware with respect to behavior and timing. It's the intricacies of various platform I/O and serial hardware port behavior that offers the most benefit while unit testing. I also recognize that in an ideal world one would like to unit test their code without operator involvement and in a fully automated fashion, but in the world of physical hardware this is rarely feasible.

That is esssentially what my proposal is -- make SerialPort a struct and let people mock read/write and optionally add a trait for the serialport controls, if needed. Note that because of how traits work in rust, anyone who really needs it could just write their own 'serialport controls' trait to test against and implement it for our SerialPort type, so I don't even think it's necessary to provide such a trait, but I wanted to describe the option in case other people thought it was important.

Alright, then we are on the same page. This is the way I would like to go forward with it. And just not confusing things: Are you referring to PR #34 from https://github.com/serialport/serialport-rs/issues/28#issuecomment-1047831210?

Per 'sirhcel's comment above, with regards to this crate's unit test having the ability to operate without physical serial ports (using mocked hardware), I believe a key question to ask is how beneficial this approach can realistically be. Writing code that in-fact copies data streams around in memory (rather than actually transferring over a physical port) is quite difficult to accurately and precisely implement in a way that comes close to modeling real hardware with respect to behavior and timing. It's the intricacies of various platform I/O and serial hardware port behavior that offers the most benefit while unit testing. I also recognize that in an ideal world one would like to unit test their code without operator involvement and in a fully automated fashion, but in the world of physical hardware this is rarely feasible.

Sorry I did not read the original message right in the first place. Now I understand that @zstewar1 talked about the possibilities to mock the proposed static serial port. Yes, a static serial port struct would be harder to mock completely. But to me this does not seem relevant as - as you already said - mocking timing behaviour and other nitty gritty details won't get you both finished and far.

So from my Pareto-perspective, implementing io::Read and io::Write is fine for almost anyone. And if there is a significant demand, I bet we will learn about it when working towards 5.0.

In theory, we could statically analyze code of our known and actively maintained dependents. But I have only vaguely heared about this and crater so far. Any show and tell is welcome.

serialport / serialport-rs