Are separate vty-* packages really needed?

Apologies if this has already been explained elsewhere; I read through https://github.com/jtdaugherty/vty/issues/260 and still don't understand the utility of the separate vty-unix, vty-windows, and vty-crossplatform packages. I'm super glad Windows support has been added though! Amazing work by everyone involved :)

I'm wondering why we can't have a single vty package that "just works" on both platforms. It seems to me that all this code could live in vty itself, and vty could contain the Cabal conditional that builds either the Unix or Windows version based on the platform you're on. Then users wouldn't have to add two separate packages when they want to use vty.

The two packages isn't a problem per se, but it's causing me pain when it comes to backwards compatibility. I maintain packages that depend on vty/brick, and thus far I've been able to keep total backward compatibility through CPP pragmas. But now, the fact that older versions of vty are standalone while newer versions require an extra package, is something I can't work around with CPP or Cabal conditionals. I'm forced to make a hard new release that depends on vty-6+ / brick-2+.

BTW, I think the problem could potentially be lessened for brick users if brick could just re-export certain key functions from vty-crossplatform such as mkVty. For many use-cases, brick users then wouldn't need a direct dependency on vty.

Hope this makes sense. Thanks as always for the great libraries.

Good questions, and ones that I thought a lot about when I was working with the folks who did the Windows implementation!

Here are some answers:

In terms of what the packages actually do, the platform packages (for Unix and Windows) provide the platform-specific terminal initialization code and vty itself provides the rendering algorithm, event dispatching, and image combinators/abstraction.
As far as why they're split up as they are: a key concern I had when adding Windows support was that I am not willing to maintain it personally. I'm not a Windows developer and I don't have plans to become one, so not only did I not have the expertise to write the Windows-specific code, I also didn't want to end up in a situation where that code was in a library that I maintained. Even if that code had gone into vty itself, any maintainer I got on board to help with that could vanish and I'd be left with code in my package that I don't want to deal with. And beyond that, it became very clear, once vty-windows development was underway, that putting them both in the same package would have been a huge mess that would have been difficult to avoid. I decided that it would be best to divide up the Vty implementation along both platform and maintenance lines by splitting up the core and "driver" code. That was also intended to serve as a test of the core library's interface abstractions, since if I couldn't divide things up this way, that meant something was too Unix-y in the core and needed to be generalized.
As far as the approach we took to do this, the approach we used avoids CPP (which I consider a brittle and unmaintainable approach to be avoided unless you can keep it very localized - not possible in this case). The amount of platform-specific differences are great enough that putting all the code into one package and CPP'ing our way to a working build would be hard to maintain.
Re-exports are a way to deal with some of the pain that this approach might cause, but since the application developer needs to add only a package dependency and change an import or two in most cases, I don't consider that a very big burden. Some people are a fan of re-exports; I am not, and I think they cause more pain than benefit. They make API comprehension hard in the same way that type aliases do: you have to resolve the layers of indirection to understand where something comes from. Keeping the packages separate makes it very, very clear what lives where, and I think that's an overall benefit to keeping things tidy and understandable. Re-exporting something means making a commitment to providing it in your API even if the package that provided it stops doing so, and I try hard not to make that kind of commitment as a maintainer.
As far as backwards compatibility goes, I suggest just upgrading. The upgrade path should be straightforward regardless of whether you want to support just Unix or both Windows and Unix. I recognize that this might not be a satisfying answer. I can really sympathize with the desire to maintain backwards compatibility across minor updates, but I don't know how one could really have a good story for backwards compatibility across major updates. If you are facing a specific situation where this is causing a problem, I'd like to understand more about it.

I hope this is helpful!

Thanks for explaining! Makes sense, and I'll close this issue. Just for the record I'll put down a few thoughts, but feel free to ignore.

On the philosophy about re-exports: I understand these arguments in general, but don't think they hold particularly well for the Brick + Vty situation. The API in Brick.Main is so tightly coupled to Vty, to the extent that you need an IO Vty to use the custom main functions. When the lack of a mkVty export is largely the only thing preventing Brick from presenting a self-contained interface, it just feels odd. Brick could very reasonably treat mkVty as a part of its "own" API, as it is a near-essential ingredient for using Brick. In other words: if Vty stopped providing mkVty then Brick would become nearly unusable, so it's hardly worth designing for that case. Also, you control Vty :)

My interest in backwards compatibility is driven mainly by Stackage. One of my projects is a testing library, and I like to be able to run a CI workflow for it that builds against several of the most recent Stackage LTSes. That way, I know that those LTSes will be able to pick up a new version of my library and it will still work against the Brick/Vty that they use (which is pretty much fixed, as it would require upgrading all dependent packages to bump its major version). I know not everyone cares about Stackage, but I've been able to keep this going through at least one major version bump of Brick/Vty and I was stubbornly hoping to keep doing so.

If you don't mind me saying, it seems like that the main obstacle to re-merging the vty-* libraries is organizational, not technical. Now that the interface abstractions have been proven, I suspect the different source trees could be copied into the vty repo and then a simple Cabal conditional could choose which one to build. (Or maybe a git submodule for the Windows stuff...) It seems to me that the main (technical) difficulty would be getting sensible Haddocks on Hackage, but perhaps that could be solved with some trickery. I won't hold my breath, but I might privately hope that this could happen someday.

Anyway, thanks again and best wishes!

it seems like that the main obstacle to re-merging the vty-* libraries is organizational, not technical

This is true in the main, although -

the different source trees could be copied into the vty repo and then a simple Cabal conditional could choose which one to build

This is not.

Regardless, I don't want to get into a debate since there is not a right answer here. I needed to make a lot of trade-offs in this work and aim for the best overall outcome, both in the short term for users and in the long term for users and maintainers. It was important to me to think longer-term about avoiding maintenance headaches that could make the apparent support for Windows a disappointment down the road due to the technical and organizational headaches that would be created by putting them in the same package. While I don't plan on merging the packages, I suspect that it'd be less work to merge them later than to go the other way. And if someone really wanted to create a big package of nothing but re-exports, there is nothing standing of the way of that now.

At the same time, I recognize that some of the decisions I made might not meet your specific needs and I'm sorry to hear it. I needed to anticipate the needs people would have and attend to them as best I could given the circumstances. At the end of the day that could mean that you're still experiencing some irritation or frustration about the impact of the changes in this release. Fortunately, it sounds like the issue it created for you isn't a show-stopping problem, which is a relief for me.

At the end of the day that could mean that you're still experiencing some irritation or frustration

Not at all, I'm more than happy about the tradeoff of having Windows support! It's all good 👍

jtdaugherty / vty

Are separate vty-* packages really needed? #269