zed-industries / zed

Code at the speed of thought – Zed is a high-performance, multiplayer code editor from the creators of Atom and Tree-sitter.
https://zed.dev
Other
39.31k stars 2.04k forks source link

Linux Roadmap #7015

Open mikayla-maki opened 5 months ago

mikayla-maki commented 5 months ago

This is Zed's official roadmap and progress tracker for porting Zed to Linux.

Contributor Guide

There's a lot of surface area to cover and we would love contributions for anything from fixing a small todo(linux), to adding a large feature. That said, it'd probably be good to check in with us or our community in the #linux-port channel of our official Discord. Once you have a project underway, please open a draft PR with a description and small checklist of what you expect should be done, like so, so we can follow along, make suggestions, etc. If you have code that is stubbed out or out of scope, please mark it with a comment containing todo(linux), so we can search for it later.

To setup your machine to build and run Zed, run or reference the script in script/linux.

TODO

jgcodes2020 commented 5 months ago

Priority is to make a 'native feeling' window, if we can do that with X11 and Wayland directly, like winit does, then we should do that.

Window decorations on Linux are a mess.

  1. All X11 window managers will provide decorations for you.
  2. Some Wayland environments can provide decorations for you via xdg-decoration. This includes KDE, plus Sway and other wlroots-based compositors.
  3. Other Wayland environments (read: GNOME) want you to draw your own decorations. GNOME developers are adamantly against server-side decoration. I can't tell if Zed currently uses client-side decoration, but if not, that is a high priority. See this GNOME blog post for their guidelines.

EDIT: Just read the task list again. GTK 4 will not let you create a simple window with a fullscreen Vulkan context as you might want.

JamesMowery commented 5 months ago

+1 to please officially support both Flatpak and AppImage formats.

julianbraha commented 5 months ago

Rust SDK is cross platform, but it uses tokio, which we don't want to use.

@mikayla-maki could you expand on this? I've had a good experience with tokio and am unaware of the downsides.

ringods commented 5 months ago

You might want to look at the reasoning of @mitchellh for not providing Linux binary builds, but working with package maintainers:

https://x.com/mitchellh/status/1750942357510627605?s=20

mikayla-maki commented 5 months ago

@julianbraha we provide our own executor that integrates with the macOS runtime and would prefer to not be running multiple independent executors if we can help it.

Tokio is awesome and I love the project, it’s just not in our particular niche :)

julianbraha commented 5 months ago

we provide our own executor that integrates with the macOS runtime and would prefer to not be running multiple independent executors if we can help it.

I see, so it would be easier to port this executor to other platforms, than to replace Zed's executor with Tokio?

mikayla-maki commented 5 months ago

Exactly!

Akselmo commented 5 months ago

Flatpak is good for something you want on multiple linux distros with least amount of effort. However remember that flatpak has some security systems in place, so for example reading header files from /usr/... can be difficult for language-server processes that are launched within the flatpak. (This may be changed over time, dunno)

Having a tarball that distro packages can just make installable (or people can just drop it in ~/.local/bin/ is probably the best at first.

Look and feel wise, i hope you go for a solution that works on most platforms: KDE Plasma uses QT, GNOME uses GTK and so on. GTK is likely the most flexible across multiple desktop environments (or bare WM's), because Plasma can "modify" GTK themes to fit in (albeit it's never really perfect fit).

Decorations, and well everything-else wise: Just use xdg portals. They're the bread and butter of modern linux desktop. :)

Edit: To add, avoid targeting X11. It is on it's way out of many desktop environments. Wayland is the preferred target.

aminya commented 5 months ago

I made three pull requests to get the ball rolling.

https://github.com/zed-industries/zed/pull/6854 https://github.com/zed-industries/zed/pull/6855 https://github.com/zed-industries/zed/pull/6859

The next step is to implement a Wgpu or Vulkano backend. I started the work here, however, Gpui assumes an interface that is hard to implement for things other than MacOS Metal, and it takes quite a lot of time and expertise in GPU to do that. So it needs to be teamwork.

I have seen the Windows implementation here, but not sure if it is sustainable to duplicate the work for each operating system instead of using a cross-platform abstraction. https://github.com/PianoPrinter/gpui_win

Originally posted by @aminya in https://github.com/zed-industries/zed/issues/6868#issuecomment-1913485159

flukejones commented 5 months ago

Priority is to make a 'native feeling' window, if we can do that with X11 and Wayland directly

Please don't expend effort on X11, it is very very much on the way out the door and some distros, and many desktops are now forcing wayland only.

flukejones commented 5 months ago

You might want to look at the reasoning of @mitchellh for not providing Linux binary builds, but working with package maintainers:

https://x.com/mitchellh/status/1750942357510627605?s=20

Both can be done. It's not unusual for projects based in rust to provide static builds and many folks who come to the github project will not care so long as they can download it. On the same note, I've seen more than a few projects keep a table of known distro builds that others contribute, plus it's not unlikely to see distros pick up a popular editor.

And lastly - flatpak is certainly another option (and distribute through flathub.org). Between static builds plus flatpak it pretty much covers all bases.

kvark commented 5 months ago

@aminya For completeness, I'd also like to point out the following possible approaches (in addition to wgpu and Vulkano):

  1. Blade-graphics. Gives you portable Metal/Vulkan/GLES/WebGL with little to no overhead. Notably, a very easy porting path from Metal, e.g. resource handles can be copied around and stored freely.
  2. wgpu-hal. Gives you portable Metal/Vulkan/D3D12/GL/GLES/WebGL/WebGPU backend with little to no overhead. Plus side - being well tested as it sits below wgpu. On the minus side it's a bit verbose and requires some care.
  3. ash. Just a raw Vulkan API, so would sit alongside the existing Metal backend instead of replacing it (I don't expect you to be interested in MoltenVK). Very verbose and quite difficult to program for.

I've been working on the Blade port of GPUI in here - https://github.com/kvark/zed/tree/blade. It should be able to replace the existing Metal backend of GPUI once we see promise on Linux/Windows.

aminya commented 5 months ago

@julianbraha we provide our own executor that integrates with the macOS runtime and would prefer to not be running multiple independent executors if we can help it.

Tokio is awesome and I love the project, it’s just not in our particular niche :)

@mikayla-maki Could you elaborate on your criteria for evaluating a runtime?

I think the approach should be to write cross-platform code with maximum maintainability rather than reinventing all the wheels. That would just delay shipping features, and it would make it harder to maintain and develop later. In the open-source community, it is important to work together and reuse where possible

WeetHet commented 5 months ago

@mikayla-maki, as far as I see, live kit only provides client side sdk for rust and not the server side one. Is the plan to write the server side SDK ourselves?

mrnugget commented 5 months ago

@mikayla-maki, as far as I see, live kit only provides client side sdk for rust and not the server side one. Is the plan to write the server side SDK ourselves?

The server (see our collab crate) is already running on Linux machines, as far as I know.

theArianit commented 5 months ago

snap would be fine too, since it is better for accessing the terminal (out of the box, without any configuration) than flatpak.

alerque commented 5 months ago

Please don't waste time on X11. In the last couple years there has been a complete sea change and X11 development (from the X server side) is completely dead. All new development is being done on the Wayland side, all distros have been switching to it by default, it is the only way to support new hardware, etc. X11 stuff will hang around for a long time but for a big new project to do a bunch of coding specific for it would be a bad investment of time. Architect for Wayland and move on. If the community backports to X11 from there that's fine, but trying to architect for it out of the box will hold things back significantly.

WeetHet commented 5 months ago

Please don't waste time on X11. In the last couple years there has been a complete sea change and X11 development (from the X server side) is completely dead. All new development is being done on the Wayland side, all distros have been switching to it by default, it is the only way to support new hardware, etc. X11 stuff will hang around for a long time but for a big new project to do a bunch of coding specific for it would be a bad investment of time. Architect for Wayland and move on. If the community backports to X11 from there that's fine, but trying to architect for it out of the box will hold things back significantly.

While this is true, a lot of accessibility features are not available under Wayland, so many people use X11 still, and those should not be disregarded

alerque commented 5 months ago

While this is true, a lot of accessibility features are not available under Wayland

This was much more true a year or two ago, it has come a long ways. It is not 100% on par yet, but 100% of the display server developer momentum is behind Wayland. X11 is abandoned even by it's own former core developers. While some existing apps may have a case for not ditching support quite yet if holding onto existing support isn't costing them much, but I do not believe many new apps have much of a case to make for developing for something that is a known dead end. In Zed's case in particular working on a custom UI toolkit it would be a poor choice to design it around a legacy system that does not support new hardware, stopped getting bug fixes years ago, and distros are actively trying to deprecate (in some cases successfully).

GoldsteinE commented 5 months ago

that does not support new hardware

Which new hardware are you talking about?

flukejones commented 5 months ago

that does not support new hardware

Which new hardware are you talking about?

Graphics for one. While there will be a general level of usability for now, you should expect it to get worse over time as things advance well past what X11 is capable of. X11 has many issues, not least of which is poor multi-display support (on hybrid laptops especially, and in general).

2024 will see a lot of momentum behind wayland. To try supporting X11 at this time is poor time and effort investment.

flukejones commented 5 months ago

snap would be fine too, since it is better for accessing the terminal (out of the box, without any configuration) than flatpak.

Except that it is supported only by canonical and locked down by them. There is pretty much zero chance of use outside of ubuntu.

GoldsteinE commented 5 months ago

Graphics for one

Which, exactly? Could you name some models? Wayland has a lot of issues with all of the NVIDIA GPUs, so most GPUs on the market. You cited some unnamed “new graphics hardware” without any details.

flukejones commented 5 months ago

@GoldsteinE I see that you are heavily invested in xorg with your nix setup (and bspwm - nice tiler tbh, I used to use it) so this is going to be my only response to you as I'm wary of emotional investment in such things. I understand you are likely anxious due to talk about not supporting xorg while you still use it.

Asus send me ROG laptops to support under linux every year, The general trend I've seen is the nvida based external outputs are very poorly supported in xorg. This drastically reduces usability, meanwhile KDE-6 which is shifting to wayland only handles nvidia-outputs with different resolutions and scaling very smoothly - no issues at all so far in my testing. The step up in quality and usability over the old xorg session is incredible (oh and you get better touch/gesture support). The same can be said of the coming Cosmic desktop suite. And Budgie is moiving to wayland only, so is Gnome. XFCE is working on it. Cinnamon is.... The 2 major desktops plus a large smattering of others.

So poor display handling. Poor performance in some instances. And the general momentum. Xorg is an evolutionary deadend that is no-longer supported.

GoldsteinE commented 5 months ago

You still did not name any specific hardware, but okay, sure.

luispabon commented 5 months ago

At this point, a Wayland backend should be the target for any new software, and an x11 backend a best effort by whoever wants to contribute it. Ubuntu is switching to wayland by default, Fedora has been for a while now and Mint already have an experimental Cinnamon Wayland session. The majority of GPUs out there I'd dare say are integrated graphics, not nvidia, especially on non-gaming setups and laptops. The Nvidia situation is far better today than it was a year ago.

furkanmustafa commented 5 months ago

Oh god. X11 terminators flooded here. Please don't flood productive projects with your rants. X11 and Wayland are both >> actively used << software, with no vanishing in sight for many years at this time. There is zero point in discussing it or advocating for such things here.

For an application developer, it should be an abstracted detail. I would definitely suggest using (/re-shaping) a library that abstracts this detail, and provides you a nice start, eg. a window with a Vulkan drawing context, on any OS, on any stack, or on any GPU.

Linux Non-MacOS Support

By using slightly nice abstractions, and by avoiding Linux specifics from day 1, OS support could be expanded well beyond Linux. BSD or other OS users could benefit as well.


Input Method

On the other hand, there are a few problems that all new software experiences if the UI stack is a scratch one. Input Method support is the first in the list. I think winit now has it partially covered. Details could be seen here for reference => https://github.com/rust-windowing/winit/issues/1497 It is a complete deal-breaker, when users cannot input non-latin/CJK into an editor.

Not sure how suitable SDL would be as a dependency, for dealing with low level graphics setup, AFAIK, it also provides good level of abstraction, yet allows the app to do the rendering. SDL's platform support is no-joke. Overhead, I don't know but probably less than GTK4.


Packaging and Distribution

This is where the community could be most helpful, and it would not effect specific details of the application design much I guess? Although, packaging sometimes means sandboxing as well. (next item)


Sandboxing and portals

Supporting for xdg portals from the beginning would be very healthy. Firefox might be a good example there, it's file open/save dialogs work with portals if portal support is found on session bus, otherwise it attempts native access to the files/folders. Thus it works with both sandboxed and non-sandboxed cases. For Zed editor, I believe it would be relevant for opening/saving files and folders, and for clipboard access.

Accessing global git configuration or git/ssh/gpg credentials might be tricky in sandbox situations. Not accessing credentials is even better, probably by leveraging ssh-agent/gpg-agent and such.

If this other needs for credentials, keychain/wallet~ish subsystems are not standardized yet afaik. or org.freedesktop.portal.Secret is there for it? it might need checking. As an example, I see that Chromium making it a CLI argument to enable, https://wiki.archlinux.org/title/KDE_Wallet#KDE_Wallet_for_Chrome_and_Chromium


Other related projects, also working on cross platform support;

theArianit commented 5 months ago

Except that it is supported only by canonical and locked down by them. There is pretty much zero chance of use outside of ubuntu.

well, you can install snaps on other distros too, like for example all big distros

maxdeviant commented 5 months ago

This is a tracking issue for our plan for Linux support.

Let’s not get into a debate about X11 vs Wayland here. If you’d like to discuss the merits/downsides of them, please open a separate GitHub discussion.

Same goes for packaging approaches. If you have opinions on the matter, please open a discussion.

iddm commented 5 months ago

I suggest contacting the warp.dev to perhaps discuss the solution they use to achieve the same thing. The experience has always been great on macOS and Linux with Warp, and this is also a Rust project with (IIRC) a custom renderer.

julianbraha commented 5 months ago

I suggest contacting the warp.dev to perhaps discuss the solution they use to achieve the same thing.

Warp.dev uses Alacritty, which uses openGL for rendering.

iddm commented 5 months ago

I suggest contacting the warp.dev to perhaps discuss the solution they use to achieve the same thing.

Warp.dev uses Alacritty, which uses openGL for rendering.

How does Warp use alacritty? Anyway, I was not talking just about the rendering but about the overall experience of porting a macOS-only app to Linux. Warp already works great on Linux for many people and it will only improve from there, why not just talk to the guys over there and see where they have found issues and how they solved those instead of walking exactly the same path twice by two different groups of people?

julianbraha commented 5 months ago

I suggest contacting the warp.dev to perhaps discuss the solution they use to achieve the same thing.

Warp.dev uses Alacritty, which uses openGL for rendering.

How does Warp use alacritty?

See the Data Model section of the How Warp Works blog: https://www.warp.dev/blog/how-warp-works

Also, on a second reading, it seems that Alacritty was only forked for this aspect, and it does its own rendering, so nevermind about openGL.

Eonfge commented 5 months ago

snap would be fine too, since it is better for accessing the terminal (out of the box, without any configuration) than flatpak.

This is a valid comment and I don't think you deserve the flak that you're receiving. That said, I don't think that your concerns applies here: Zed is designed as a thin-client, with all the actual compiling and such is done in a centralised server.

In that regard; Using Flatpak (or snap) is even more important... Since you don't want to support clients that are part of an LTS release for 5+ years

Collaborate with distro packagers?

Word of warning. Different distributions have their own rules about support, compiler toolchains, and licencing. If you want to submit your package to Debian, you must promise 5+ years in backwards compatibility because that's the 'stable' release cycle they adhere too. Fedora might reject your package if it can't be built with GCC, and your non-free back-end might also disqualify you for both. Flathub in the other hand does not care; you decide what's best for your application and users.

As a flathub volunteer, I trust distribution maintainers in providing me with a reliablecore system... But I don't go to them if I need the latest or bleeding edge.

In the end, there are a few hard decisions to be made.

jgcodes2020 commented 5 months ago

snap would be fine too, since it is better for accessing the terminal (out of the box, without any configuration) than flatpak.

This is a valid comment and I don't think you deserve the flak that you're receiving. That said, I don't think that your concerns applies here: Zed is designed as a thin-client, with all the actual compiling and such is done in a centralised server.

In that regard; Using Flatpak (or snap) is even more important... Since you don't want to support clients that are part of an LTS release for 5+ years

Collaborate with distro packagers?

Word of warning. Different distributions have their own rules about support, compiler toolchains, and licencing. If you want to submit your package to Debian, you must promise 5+ years in backwards compatibility because that's the 'stable' release cycle they adhere too. Fedora might reject your package if it can't be built with GCC, and your non-free back-end might also disqualify you for both. Flathub in the other hand does not care; you decide what's best for your application and users.

As a flathub volunteer, I trust distribution maintainers in providing me with a reliablecore system... But I don't go to them if I need the latest or bleeding edge.

In the end, there are a few hard decisions to be made.

You might be able to partner with some of the people at Arch Linux since Arch runs on the bleeding edge. Barring that, you can still make and provide an AUR package.

felipeacsi commented 5 months ago

You might be able to partner with some of the people at Arch Linux since Arch runs on the bleeding edge. Barring that, you can still make and provide an AUR package.

There is already a package submitted to AUR (obviously currently not working): https://aur.archlinux.org/packages/zed-editor

theArianit commented 5 months ago

is is a valid comment and I don't think you deserve the flak that you're receiving.

this is a discussion about linux. Instead of being happy that zed will be available for linux too, they complain about snaps, x11, wayland and such things. I personally replaced a lot of distro packages with snaps and flats. and to be honest, in some cases snaps feel better than flats. with snaps I don't need to do any configuration as I need with flatpaks, like the case of the terminal in vsCode. and it is as you said, independent from lts versions and distros, you have just one flat or snap which is distro agnostic and lts version agnostic. and in the end of the day, it's just an app, I don't care how it is distributed, as long as it is being maintained and updated. linux elitist can't get over this fact

sk337 commented 5 months ago

any plans to at least distribute a rpm package and regarding @Eonfge's comment you could get it could under the non-free / 3rd party section

jansol commented 5 months ago

I don't think distribution is going to be a problem now that the editor itself is GPLv3 (and the server is AGPL so that should be fine for pretty much all distros too). Actually building & long term maintaining packages seems like it'd be the more pressing issue.

antaz commented 5 months ago

Any updates on the progress or who is currently working on what?

aminya commented 5 months ago

As a note for those who are unaware. We do not even have the GUI library that Zed is built on for Linux. That's a huge work and will take a lot of expertise and effort to implement.

The arguments over the distribution and the package formats are so out of scope right now! So please keep this focused.

antaz commented 5 months ago

Priority is to make a 'native feeling' window, if we can do that with X11 and Wayland directly, like winit does, then we should do that.

Is there any reason as to why not rely on winit directly to provide the window abstraction?

mikayla-maki commented 5 months ago

Any updates on the progress or who is currently working on what?

You can see the current state of what people are working on in the issue text, note that as of right now (2/1/24) me and Julia are tagged on two bullet points, those are the things we're actively working on. As for progress updates, we'll be making PRs in the main Zed repo and marking down details in this tracking issue as we go :)

I don't think we have any plans for regular 'the state of Zed on Linux' style posts, we're just gonna keep working at it until it's done.

mikayla-maki commented 5 months ago

Is there any reason as to why not rely on winit directly to provide the window abstraction?

At Zed, we generally prioritize long term agility by taking ownership of as many parts of the stack as we can. Like with Tokio, the winit folks are fantastic and do great work, and I personally have learned a lot from both of them. But using their libraries directly just doesn't fit with how we've decided to approach these hard problems. This does mean we have to do more work, and re-discover bugs many people have already found, but it also means we can fix things as soon as they come up and explore paths not taken. Changing this approach is not up for discussion.

mikayla-maki commented 5 months ago

and your non-free back-end might also disqualify you for both

Is AGPL not considered free @Eonfge? Our backend is in this repo, under crates/collab, and is licensed under AGPL. I think the only non-public code we have is for our website, which is a glorified brochure that doesn't do much more than provide access to the github login flow and host our blog. That said, I doubt we'll ever be interested in providing 5 year LTS 😂, so the question is probably moot.

AlexDaniel commented 5 months ago

Is AGPL not considered free @Eonfge?

@mikayla-maki, it's as free as it gets, but perhaps the state of the backend wrt the license should be mentioned more prominently. It's still a bit unbelievable (too good to be true) that it is free/open-source, because most people probably expected that it's what you'd be monetizing. The website says “It's also open source.” but feel free to make louder claims – “fully open-source”, “both the editor and the backend implementation are open-source”, etc.

mikayla-maki commented 5 months ago

It's still a bit unbelievable (too good to be true) that it is free/open-source, because most people probably expected that it's what you'd be monetizing.

Fundamentally, we want to change the way people write software. As we say in our repository subtitle Zed is "the fast and social code editor." (Emphasis added). We think the best way to drive this kind of change, is to let everyone in! Show everybody that our vision can work! We're not aiming to standardize or stabilize things for a long time (if ever) but this is the next best thing.

Akselmo commented 5 months ago

I actually did not know this is all open, I am delighted to hear that. Be proud of your AGPL license, use it as a "selling" point! ;)

Also UI toolkit wise, since you seem to be using Rust, GTK might be easier for that since afaik it works with Rust.

I think there are other Rust UI solutions that work on Linux though, but I do not know enough to advice on that.

Very interested in this project, always happy to see more text editors so that we all just don't use VSCode ;)

theArianit commented 4 months ago

If I am allowed to make this kind of comment, I would suggest to take a look at lapce. Another open source editor made with rust. They use wgpu for renderin, and in their github repo they mention all the other things they use. Maybe getting some inspiration from them for linux would not hurt anyone.

eyelash commented 4 months ago

Switch to Vulkan if it doesn't work out

I hope you are also considering Linux running on older hardware with no or bad Vulkan support. Many hardware-accelerated UIs on Linux such as Firefox or GTK 4 still support OpenGL 3.x.