Terminal

[andrasbacsai]

Currently, /command-center executes one command at a time on the server.

We need a fully functioning terminal instead of that.

It should be reusable on other parts of the UI as well, for example by opening a terminal to a specific docker container (like WordPress) to edit files.

[andrasbacsai]

/bounty 50

[algora-pbc[bot]]

💎 $50 bounty • Coolify

💎 $10 bounty • Muhammed Mustafa Akşam

💎 $30 bounty • CloudGakkai

💎 $10 bounty • Martynas Sklizmantas

Steps to solve:

1. Start working: Comment /attempt #2298 with your implementation plan
2. Submit work: Create a pull request including /claim #2298 in the PR body to claim the bounty
3. Receive payment: 100% of the bounty is received 2-5 days post-reward. Make sure you are eligible for payouts

Thank you for contributing to coollabsio/coolify!

Add a bounty • Share on socials

Attempt	Started (GMT+0)	Solution
🟢 @LEstradioto	Jun 13, 2024, 5:53:54 AM	#2586

[paoloricciuti]

I don't think I have the energy and time to fully work on this but I guess sharing ideas is a form of contributing too:

You can use XTerm.js which is a fully featured implementation of a terminal in JavaScript.

Here's the docs for it

https://xtermjs.org/docs/

I have a bit of experience in it (not too much) so if anybody need help feel free to contact me here or anywhere else (obviously no part of the bounty required lol, just here to help)

[raikasdev]

I'd be interested in working on this, but I'll have to dig more into how Coolify technically works before I'll attempt this if someone that already knows the bells and whistles wants to try. I write PHP for work but haven't used Laravel before.

I have experience of similar implementations with Xterm as I have created the ohjelmoi.fi development environment's interactive terminal with it.

[muhammedaksam]

Very nice idea.

[Thanzex]

I'd like to contribute too, I'm switching to Coolify and this feature would be very helpful indeed.

I did take a peek at how other tools are implementing this feature, Portainer takes a fairly simple approach, which is to simply proxy all requests to the Docker api. There's a little bit of logic in the frontend to send and receive via a websocket and setup the session, on the backend it checks if the session has been terminated via command or hotkey.

The frontend calls a .../docker/containers/{hash}/exec endpoint which gets proxied by the backend to the Docker API, converted to a WS session and then the rest is handled by Xterm.

I'm not an expert of Laravel and in this time I haven't been able to find how to implement this, but if the reverse-proxy aspect cannot be easily implemented I would guess a simple forward of the websocket messages could be the solution, even reusing the existing infrastructure that handles the logs

[LEstradioto]

/attempt #2298

Followed your ideas, xterm.js and @Thanzex approach.

I've attempted to implement WebSockets in PHP, but it turned out to be more challenging than expected. I then switched to using Node.js with XTerm and pty, which has been working well. I like the architecture, but I think @andrasbacsai should review it to assess any potential security risks, as I'm not fully aware of them.

https://github.com/coollabsio/coolify/assets/22579355/f717a407-6c3d-4167-88e6-fabdd1d2cbf3

Currently, the terminal only appends to a server, but I don't see any issues with extending this functionality to containers as well.

Later on, I'll post the wip branch here

Algora profile	Completed bounties	Tech	Active attempts	Options
@LEstradioto	2 bounties from 2 projects			Cancel attempt

[Thijmen]

Wow @LEstradioto, that looks very promising! I know my fair share of Laravel and security, feel free to whip up a draft PR and I will take a look and address possible concerns.

How do you feel about that? Happy to help!

[LEstradioto]

I would really appreciate it! Here's the PR #2586.

I'm still getting used to Laravel, but this draft works. I think a review would really help !

[jerch]

@LEstradioto On a sidenote - you dont have to use websockets with xterm.js, you can also use good old request-response cycling with ajax/fetch long-polling. While it is not be as performant as with websockets (con), it would also inherit security mechs of your normal site setup (pro).

[LEstradioto]

Oh yeah. Nice to know. That PR have some good additional observations too. But terminal worked nicely

[kemboi22]

Is this still open i would like to contribute

[LEstradioto]

Its still open. But, Ive open a PR #2586 and awaiting @andrasbacsai position on this

I think, the challenge right now is to re-use services that exists without adding a new service. Ive added a nodejs websocket to work with the xterm.js terminal.

Do you have any suggestions on this? Or, If you want to take a shot, no problem for me at all

[andrasbacsai]

Its still open. But, Ive open a PR #2586 and awaiting @andrasbacsai position on this

I think, the challenge right now is to re-use services that exists without adding a new service. Ive added a nodejs websocket to work with the xterm.js terminal.

Do you have any suggestions on this? Or, If you want to take a shot, no problem for me at all

I will test it with the current implementation (node) and if it works and does not take up too much resource, then I will merge it.

We can migrate later.

[AliShaatani]

guys thank you brilliant work, please make this terminal comes to live, cuz there is a lot of cases when we want to use vim/nano, which is impossible with the current versions

[LEstradioto]

The PR is concluded, waiting @andrasbacsai on a new review. Just a bump here.

Terminal process runs within Soketi service, so no more extra container About security concerns Ive added authentication on Websocket using verifyClient option to authenticate user on Laravel

So its pretty much done!

[emjayess]

I have only just scanned the comments here, and lightly glanced over additional remarks in the PR, and I wonder has anyone considered or already attempted to do this using Laravel Reverb?.. seems if that could fit the bill for websockets, it would alleviate the need for an extra container and soketi dependency (if I'm following along well enough).

[LEstradioto]

Hi @emjayess. I dont know Reverb well enough, it just seems the same as Pusher.

Problem is that we need to use node-pty as pseudo-terminal. If we use PHP we must create that lib from the ground, to be compatible/synced with xterm.js

Another one, is creating a long running process inside a Laravel Event. Ive tried and failed, perhaps its possible, but even with this we need a pseudo-terminal then, meaning, need somethin similar to node-pty

[jerch]

Hmm I just checked PHP corelibs and indeed - seems PHP has no support for spawning PTYs. Thats a bummer, it means that any PHP-related solution needs to be a native/C module. Bad.

Here is such a lib for PHP, idk whether it is any good or bad in quality/maintenance.

Idk your tech stack, is writing/maintaining your own C module within your goals or far overstretched? (Thats basically what we also had to do with node-pty for xterm.js in nodejs env...)

Edit: ssh2_exec kinda contains PTY spawning within, so you might get somewhere with that. Idk if it exposes enough PTY functionality (like for resizing).

Edit2: If anyone wants to dive into C/C++ madness, here is my impl of PTY-related C-lib primitives for nodejs (imho most aspects could be easily translated into PHP realms):

• Typescript part: https://github.com/jerch/node-newpty/blob/master/src/pty.ts
• C++ part: https://github.com/jerch/node-newpty/blob/master/src/pty.cpp

[LEstradioto]

Hi @jerch ! Great work! 👍 I saw TeaOpenPTY during my research, but C modules were out of my comfort zone. Your solution is much more elegant.

Quick question: Is there a way to reconnect to an existing process/PTY across requests? I tried using Symfony/Laravel Process to start a PTY, it worked, but couldn't figure out how to reattach on subsequent requests. Any thoughts on this?

[jerch]

Quick question: Is there a way to reconnect to an existing process/PTY across requests? I tried using Symfony/Laravel Process to start a PTY, it worked, but couldn't figure out how to reattach on subsequent requests. Any thoughts on this?

Well thats always possible even across processes with fifos. Not very elegant as it needs FS invocation, but would get the job done. A fifo would also correctly block process execution, if there is no data consumer (e.g. request cycling is much slower than the application creating data on that PTY - basically the request/response reader would slow down the PTY slave app here). Note that this still needs a way to place a long running process somehow, because the PTY spawned has to live in a process context somewhere (well - the master part of it). In nodejs this is normally not an issue due its long running server process nature. Again idk enough about your tech stack, but short-living request/resonse processes as typical for many PHP envs do not fit that requirement.

Edit: Btw symfony's process seems to be a good starting point, as it seem to allow to set the IO stream endpoints. Thats where the PTY fifos could be attached to. Then from your request process simply attach to the other fifo ends for read/write. But note - it is IMPORTANT, that the fifos point to PTY file descriptors, otherwise interactive apps like vim never gonna work properly. Symfony's process abstraction also has this setTty(true) thingy - but idk how that works or to which TTY/PTY exactly that binds. (Most likely to the controlling terminal of the parent process, which might be the server startup process - at least thats what I read out of the docs. If thats really the case - its def. not what you would want...)

[LEstradioto]

Yep yep, two big PHP Laravel challenges. Well, @andrasbacsai should evaluate this 👀

[blinkinglight]

https://www.supertechcrew.com/wetty-browser-ssh-terminal/ or maybe https://github.com/yudai/gotty ? or https://gist.github.com/rochacon/52420a3ab36b131e1437

[LEstradioto]

I've seen those libraries as well. They were definitely inspirational for that PR, which essentially has the same logic as well, but adapted to Coolify's ssh. However, they still requires websocket and node-pty

[LEstradioto]

Here's what I'm thinking for next steps (and a summary for future suggestions):

1. I'll finish the PR to adapt the Dockerfile. This should serve as a good starting point for version 1.

For a future version of Terminal, we could consider:

2. Rewriting node-pty in PHP (would require expertise in both C and PHP)
3. Replacing node websocket with PHP equivalent

This would be a challenge but real fun!

(Apologies to issue followers - this seems to evolved into more of a discussion than a typical issue)

[blinkinglight]

https://github.com/coollabsio/coolify/discussions/3362

[jerch]

For a future version of Terminal, we could consider:

2. Rewriting node-pty in PHP (would require expertise in both C and PHP)
3. Replacing node websocket with PHP equivalent

Well if you are really up for your own low level impl, I can help you with the C part. But I'd need some guidance with your process/thread model in coolify, esp. where to put long living PTYs (must survive request/response cycles).

[algora-pbc[bot]]

💡 @LEstradioto submitted a pull request that claims the bounty. You can visit your bounty board to reward.

[andrasbacsai]

Any further iteration / feature will be handled in a different issue. Thank you @LEstradioto again!

[algora-pbc[bot]]

@LEstradioto: You've been awarded a $50 bounty by coollabsio! 👉 Complete your Algora onboarding to collect the bounty.

[andrasbacsai]

Please also do the Algora reward @cloudgakkai, @muhammedaksam, @saint

[DeVoresyah]

Please also do the Algora reward @CloudGakkai, @muhammedaksam, @saint

will be process by 25th this month from our side.

[LEstradioto]

Thank you!

[DeVoresyah]

hi @LEstradioto @andrasbacsai , idk why on my Algora account there is no "Reward" button that usually used to send the reward

[muhammedaksam]

Please also do the Algora reward @CloudGakkai, @muhammedaksam, @saint

I am still trying to pay but it won't let me process.

[DeVoresyah]

Please also do the Algora reward @CloudGakkai, @muhammedaksam, @saint

I am still trying to pay but it won't let me process.

I think Algora has an issue, I also cannot process because the "Reward" button didn't appear

[LEstradioto]

No problem, guys! I've sent an email to them. Let's wait a bit.

[DeVoresyah]

is there any updates on this? the Reward button still won't appear on my side