Feature: Switch from Sui binaries to Docker images

[kkomelin]

Problem/Motivation

In order to prevent the issues like this on some platforms, I suggest switching from downloading Sui binaries from Sui releases or compiling them from sources to using prebuilt Sui Docker images like this.

What would we gain from the switch:

• Eliminate cross-platform issues.
• Simplify the Suibase code, responsible for installing Sui binaries and compiling from sources.
• Reduce the number of Suibase dependencies (at least those which are used for compiling source codes).

What's not yet clear:

• Do https://github.com/shinamicorp/sui-docker docker images have everything we need (console commands and interfaces)?
• How can we prepare for possible Docker-related issues, like chroot, port conflicts, etc.? Documentation (FAQ/Troubleshooting guide),

[mario4tier]

Follow up about sui-docker (in context of Suibase):

• Seems intended more for ops than devs. As an example, neither sui-test-validator or sui-faucet are included. Suibase depends on sui-faucet for localnet.

• Build seems manually done after requiring a shinami-side code change. Not better/worst than Mysten (surprisingly, Mysten build the assets sometimes "days" after the tagging).

Ideally, a good binaries source would be CI and triggered by upstream Mysten Labs tagging.

Maybe Suibase should release its own binaries? Will give us full control on what to include (e.g. suibase-daemon or any other tools in the future)... and furthermore we can automate a "Quality Control" step to make sure all required binaries are included.

One disadvantage of Shinami or Suibase building their own: it might be perceived as less safe, not "official".

===

Conclusion: As of right now, using sui-docker is a no-go.

It is still an open question to use docker or not (have not fully evaluated yet).

[kkomelin]

@mario4tier Thank you for your research and the detailed elaboration on your opinion. I tend to agree with you. So we could either close this issue for now or postpone it for a later point when we have more suitable Docker images or ready to create own binaries/images.

[mario4tier]

@kkomelin

I did start to investigate devex through Docker, VSCode DevContainer versus Suibase bash/rust/linux requirements.

Before reaching a conclusion, I have a question about your Docker+suiexplorer use case.

Do you think in the context of blockchain devs it would be better to depend on Node instead of Docker?

Of course Docker/Node serve different purpose, but wondering more about the end-user experience. It seems that imposing to install one or the other would have work in your scenario.

(Do not feel obligated to answer now, we can learn from how it goes in the coming months with Docker).

[kkomelin]

Hey @mario4tier ,

I don't yet have enough feedback from my Local Sui Explorer users. The first ones which had no previous experience with Docker struggled to run my project. I'm not sure about all blockchain developers but some of them have to setup a local blockchain node at least once and Docker is a good solution for that, so it's likely some of them have some experience with Docker. Also, Node is more often used for frontend than for backend, so if it's a blockchain developer, they usually more comfortable with backend stuff like Rust, Go, etc.

For me personally, as for a full-stack developer, Docker is a must-have skill or even more - Kubernetes.

In case of my project, I actually require both Node and Docker to be present n the system. Node is used for a CLI script, which communicates with Docker.

The main advantage of a Docker solution for the end-user is that they don't need to care about Node version and other Node dependencies to be able run the app itself. It's all isolated inside the container.

I'm not sure if I answered your question fully, but when I have more feedback from users, I will be able to tell more.

[kkomelin]

it'd be good to learn this folder https://github.com/MystenLabs/sui/tree/main/docker

[kkomelin]

Hey @mario4tier ,

I've just received feedback about my starter from a very experienced Ethereum developer. The idea is that he found overall Sui installation experience terrible, named it "all that Rust-etc. Hell". It's in comparison with what Hardhat provides to ETH developers. You just install it as a dependency to your project and optionally as a VSCode plugin, and you can code locally, compile and format Solidity code, run tests, deploy, etc.

While building my starter I actually wanted to provide comparable experience to Hardhat because I developed a little bit for Ethereum myself. That's why I actually ended up integrating with Suibase because it was the best tool available. Suibase is great but is not yet close to Hardhat and at this stage it even serves a bit different purpose but why not extend the purpose. I don't know if Hardhat uses Docker under the hood but I'm sure that Suibase can become a hardhat for Sui developers.

Sharing this feedback here for us to learn how Hardhat works under the hood and what features/approaches/ideas we could potentially borrow.

[mario4tier]

Thanks for the info.

Hardhat dependency is only Node.

Glad you mention it. I want to reassure you that Suibase intent is to slowly be Hardhat-like. As you know, making things easy for the user is... hard.

About Rust Dependency

Something I got wrong initially with Suibase was to focus on Rust/Python backend devs. I was also going with the assumption that the Rust compiler must be installed because of Mysten Labs pre-requesites.

... but it seems that most users approach Sui from front-end (the Rust SDK is challenging to use). I did complete 2 Move training, both using only Typescript SDK (I will generalize that most users will be OK to have Node as a dependency).

... and compiling Sui binaries is painful (>30 minutes for some users).

Therefore supporting pre-compiled binaries became a priority to Suibase ~Q2 2023 (while still falling back to build when the binaries are not available).

As of right now, the building of the backend suibase-daemon is the only reason left for Suibase to depends on Rust.

Action Items

(1) Add Github CI to produce suibase-daemon binaries. Eliminate Rust related dependencies until someone actually use the Rust SDK. Not very hard to do (just not allocating my time right now to do it).

(2) Make sure to cover helping the more common FE devs. That means developing a Suibase NPM in addition to the current Rust/Python helpers. This is plan for later this year, potentially as part of a grant.

(3) Use the VSCode extension as the easy on-boarding (particularly, actively detect and recommend fix on installation issues). This is being develop right now as part of the on-going grant.

[kkomelin]

Thank you @mario4tier for describing your vision and future plans. It makes a lot of sense. Let's see if I can help with something here.

[mario4tier]

Update: Action Items (1) is now closed by #101