Open carsonfarmer opened 1 year ago
If you are currently storing files and/or data on Textile’s hosted Hub infrastructure, it is time to think about migrating your data elsewhere, or hosting your own Hub infrastructure. But don’t worry, we are here for you every step of the way, and there are multiple options for migrating data off the Hub:
It’s just SQL! Get started with the docs here. We also have a number of command line tools to help you migrate data via SQL statements, with more tooling improvements each week. For questions, create a support ticket in our Discord. Tableland is our core focus right now, so we’re happy to onboard you to this exciting web3 option.
Filecoin is ready for prime-time and doesn't require tools like the Hub make it useful in your application. To leverage all the power that Filecoin has to offer, you might want to consider NFT.Storage and Web3.Storage, both projects that are crushing Filecoin data onboarding. We also highly recommend Estuary, which is another great solution for getting data onto Filecoin and pinning it on IPFS… the easy way! Additionally, IPFS ecosystem players like Filebase, Fleek, and Pinata run excellent and widely used IPFS pinning infrastructure. And these groups all have payment plans that are comparable to what the Hub was offering.
For folks that want to continue to leverage a “Hub” like experience without the hosted infrastructure provided by Textile, you may choose to self-host a Textile Hub. As with all things Textile, the Hub (and Buckets, ThreadDB, Mailbox, etc) is open source and ready for you to clone, fork, and/or deploy on your own infrastructure.
If you are only using one or two of the hosted services provided by The Hub (say, Buckets or ThreadDB), you might want to consider deploying and hosting only those services you require. For instance, it is relatively easy to host your own Buckets API, or to host a ThreadDB service and relayer. Instructions for deploying these open source libraries are available right within their READMEs:
If you require a more full-featured Hub setup, then you’ll probably want to deploy your own Hub to Google Kubernetes Engine. Before going down this road, ensure that you, or someone from your team, is familiar with Kubernetes deployments (or is able to leverage Docker and other infra tooling more directly).
There is a getting started guide in the README of the Hub GitHub repo, but this GKE setup for the Hub is probably the most straightforward way to deploy your own Hub infrastructure. In theory, you can get the Hub deployed without much more than starting in that folder, and executing kubectl -n <your-k8-namespace> apply -f .
. The envs in those YAML
files are well configured, so it should “just work”. In practice, however, your setup may be more nuanced than that, so there are some other things to consider.
The above YAMLs were testing using both Azure managed K8 and GKE managed K8. For those interested in using EC2, if you guide your installation setup by reading the YAML files in that setup, that should provide you with the recommended setup. Everything you need to know should be in those files. However, the Textile team highly recommends GKE if that is at all possible. The cluster management and auto-scaling of cluster resources are pure gold!
There are a few items to consider when deploying. For instance, to disable the Billing API, you can simply not run the Billing Daemon. The default value for the Billing API in hubd
is "not set”. Related the CustomerIO service can also be “ignored”. For example if you wanted to put The Hub behind your own API (which is likely what most folks will want), you'll just have one "account". You can confirm that account manually by setting this flag in code, and then visiting http://localhost:8006/confirm/[secret]
.
As always, for questions or support on any of these options, please create a support ticket in our Discord, and our team will do out best to get you going. Unfortunately, our team is not offering dedicated support for running your own Hub instance. This is an open source project, and there are many community members that are able to help, but we are not offering dedicated support services at this time.
What’s the Textile Hub?
Built on Libp2p, IPFS, and Filecoin, the Textile Hub provides a centralized API to three peer-to-peer stacks:
The Hub was built to on-board developers to the Textile stack who didn’t want to run their own ThreadDB peer, IPFS peer, or Powergate service. It also serves as a large ThreadDB relayer and encrypted backup service.
Since it’s release in 2019, the Hub has become… well, a true “hub” of activity:
Why are we shutting it down?
We’re sad to let the Hub go, but it was never meant to be a long-term solution for folks to build on. It was meant to be an on-ramp for developers, and it no longer serves that purpose in a sustainable way. As it turns out, centralized on-ramps tend to be sticky!
However, we remain completely dedicated to building a database layer for the next web. That’s why last year we kicked off Tableland, a decentralized database and network built on SQLite and the EVM. The knowledge and experience we gained over the years with the Textile stack has been instrumental in the design of Tableland. At its heart is a ruthless focus on sustainability and developer experience. Unlike the Textile stack, Tableland will be an incentivized network, removing the need for costly federated relays and backup nodes in the long term like the Hub.
What’s happening with my Hub data?
Textile has been a member of the IPFS/Filecoin community since the early days, and we’re here to stay. That is why we are committed to working with folks to ensure a smooth transition off the Hub. Our intent is to shut down the Hub for good on January 9, 2023. At this time, all ThreadDB and Bucket data will no longer be available, and will subsequently be removed.
If you need more time to migrate, no problem! Just let us know by opening a support ticket on our Discord.
Here’s how it is going to play out:
We are here for you every step of the way, and there are multiple options for migrating data off the Hub:
What’s happening with my FIL balance?
A small number of you transferred FIL into the deal making address that was associated with your Hub account. If that’s you, and you’d like for us to transfer that balance to a new wallet, let us know by opening a support ticket on our Discord, by January 9th, 2023. After January 9th we will require you to verify ownership of the account email address. Any remaining funds will be donated to future Filecoin hackathon teams.
FAQ