storacha / project-tracking

🐾 Used as central/default repo for project management, backlog, etc.
0 stars 0 forks source link

implement replacement for dagcargo as it is deprecating #7

Closed alanshaw closed 6 months ago

alanshaw commented 7 months ago

See also

Reviewer: @vasco-santos

heyjay44 commented 7 months ago

Deals that were made pre Spade were dealt with and we do not need to worry about renewals.

gobengo commented 7 months ago

convert NFT storage infra to use w3up client to act as a normal customer of w3up and go through normal flow to add the data into w3up. Then all existing filecoin pipeline stuff just happens by default

This PR has an implementation: https://github.com/nftstorage/nft.storage/pull/2522

gobengo commented 7 months ago

w3up-client integration is deployed behind a feature switch in staging.nft.storage.

Good news is that it can be used to forward an 8MB upload to w3up without error.

Bad news is that it doesn't work in all circumstances, and we're still debugging why In testing we see that uploads that use it can fail in at least two ways:

Update: This helped with the OOM errors and maybe more https://github.com/nftstorage/nft.storage/pull/2551

alanshaw commented 6 months ago

I am pg_dumping dagcargo tables: aggregate_entries, aggregates, deals, dags, metrics, metrics_log.

I will import into a new postgres in Heroku and check it can be attached as a foreign data wrapper to classic.nft.storage/old.web3.storage staging DBs. If successful I'll attach it to production DBs.

alanshaw commented 6 months ago

New DB in Heroku created: dagcargo

Contains exported subset of dagcargo: aggregate_entries, aggregates, deals, dags, metrics, metrics_log.

Created 2 read only users nftstorage and web3storage - i.e. different user for each product but staging and production use the same user.

hannahhoward commented 6 months ago

@travis just testing api -- deploy to staging today, then prod soon, then we're done! 4/15/24