search

TrustedDataGraph / HackFS2023DataVerification

DataVerification and Reputation System
MIT License
1 stars 1 forks source link

Basic FVM Store, Retrieve #4

Open kozayupapa opened 1 year ago

kozayupapa commented 1 year ago

https://www.youtube.com/watch?v=BecGGm-YnPc

HugeData : DirectDeal

https://github.com/filecoin-project/filecoin-fvm-localnet https://github.com/filecoin-project/fevm-hardhat-kit

  1. docker compose up to launch filecoin on local
  2. deploy deal client contract to localnet with hardhat
  3. check deals localhost:8080/storage-deals/
  4. DL data with `lassie fetch --providers XX
  5. unpack data with `ipfs-car --unpack {CAR_NAME}.car
  6. original file extract to {CAR_NAME} dir

small Data. aggregator

Javascript which monitor event and call Edge.

  1. node fdt-deal-monitor-ethers.js
  2. yarn hardhat store-data --contract {$LOCAL_CONTRACT_ADR} --url {LIGHTOUHSE_CAR_URL} --network localnet
  3. wait 15 seconds on fdt-monitor, then showing sending payload logs(including Download)
  4. you can download with curl {$DOWNLOAD_URL} > out.car
  5. ipfs-car --unpack out.car
  6. the file will be extracted to {CID} dir
  7. we can check aggregate status with `curl {$AGGREGATE_URL} | jq .
kozayupapa commented 1 year ago

test with my local filecoin

generate car on local and Store to web3

(base) go-generate-car % ./generate-car --single -i ./tmp/hellodao.txt -o tmp -p ./tmp/ 
{"Ipld":{"Name":"","Hash":"bafybeia3dzqclgnjxstztyy26js5qrno7erfptwkthtetxhzv3btzsc424","Size":0,"Link":[{"Name":"hellodao.txt","Hash":"bafkreigtr2tk3dvbyjmfup5hdmklj6gsdwa3fkgqkk6b3n73hllvquvyqa","Size":33,"Link":null}]},"DataCid":"bafybeia3dzqclgnjxstztyy26js5qrno7erfptwkthtetxhzv3btzsc424","PieceCid":"baga6ea4seaqer23asz6azhqynffuzb5aoaaxglf7sexu6un533sylqriqtv2yda","PieceSize":256,"CidMap":{"":{"IsDir":true,"Cid":"bafybeia3dzqclgnjxstztyy26js5qrno7erfptwkthtetxhzv3btzsc424"},"hellodao.txt":{"IsDir":false,"Cid":"bafkreigtr2tk3dvbyjmfup5hdmklj6gsdwa3fkgqkk6b3n73hllvquvyqa"}}}

generate car on lighthouse

  1. post to light house and get car json 
    {"piece_CID":"baga6ea4seaqer23asz6azhqynffuzb5aoaaxglf7sexu6un533sylqriqtv2yda","piece_Size":256,"car_Link":"https://data-depot.lighthouse.storage/api/download/download_car?fileId=11ba5088-c8a7-4f1b-8960-c0415985455c.car","car_Size":226}

fund to deal client

we need to fund clinet I modified task script

task(
    "deal-fund",
    "Gets a deal's status from the piece cid"
  )
    .addParam("contract", "The address of the deal client solidity")
    .addParam("value", "fund tFL")
    .setAction(async (taskArgs) => {
        const contractAddr = taskArgs.contract
        const value = taskArgs.value
        const networkId = network.name
        console.log("Getting deal status on network", networkId)

        //create a new wallet instance
        const wallet = new ethers.Wallet(network.config.accounts[0], ethers.provider)

        //create a DealClient contract factory
        const DealClient = await ethers.getContractFactory("DealClient", wallet)
        //create a DealClient contract instance 
        //this is what you will call to interact with the deployed contract
        const dealClient = await DealClient.attach(contractAddr)

        //send a transaction to call makeDealProposal() method
        //transaction = await dealClient.getDealProposal(proposalID)
        let result = await dealClient.addBalance(value)
        console.log("The deal status is:", result)
    })
(base) fevm-hardhat-kit % yarn hardhat deal-fund --contract $DCLIENT --value 2910400000

make deal proposal

result

check DL from local lots node

check locl lotus listen port p2p

(base) filecoin-fvm-localnet % docker compose exec lotus lotus net listen                                               
/ip4/172.18.0.4/tcp/9090/p2p/12D3KooWRYivrM1DAbk38JfHyUt21YVSEJhBMjVeX3oLAj7adQcB
/ip4/127.0.0.1/tcp/9090/p2p/12D3KooWRYivrM1DAbk38JfHyUt21YVSEJhBMjVeX3oLAj7adQcB

fetch with lassie

https://github.com/filecoin-project/lassie clone and run go build ./cmd/lassie

Deal Status

Status  Sealer: ProvingThe deal can be in one of the following states:
Transfer Queued
The storage deal proposal has been accepted, and Boost is about to start the data transfer.

Awaiting Offline Data Import
The client has made an offline deal proposal, and Boost is waiting for the Storage Provider operator to import the deal data.

Transferring
The data for the deal is transferring.

Transfer Complete
The data transfer is complete and Boost is verifying the data matches commp.

Ready to Publish
The deal is in the batch publish queue, ready to be published.

Awaiting Publish Confirmation
Boost sent a publish deal message for the deal and is waiting for on-chain confirmation.

Adding to Sector
Boost is handing the deal off to the lotus-miner sealing subsystem to be added to a sector.

Announcing
Boost is announcing the deal to the network so that clients know where to retrieve it.

Sealing
The deal has been added to a sector and is now sealing.

Complete
The deal has reached its duration and expired.

Cancelled
The deal was cancelled.

extract car file

kozayupapa commented 1 year ago

Aggregate

preparation Estuary Edge/Delta

https://docs.filecoindata.tools/about/delta-tech-stack/delta-user-guides/getting-api-token-key

curl --location --request GET 'https://auth.estuary.tech/register-new-token'

deal-monitor

https://github.com/hammertoe/fdt-deal-monitor

git clone git@github.com:hammertoe/fdt-deal-monitor.git
cd fdt-deal-monitor
yarn
yarn hardhat deploy
[
  'https://data-depot.lighthouse.storage/api/download/download_car?fileId=11ba5088-c8a7-4f1b-8960-c0415985455c.car'
]
Sending payload to:  https://hackfs-coeus.estuary.tech/edge/api/v1/content/fetch-url
API response: {
  status: 'success',
  message: 'File uploaded and pinned successfully. Please take note of the ids.',
  contents: [
    {
      ID: 17907,
      name: 'bafybeid6dici3ld6bhrw5abw6iely4afwxasmqw5xsynprcxakp5twfdoi',
      size: 226,
      cid: 'bafybeid6dici3ld6bhrw5abw6iely4afwxasmqw5xsynprcxakp5twfdoi',
      selective_car_cid: '',
      delta_content_id: 0,
      delta_node_url: 'http://localhost:1414',
      bucket_uuid: '77a69e06-06a8-11ee-824a-40a6b7203650',
      status: 'pinned',
      piece_cid: '',
      piece_size: 0,
      inclusion_proof: null,
      verifier_data: null,
      deal_id: 0,
      deal_uuid: '',
      last_message: '',
      miner: '',
      make_deal: true,
      created_at: '2023-06-09T09:49:09.550372125Z',
      updated_at: '2023-06-09T09:49:09.550372185Z'
    }
  ]
}
Storing data locally
Success stored id 17907
Download: https://hackfs-coeus.estuary.tech/edge/gw/bafybeid6dici3ld6bhrw5abw6iely4afwxasmqw5xsynprcxakp5twfdoi
Status: https://hackfs-coeus.estuary.tech/edge/open/status/content/17907
Updating smart contract
Complete! {
  to: '0x144a27553991F2e5964B903665b2003254edaf80',
  from: '0x3e7311e0Fc89fA633433076be268ae007A1b827a',
  contractAddress: null,
  transactionIndex: 0,
  root: '0x0000000000000000000000000000000000000000000000000000000000000000',
  gasUsed: BigNumber { _hex: '0x75f9d0', _isBigNumber: true },
  logsBloom: '0x00000000000000000000000000000000000010000000000040000000400000000000000000000005000420008000000000004200000000004000000000000000000000000080400000000000040000018000000000000000000000000000000400000000120000000008000020000808000000100000000000000000000000002000000080000002000000010000000000000000020001000000001000000000000010000000000000000000000000000000048001048000000000000000000000400001000000000000004004080000000000000000020000000200000020000040004000000000081000000000000000000400000000000004000000000000',
  blockHash: '0x353b32aec5157ad75fabb18823f0fe8d344d44cef61570f17ce7cd3aed78eedc',
  transactionHash: '0x14f610754c52ff3a1591cbf3118cf8632e902a0f41220565ca8cb4dcb3523cf0',
  logs: [],
  blockNumber: 16644,
  confirmations: 2,
  cumulativeGasUsed: BigNumber { _hex: '0x00', _isBigNumber: true },
  effectiveGasPrice: BigNumber { _hex: '0x6fa55e33', _isBigNumber: true },
  status: 1,
  type: 2,
  byzantium: true,
  events: []
}