Unable to PIN existing CID in IPFS browser node

[logesh2496]

Pinning only works If I upload some data using unixfs and then use that CID. I couldn't pin a CID thats already available on ipfs.

So how do I use a CID thats already available on ipfs and pin it in my node?

example: I want to pin this CID: QmRrzxiXGefcF9VbbThGrkhiQeuhPFjRbGgqYZmxYXYkHr on my node

At the moment Im getting the below error:

errors.js:27 Uncaught (in promise) Error: Not Found at Module.notFoundError (errors.js:27:1) at MemoryBlockstore.get (memory.js:19:19) at #walkDag (pins.js:97:1) at eval (pins.js:52:1) at eval (index.js:111:1) at PQueue._PQueue_tryToStartAnother (index.js:285:1) at eval (index.js:135:1) at new Promise () at PQueue.add (index.js:99:1) at PinsImpl.add (pins.js:51:1) at async onPin (ipfs.js:69:17)

[achingbrain]

Can you please share a full repro case please? Pinning will fetch missing blocks from the network but the error thrown looks like you may be running offline or it could be unrelated.

[logesh2496]

I'm working on the nextjs examples provided in the helia-examples/helia-nextjs

export default async function getLibp2p ({ datastore }) {
  return await createLibp2p({
    datastore,
    addresses: {
      listen: [
        '/webrtc'
      ],
    },
    transports: [
      webSockets(),
      webRTC()
    ],
    connectionEncryption: [
      noise()
    ],
    streamMuxers: [
      yamux()
    ],
    services: {
      identify: identifyService()
    }
  })
}

const blockstore = new MemoryBlockstore()
const datastore = new MemoryDatastore()

const libp2p = await getLibp2p({ datastore })

// create a Helia node
return await createHelia({
  datastore,
  blockstore,
  libp2p
})

const cid = CID.parse('QmRrzxiXGefcF9VbbThGrkhiQeuhPFjRbGgqYZmxYXYkHr');
const pin = await helia.pins?.add(cid);
//Error here

[achingbrain]

Ah, I think there was a small bug with how the internal blockstores were being created - please can you try with helia@1.2.1?

[logesh2496]

That worked, thanks! btw is there any option to get progress from the pin operation? It takes a alot of time and the UI is waiting cluelessly, it would be great to have a progress bar that shows the % downloaded.

So How do I know how much of the content is downloaded, Thanks again!

[achingbrain]

Almost every API accepts a onProgress event callback with information about the current operation.

Note that it's impossible to know in advance how big a DAG is, unless it's part of the codec or data encoding.

For example @helia/unixfs has a .stat method that will return the UnixFS metadata from the root node that you can use before pinning to see how big the DAG is, but other codecs like dag-cbor may not provide this feature.

[logesh2496]

Tried, but it isn't working. Below is the code:

const cid = CID.parse('QmVXy4WyMaz4ajm2LUUjyu33cH7UDdytHvctdYmchdvdMy')

    const pin = await helia.pins.add(cid, {
        onProgress: (evt) => {
        console.info('pin event', evt.type, evt.detail)
      }
    });

You can try with the same CID, its an image and I expected to see some progress but didn't do anything, and no error too.

[logesh2496]

@achingbrain any help here?

[whizzzkid]

@logesh2496 I tried your example, but CustomProgressEvent does not have detail property has the detail property, but the value is generic according to the event type, so modifying your example gives me:

const testcid = CID.parse('QmVXy4WyMaz4ajm2LUUjyu33cH7UDdytHvctdYmchdvdMy')

const pin = await helia.pins.add(testcid, {
    onProgress: (evt) => console.log('pin event', evt)
});

and I get:

Added file: bafkreih7eug2oqxx7ft427q4fcce6lnl73q6kmtzaxeldllx266mxr6os4
pin event CustomProgressEvent {
  type: 'blocks:get:bitswap:get',
  defaultPrevented: false,
  cancelable: false,
  timeStamp: 1845.2695000171661
}
pin event CustomProgressEvent {
  type: 'bitswap:network:find-providers',
  defaultPrevented: false,
  cancelable: false,
  timeStamp: 1846.5817499756813
}
pin event CustomProgressEvent {
  type: 'bitswap:network:dial',
  defaultPrevented: false,
  cancelable: false,
  timeStamp: 2155.9229999780655
}
pin event CustomProgressEvent {
  type: 'bitswap:want-block:block',
  defaultPrevented: false,
  cancelable: false,
  timeStamp: 3077.842584013939
}
pin event CustomProgressEvent {
  type: 'bitswap:want-block:block',
  defaultPrevented: false,
  cancelable: false,
  timeStamp: 3078.4099590182304
}
pin event CustomProgressEvent {
  type: 'blocks:get:blockstore:put',
  defaultPrevented: false,
  cancelable: false,
  timeStamp: 3079.382166981697
}
pin event CustomProgressEvent {
  type: 'helia:pin:add',
  defaultPrevented: false,
  cancelable: false,
  timeStamp: 3080.369666993618
}
pin event CustomProgressEvent {
  type: 'blocks:get:bitswap:get',
  defaultPrevented: false,
  cancelable: false,
  timeStamp: 3081.4172499775887
}
pin event CustomProgressEvent {
  type: 'bitswap:network:find-providers',
  defaultPrevented: false,
  cancelable: false,
  timeStamp: 3081.5880420207977
}
pin event CustomProgressEvent {
  type: 'bitswap:network:dial',
  defaultPrevented: false,
  cancelable: false,
  timeStamp: 3084.3530420064926
}
pin event CustomProgressEvent {
  type: 'bitswap:network:send-wantlist',
  defaultPrevented: false,
  cancelable: false,
  timeStamp: 3084.5707499980927
}
pin event CustomProgressEvent {
  type: 'bitswap:network:dial',
  defaultPrevented: false,
  cancelable: false,
  timeStamp: 3185.429834008217
}
pin event CustomProgressEvent {
  type: 'bitswap:want-block:block',
  defaultPrevented: false,
  cancelable: false,
  timeStamp: 3683.4932919740677
}
pin event CustomProgressEvent {
  type: 'bitswap:want-block:block',
  defaultPrevented: false,
  cancelable: false,
  timeStamp: 3684.056666970253
}
pin event CustomProgressEvent {
  type: 'blocks:get:blockstore:put',
  defaultPrevented: false,
  cancelable: false,
  timeStamp: 3685.3889999985695
}
pin event CustomProgressEvent {
  type: 'helia:pin:add',
  defaultPrevented: false,
  cancelable: false,
  timeStamp: 3685.647750020027
}
pin event CustomProgressEvent {
  type: 'blocks:get:bitswap:get',
  defaultPrevented: false,
  cancelable: false,
  timeStamp: 3686.0805420279503
}
pin event CustomProgressEvent {
  type: 'bitswap:network:find-providers',
  defaultPrevented: false,
  cancelable: false,
  timeStamp: 3686.1782090067863
}
pin event CustomProgressEvent {
  type: 'bitswap:network:dial',
  defaultPrevented: false,
  cancelable: false,
  timeStamp: 3687.336709022522
}
pin event CustomProgressEvent {
  type: 'bitswap:network:send-wantlist',
  defaultPrevented: false,
  cancelable: false,
  timeStamp: 3687.422375023365
}
pin event CustomProgressEvent {
  type: 'bitswap:network:dial',
  defaultPrevented: false,
  cancelable: false,
  timeStamp: 3792.3495839834213
}
pin event CustomProgressEvent {
  type: 'bitswap:want-block:block',
  defaultPrevented: false,
  cancelable: false,
  timeStamp: 4064.3595420122147
}
pin event CustomProgressEvent {
  type: 'bitswap:want-block:block',
  defaultPrevented: false,
  cancelable: false,
  timeStamp: 4064.498000025749
}
pin event CustomProgressEvent {
  type: 'blocks:get:blockstore:put',
  defaultPrevented: false,
  cancelable: false,
  timeStamp: 4065.210792005062
}
pin event CustomProgressEvent {
  type: 'helia:pin:add',
  defaultPrevented: false,
  cancelable: false,
  timeStamp: 4065.5434169769287
}
pin event CustomProgressEvent {
  type: 'blocks:get:bitswap:get',
  defaultPrevented: false,
  cancelable: false,
  timeStamp: 4065.7790840268135
}
pin event CustomProgressEvent {
  type: 'bitswap:network:find-providers',
  defaultPrevented: false,
  cancelable: false,
  timeStamp: 4065.855333983898
}
pin event CustomProgressEvent {
  type: 'bitswap:network:dial',
  defaultPrevented: false,
  cancelable: false,
  timeStamp: 4068.029708981514
}
pin event CustomProgressEvent {
  type: 'bitswap:network:send-wantlist',
  defaultPrevented: false,
  cancelable: false,
  timeStamp: 4068.1246669888496
}
pin event CustomProgressEvent {
  type: 'bitswap:network:dial',
  defaultPrevented: false,
  cancelable: false,
  timeStamp: 4314.306874990463
}
pin event CustomProgressEvent {
  type: 'bitswap:want-block:block',
  defaultPrevented: false,
  cancelable: false,
  timeStamp: 4371.186583995819
}
pin event CustomProgressEvent {
  type: 'bitswap:want-block:block',
  defaultPrevented: false,
  cancelable: false,
  timeStamp: 4371.299166977406
}
pin event CustomProgressEvent {
  type: 'blocks:get:blockstore:put',
  defaultPrevented: false,
  cancelable: false,
  timeStamp: 4371.788959026337
}
pin event CustomProgressEvent {
  type: 'helia:pin:add',
  defaultPrevented: false,
  cancelable: false,
  timeStamp: 4372.11087501049
}
pin event CustomProgressEvent {
  type: 'blocks:get:bitswap:get',
  defaultPrevented: false,
  cancelable: false,
  timeStamp: 4372.401666998863
}
pin event CustomProgressEvent {
  type: 'bitswap:network:find-providers',
  defaultPrevented: false,
  cancelable: false,
  timeStamp: 4372.487167000771
}
pin event CustomProgressEvent {
  type: 'bitswap:network:dial',
  defaultPrevented: false,
  cancelable: false,
  timeStamp: 4373.6427500247955
}
pin event CustomProgressEvent {
  type: 'bitswap:network:send-wantlist',
  defaultPrevented: false,
  cancelable: false,
  timeStamp: 4373.714959025383
}
pin event CustomProgressEvent {
  type: 'bitswap:network:dial',
  defaultPrevented: false,
  cancelable: false,
  timeStamp: 4466.31320899725
}
pin event CustomProgressEvent {
  type: 'bitswap:want-block:block',
  defaultPrevented: false,
  cancelable: false,
  timeStamp: 4910.378666996956
}
pin event CustomProgressEvent {
  type: 'bitswap:want-block:block',
  defaultPrevented: false,
  cancelable: false,
  timeStamp: 4910.617749989033
}
pin event CustomProgressEvent {
  type: 'blocks:get:blockstore:put',
  defaultPrevented: false,
  cancelable: false,
  timeStamp: 4911.877375006676
}
pin event CustomProgressEvent {
  type: 'helia:pin:add',
  defaultPrevented: false,
  cancelable: false,
  timeStamp: 4912.259917020798
}

Reiterating what @achingbrain said before, it would be hard to use this to create a progress bar, but seems to work as expected. Can you please share a repo/pr where this fails?

[achingbrain]

CustomProgressEvent does not have detail property

Yes it does, but the value is generic according to the event type.

[logesh2496]

Thanks @achingbrain & @whizzzkid for taking the time to resolve my query.

However the method pins.add() itself wasn't working in the 1.2.1 version. I noticed that many changes had been made to the helia-examples for nextjs now: https://github.com/ipfs-examples/helia-examples/commit/ccc40f0546eee8d9124a46f7f7651296577c365a

So I updated the helia package version to 1.3.2 and removed getLibp2p as shown in the current examples repo to keep it simple. I do get some events now, but it errors out pretty quickly:

I understand that the event types are generic. Where can I read about all the event types during the pinning operation? any events related to the below list could be helpful for better UX.

• Pinning Started
• Pinning Completed
• Pinning Errored

Questions:

1. Pinning doesn't work and errors.
2. Does helia package provide the above-listed event types? & reference to all other event types during pinning operation.

Thanks.

[marten-seemann]

@logesh2496 Are you sure these nodes are online? The errors look like handshake timeouts, which you'd expect to see if the nodes has gone offline.

[github-actions[bot]]

Oops, seems like we needed more information for this issue, please comment with more details or this issue will be closed in 7 days.

[logesh2496]

the page still shows the status of the node as "Online"

[achingbrain]

No, Marten means the remote nodes, the ones being connected to.

[logesh2496]

ah, got it. Shouldn't it find another node that has a copy of the file? How do I handle this situation where I need to pin a file?

[SgtPooki]

I can't seem to get files to pin using helia either

[BigLep]

@SgtPooki : what repo case were you trying? (I just want to make sure we're all trying the same thing.)

[SgtPooki]

what repo case were you trying? (I just want to make sure we're all trying the same thing.)

I can't remember at the moment but I will come back to this when I can