search

tomeshnet / ipfs-live-streaming

Like HTTP live streaming, but with IPFS which is maybe better?
GNU General Public License v3.0
449 stars 74 forks source link

ipfs pin add performance #46

Open benhylau opened 6 years ago

benhylau commented 6 years ago

ipfs pin add takes about a minute, too slow for pinning 15 second ts chunks: https://github.com/tomeshnet/ipfs-live-streaming/blob/master/terraform/ipfs-mirror/ipfs-pin.sh#L6

benhylau commented 6 years ago

Our production endpoint live.mesh.world is currently running:

#!/bin/sh

IPFS_REF=$1

# Pin IPFS content
if [ -z "$(cat /home/ipfs/pin_log | grep $IPFS_REF)" ]; then
  echo $IPFS_REF >> /home/ipfs/pin_log
  wget -q "https://ipfs-gateway.mesh.world/ipfs/$IPFS_REF" -O "/tmp/$IPFS_REF"
  ipfs add /tmp/$IPFS_REF > /dev/null
  ipfs pin add $IPFS_REF > /dev/null   &
  rm -rf /tmp/$IPFS_REF

  mv /home/ipfs/pin_log /home/ipfs/pin_log.old
  tail -n 10 /home/ipfs/pin_log.old > /home/ipfs/pin_log
  rm -rf /home/ipfs/pin_log.old

fi

The ipfs-mirror instantiated by terraform ipfs-mirror-0.mesh.world pins with:

#!/bin/sh

IPFS_REF=$1

# Pin IPFS content
ipfs pin add /ipfs/$IPFS_REF
darkdrgn2k commented 5 years ago

Problem continues with IPFS 0.4.22

darkdrgn2k commented 5 years ago

a contact swarm connect was added to mirrors pointing at the main server. This seems to allow for quick ipfs add pin