search

dragonflyoss / Dragonfly

This repository has be archived and moved to the new repository https://github.com/dragonflyoss/Dragonfly2.
https://d7y.io
Apache License 2.0
6k stars 774 forks source link

support peer to peer prefetch #1488

Open hhhhsdxxxx opened 3 years ago

hhhhsdxxxx commented 3 years ago

Ⅰ. Describe what this PR did

Only first seed node will download resources from source, others will download(prefetch) from other peer nodes instead.

feature/seed implements the new p2p distribution system for image-service.

  1. high latency of downloading chunk data
  2. short live time of dfget
  1. dfdaemon/dfget have a long time live time.
  2. dfdaemon/dfget download data from source instead of supernode
  3. only dfdaemon apply as a seed successfully from supernode, can download data from seed.
  4. one resource can have >1 seeds
  5. non-seed peer can download data from seed node.
  6. supernode only record p2p network of resource and dfdaemon fetch network from supernode

all seed download from seed ------> first seed node download from seed, other seed nodes download from other seed nodes.

Ⅱ. Does this pull request fix one issue?

Ⅲ. Why don't you add test cases (unit test/integration test)? (你真的觉得不需要加测试吗？)

Ⅳ. Describe how to verify it

Ⅴ. Special notes for reviews

pouchrobot commented 3 years ago

We found this is your first time to contribute to Dragonfly, @hhhhsdxxxx 👏 We really appreciate it. Just remind that you have read the contribution guide: https://github.com/dragonflyoss/Dragonfly/blob/master/CONTRIBUTING.md If you didn't, you should do that first. If done, welcome again and please enjoy hacking! 🍻

hhhhsdxxxx commented 3 years ago

@jim3ma cc

starnop commented 3 years ago

@hhhhsdxxxx Cloud you please add a detailed description for this PR problem is this PR to solve? For example, explain the usage scenarios, the problems to be solved, etc.

The current scheduling algorithm will give priority to downloading from other peers in the P2P network. When there is no peer node available in the P2P network, it will try to download from the supernode. If the supernode is currently serving too much(In default is 4.) pieces are being downloaded, the peer will wait for a while.

also cc/ @jim3ma

hhhhsdxxxx commented 3 years ago

@hhhhsdxxxx Cloud you please add a detailed description for this PR problem is this PR to solve? For example, explain the usage scenarios, the problems to be solved, etc.

The current scheduling algorithm will give priority to downloading from other peers in the P2P network. When there is no peer node available in the P2P network, it will try to download from the supernode. If the supernode is currently serving too much(In default is 4.) pieces are being downloaded, the peer will wait for a while.

also cc/ @jim3ma

updated. It's a different p2p system service for image-service.