eingenito
commented
6 years ago Hey @OhJerry. As far as I know your question doesn't have a simple answer. I would say first that there are definitely aspects of EC2 instances that might affect the performance of IPFS when compared to your local tests, and the configuration of EC2 instances is so variable it's hard to suggest which ones you might be encountering. The storage subsystem you are using (EBS, SSD, HDD, provisioned or not, etc.) might not like IPFS's usage patterns. You might be CPU bound, which may be surprising but on fast networks IPFS can consume a lot of CPU (for lots of reasons) compared with rsync. Also - rsync is more efficient at data transfer than IPFS in general. Your local results are a little surprising in this regard.
I can offer a couple of suggestions, Apologies if you've already tried them.
- Because you are connecting peers explicitly you can try running your daemons like:
ipfs daemon --routing=noneto turn off propagation of provider records when fetching data. - You can try turning sync'ing off in your datastore (this has to be done in your config file; search for sync). It decreases IPFS's reliability but it might help you figure out your performance differences.
- You can try using the experimental, faster datastore by doing
ipfs init --profile=badgerdswhen initializing your installs. Same caveat as above.
I hope that's helpful. If you do figure something out about IPFS performance on AWS that you think is generally useful please consider posting your findings on the IPFS forums so others can read about it. Thanks.
Stebalien
Version information:
ipfs version 0.4.18-dev
Type:
Weird performance
Description:
I am trying to measure the performance of ipfs on AWS. To do so, I built up one local network on my two local machines and one local network on two EC2 instances from AWS. I tried to comparing transfers speed of same file on different networks and also comparing with the speed using rsync ssh.
Here is how I build the local networks. Start a EC2 instance:
IPFS private network:
Servers share the same key
And here are some parts of the stats I collected: Local machine: rsync -avzhe ssh: real 0m12.219s user 0m4.976s sys 0m3.720s
Local machine: ipfs pin add (faster than ipfs get): real 0m12.001s user 0m0.036s sys 0m0.008s
But when using AWS EC2, I tried both dedicated and shared tenancy, the speed of ipfs would be much slower than that of rsync:
AWS: rsync -avzhe ssh: real 0m23.613s user 0m8.629s sys 0m5.245s
AWS: ipfs pin add: real 0m50.343s user 0m1.664s sys 0m3.883s
Each local network I am building contains only two servers, so it should be a simple one-to-one transmitting, and the speed should be about the same as "rsync ssh", which is exactly the result from first pair of data.
But when using AWS EC2, the speed of ipfs would be much slower than that of rsync. I have tried using dedicated tenancy, different types of instances, or using placement group to force instances close to each other physically in order to reduce the transmitting time. Although the speeds are different, the ratio of speed of rsync over ipfs is always 2-3.
I am currently working on this and trying to find out what makes the difference. People from ipfs community have helped me, but did not really solve the problem. I have already emailed AWS support, and was informed that they did not block ipfs intentionally, so there should not be such difference.
Any features of AWS would influence the performance of ipfs? Thanks!