search

sahib / brig

File synchronization on top of ipfs with git like interface & web based UI
https://brig.readthedocs.io
GNU Affero General Public License v3.0
568 stars 33 forks source link

Improve IPFS config #24

Closed makew0rld closed 5 years ago

makew0rld commented 5 years ago

The main thing you missed is setting Swarm.ConnMgr.Type to basic.

I tried to reduce this by setting the high and low water settings to a lower range, but this didn't seem to help.

This is because you didn't change the type, otherwise IPFS wouldn't do anything with those values.

Type Sets the type of connection manager to use, options are: "none" (no connection management) and "basic"

So hopefully this will change things. I also set Swarm.ConnMgr.GracePeriod to 60s, which means no connection will close until at least 60s have passed, even if HighWater has been reached. This is recommended when lowering the HighWater and LowWater values, as it keeps connections from being closed abruptly. You might need to play with these values, but try this config out and see what you think.

makew0rld commented 5 years ago

I am confused as to why the Travis CI build failed. I am having trouble understanding the log. Does it matter?

sahib commented 5 years ago

I am confused as to why the Travis CI build failed. I am having trouble understanding the log. Does it matter?

My bad. Fix is in latest develop.

sahib commented 5 years ago

So hopefully this will change things.

Cool! I'll give it a test.

makew0rld commented 5 years ago

@sahib Great! Let me know if it needs any changes, but everything should pretty much just work, I didn't change a lot. The testing is more long-term, about deciding on good values for HighWater, LowWater and GracePeriod.

sahib commented 5 years ago

Out of curiosity I checked how the ConnMgr.Type is handled in IPFS:

It is read here, which indicates that an empty string also selects basic but without picking up the low/high water settings. The default value seems to be already basic though. Also tested on my machine:

$ cat /tmp/ali/data/ipfs/config | jq '.Swarm.ConnMgr.Type' 
"basic"

I don't think setting it explicitly will make a difference, although it definitely won't hurt. I suggest we leave this PR open for now, if you find some other stuff to tune.

makew0rld commented 5 years ago

Interesting...

if you find some other stuff to tune

I did change the GracePeriod, which may help. Also, I noticed you've enabled QUIC. Is it actually being used to connect to remotes?

sahib commented 5 years ago

Is it actually being used to connect to remotes?

I don't know. How can I tell? lsof would just show me open UDP connections. Probably the equivalent of ipfs swarm addrs?

makew0rld commented 5 years ago

I just meant are you using the QUIC muiltaddr to connect to peers, such as /ip4/<address>/udp/<port>/quic?

sahib commented 5 years ago

brig dials to other peers via their ID, not by specifying an explicit address. Since I enabled QUIC, I assume that IPFS will use it, if the remote peer is capable of it. But I'm not quite sure how to test that, since all of the code in brig is agnostic to the type of the connection - I only care if it is a more or less reliable stream.

makew0rld commented 5 years ago

To be honest I am unsure what IPFS will do at that level at all. Enabling QUIC won't hurt, but I have no idea whether it will use it there or not. Something to look into, perhaps.

sahib commented 5 years ago

I will close that PR, since there are conflicts with develop due to the now externalized IPFS code. I added the grace period on my side in a82d3013.

Thanks for the efforts!

makew0rld commented 5 years ago

Ah okay, you're welcome, and thanks for adding it.