RubenKelevra
commented
4 years ago When I read the code right, this fix should be included as well, right @marten-seemann?
Closed Stebalien closed 4 years ago
RubenKelevra
commented
4 years ago When I read the code right, this fix should be included as well, right @marten-seemann?
marten-seemann
commented
4 years ago @RubenKelevra That fix is already contained in v0.15.7, which was released with go-libp2p-quic-transport v0.3.7.
RubenKelevra
commented
4 years ago @marten-seemann ah! was thrown off by the two different commit IDs for the same fix. Thanks for the fast response :)
Stebalien
commented
4 years ago RC1 has been released: https://dist.ipfs.io/go-ipfs/v0.6.0-rc1
RubenKelevra
commented
4 years ago If this turns out to be not a noise issue, we probably want to fix it before shipping 0.6.
RubenKelevra
commented
4 years ago This bugfix in badger might be worth including since the badger datastores are running out of disk space eventually without it.
RubenKelevra
commented
4 years ago This bugfix in badger for high memory usage while GC'ing should be included as well.
RubenKelevra
commented
4 years ago @Stebalien wrote:
Although you're right, this could very much be a large cause of the "killing routers" problem. I also think there may be a simple solution we can get into the next release (attempting now).
Will we change the default settings in this release to include the server-filters, to fix #7300 and #3320 as well as https://github.com/ipfs/go-ipfs/issues/6932 and https://github.com/libp2p/go-libp2p/issues/436? :)
Stebalien
commented
4 years ago Unfortunately, those badger fixes haven't been backported to version 1 yet.
Will we change the default settings in this release to include the server-filters
No, unfortunately that would prevent nodes from dialing each other on a private network.
The proposed fix here is to avoid dialing private addresses as part of queries on the public DHT. However, that hasn't been implemented yet. But there are some tricky edge cases here and we haven't implemented it yet.
RubenKelevra
commented
4 years ago Well, local nodes are quite unlikely to find each other through the DHT. So it makes no sense to announce the private IPs at all, doesn't it? 🤔
The swarm filter on the other hand could be overruled by manual 'swarm connect' and peers which was found via mDNS.
If a user really wants to connect peers from the same or different private subnets, he/she should use either 'swarm connect' or add them to the bootstrap nodes list.
So it boils down to just filter IPs while learning them from other peers or reading them from the DHT.
Stebalien
commented
4 years ago The proposed fix here is to avoid dialing private addresses as part of queries on the public DHT.
^^
RubenKelevra
commented
4 years ago The proposed fix here is to avoid dialing private addresses as part of queries on the public DHT.
^^
Alright, maybe I've read that a bit too fast :grin:
RubenKelevra
commented
4 years ago Unfortunately, those badger fixes haven't been backported to version 1 yet.
I created a tracking issue upstream - they might not be aware :)
Stebalien
commented
4 years ago Hello early testers, we now have a fully featured and tested RC online for public testing: https://dist.ipfs.io/go-ipfs/v0.6.0-rc6
Please be aware that this RC contains a config migration. It should run pretty much instantaneously, but it will add add QUIC listener addresses to your configs by default when upgrading from the previous release.
We also recommend that you try out the often requested "peering" (sometimes called friending, bonding, etc.) feature. This feature allows specifying a set of peers to keep open connections to at all times (useful for, e.g., connecting a gateway and a storage cluster).
Early Testers:
DavidBurela
commented
4 years ago The peering support makes me so happy.
swedneck
commented
4 years ago Peering doesn't seem to work for me, unless i got the syntax completely wrong (a full example in the docs would help).
I just get Error: error loading plugins: failure to decode config: invalid character '}' looking for beginning of object key string
Stebalien
commented
4 years ago @swedneck https://github.com/ipfs/go-ipfs/pull/7466?
Stebalien
commented
4 years ago go-ipfs 0.6.0-rc7 is out with two changes:
Swarm.Transports section for configuring transports. You can use this to prefer or disable the new noise transport, or disable the new QUIC transport if necessary. This section replaces the Experimental.OverrideSecurityTransports option present in the last RC.This will almost certainly be the final RC in this release cycle. Please test widely. I'm hoping to cut a final release before the end of the week, if possible.
DavidBurela
commented
4 years ago It seems a migration isn't automatically run when upgrading to 0.6.0
I used ipfs-update to go from 0.5.1 -> 0.6.0-RC1 -> 0.6.0
0.6.0-RC1 was working fine.
But after moving to 0.6.0, running IPFS with ipfs stats repo gives the error Error: ipfs repo needs migration
I can't find any information in ipfs --help or ipfs repo --help on how to migrate the repo.
Stebalien
commented
4 years ago That's strange. ipfs-update should do this for you. IPFS was probably running when you ran ipfs-update and, unfortunately, we don't run the migration in that case.
The simplest way to run the migration is to start the daemon with ipfs daemon --migrate.
DavidBurela
commented
4 years ago I had stopped the daemon, still got the log to show it. ipfs-update usually fails to replace the binary if the daemon is still running.
Thanks, running ipfs daemon --migrate did fix the issue.
Feedback on this, it seems like you should be able to run the migration without needing to run the daemon. e.g. ipfs repo migrate. Instead of needing to run the daemon, stopping it, and then restarting the system service.
Stebalien
commented
4 years ago Ah. I assume your systemd unit sets IPFS_PATH?
I had stopped the daemon, still got the log to show it. ipfs-update usually fails to replace the binary if the daemon is still running.
On linux? What version of ipfs-update are you using?
Feedback on this, it seems like you should be able to run the migration without needing to run the daemon. e.g. ipfs repo migrate. Instead of needing to run the daemon, stopping it, and then restarting the system service.
https://github.com/ipfs/go-ipfs/issues/7471
Agreed.
DavidBurela
commented
4 years ago Ubuntu 20.0.4
ipfs was installed initially using sudo ipfs-update install latest, so that it put it into /usr/local/bin/ipfs
And just to cover all bases, this is the systemd entry, which references it also from /usr/local/bin/ipfs. Which was stopped before using ipfs-update to bring it up to the latest.
[Unit]
Description=IPFS Daemon
After=syslog.target network.target remote-fs.target nss-lookup.target
[Service]
Type=simple
Restart=always
ExecStart=/usr/local/bin/ipfs daemon --enable-namesys-pubsub
User=ipfs-admin
[Install]
WantedBy=multi-user.targetOk, got it. The issue is that sudo ipfs-update runs ipfs-update as root (with HOME=/root and IPFS_PATH=/root/.ipfs). Your systemd unit runs IPFS as ipfs-admin with IPFS_PATH=/path/to/ipfs-admin-home/.ipfs.
Unfortunately, this isn't something we can readily solve in ipfs update itself. The best we can do is make migrations less painful.
For now, I recommend passing the --migrate flag in the systemd unit. See https://github.com/ipfs/go-ipfs/blob/master/misc/systemd/ipfs.service for an example.
Stebalien
commented
4 years ago @aschmahmann could you create a 0.7.0 release issue and close this one?
go-ipfs 0.6.0 Release
RC7: https://dist.ipfs.io/go-ipfs/v0.6.0-rc7
🗺 What's left for release
🚢 Estimated shipping date
June 9th, 2020
🔦 Highlights
ipfs-404.html404 page support #4233.MIGRATION
This release contains a small config migration to enable QUIC by default. This migration will:
/p2p/Qm...syntax for multiaddrs instead of the/ipfs/Qm...syntax./ip4/0.0.0.0/tcp/1234, this migration will add a listen address for/ip4/0.0.0.0/udp/1234/quic.Changelog
Full Changelog
- github.com/ipfs/go-ipfs: - feat: bump version to 0.6.0-rc1 - feat: add noise support ([ipfs/go-ipfs#7365](https://github.com/ipfs/go-ipfs/pull/7365)) - feat: implement peering service ([ipfs/go-ipfs#7362](https://github.com/ipfs/go-ipfs/pull/7362)) - Include the git blob id of the dir-index bundle in the ETag ([ipfs/go-ipfs#7360](https://github.com/ipfs/go-ipfs/pull/7360)) - feat: bootstrap in dht when the routing table is empty ([ipfs/go-ipfs#7340](https://github.com/ipfs/go-ipfs/pull/7340)) - quic: remove experimental status and add it to the default config ([ipfs/go-ipfs#7349](https://github.com/ipfs/go-ipfs/pull/7349)) - fix: support directory listings even if a 404 page is present ([ipfs/go-ipfs#7339](https://github.com/ipfs/go-ipfs/pull/7339)) - doc(plugin): document plugin config ([ipfs/go-ipfs#7309](https://github.com/ipfs/go-ipfs/pull/7309)) - test(sharness): fix fuse tests ([ipfs/go-ipfs#7320](https://github.com/ipfs/go-ipfs/pull/7320)) - docs: update experimental-features doc with IPNS over pubsub changes. ([ipfs/go-ipfs#7334](https://github.com/ipfs/go-ipfs/pull/7334)) - docs: cleanup config formatting ([ipfs/go-ipfs#7336](https://github.com/ipfs/go-ipfs/pull/7336)) - fix(gateway): ensure directory listings have Content-Type text/html ([ipfs/go-ipfs#7330](https://github.com/ipfs/go-ipfs/pull/7330)) - test(sharness): test the local symlink ([ipfs/go-ipfs#7332](https://github.com/ipfs/go-ipfs/pull/7332)) - misc config/experimental-features doc fixes ([ipfs/go-ipfs#7333](https://github.com/ipfs/go-ipfs/pull/7333)) - fix: correctly trim resolved IPNS addresses ([ipfs/go-ipfs#7331](https://github.com/ipfs/go-ipfs/pull/7331)) - Gateway renders pretty 404 pages if available ([ipfs/go-ipfs#4233](https://github.com/ipfs/go-ipfs/pull/4233)) - feat: add a dht stat command ([ipfs/go-ipfs#7221](https://github.com/ipfs/go-ipfs/pull/7221)) - fix: update dists url for OpenBSD support ([ipfs/go-ipfs#7311](https://github.com/ipfs/go-ipfs/pull/7311)) - docs: X-Forwarded-Proto: https ([ipfs/go-ipfs#7306](https://github.com/ipfs/go-ipfs/pull/7306)) - fix(mkreleaselog): make robust against running in different working directories ([ipfs/go-ipfs#7310](https://github.com/ipfs/go-ipfs/pull/7310)) - fix(mkreleasenotes): include commits directly to master ([ipfs/go-ipfs#7296](https://github.com/ipfs/go-ipfs/pull/7296)) - write api file automically ([ipfs/go-ipfs#7282](https://github.com/ipfs/go-ipfs/pull/7282)) - systemd: disable swap-usage for ipfs ([ipfs/go-ipfs#7299](https://github.com/ipfs/go-ipfs/pull/7299)) - systemd: add helptext ([ipfs/go-ipfs#7265](https://github.com/ipfs/go-ipfs/pull/7265)) - systemd: add the link to the docs ([ipfs/go-ipfs#7287](https://github.com/ipfs/go-ipfs/pull/7287)) - systemd: add state directory setting ([ipfs/go-ipfs#7288](https://github.com/ipfs/go-ipfs/pull/7288)) - Update go version required to build ([ipfs/go-ipfs#7289](https://github.com/ipfs/go-ipfs/pull/7289)) - pin: implement pin/ls with only CoreApi ([ipfs/go-ipfs#6774](https://github.com/ipfs/go-ipfs/pull/6774)) - update go-libp2p-quic-transport to v0.3.7 ([ipfs/go-ipfs#7278](https://github.com/ipfs/go-ipfs/pull/7278)) - Docs: Delete section headers for removed features ([ipfs/go-ipfs#7277](https://github.com/ipfs/go-ipfs/pull/7277)) - Chore: update dir-index-html to v1.0.5 ([ipfs/go-ipfs#7275](https://github.com/ipfs/go-ipfs/pull/7275)) - README.md: typo ([ipfs/go-ipfs#7061](https://github.com/ipfs/go-ipfs/pull/7061)) - PR autocomment: Only comment for first-time contributors ([ipfs/go-ipfs#7270](https://github.com/ipfs/go-ipfs/pull/7270)) - Fixed typo in config.md ([ipfs/go-ipfs#7267](https://github.com/ipfs/go-ipfs/pull/7267)) - Fixes #7252 - Uses gabriel-vasile/mimetype to support additional content types ([ipfs/go-ipfs#7262](https://github.com/ipfs/go-ipfs/pull/7262)) - update go-libp2p-quic-transport to v0.3.6 ([ipfs/go-ipfs#7266](https://github.com/ipfs/go-ipfs/pull/7266)) - Updates bash completions to be compatible with zsh ([ipfs/go-ipfs#7261](https://github.com/ipfs/go-ipfs/pull/7261)) - systemd service enhancements + run as system user ([ipfs/go-ipfs#7259](https://github.com/ipfs/go-ipfs/pull/7259)) - upgrade to go 1.14.2 ([ipfs/go-ipfs#7130](https://github.com/ipfs/go-ipfs/pull/7130)) - Add module files for go-ipfs-as-a-library example ([ipfs/go-ipfs#7146](https://github.com/ipfs/go-ipfs/pull/7146)) - feat(gateway): show the absolute path and CID every time ([ipfs/go-ipfs#7219](https://github.com/ipfs/go-ipfs/pull/7219)) - fix: do not use hard coded IPNS Publish maximum timeout duration ([ipfs/go-ipfs#7256](https://github.com/ipfs/go-ipfs/pull/7256)) - Auto-comment on submitted PRs ([ipfs/go-ipfs#7248](https://github.com/ipfs/go-ipfs/pull/7248)) - Fixes Github link. ([ipfs/go-ipfs#7239](https://github.com/ipfs/go-ipfs/pull/7239)) - docs: fix subdomain examples in CHANGELOG ([ipfs/go-ipfs#7240](https://github.com/ipfs/go-ipfs/pull/7240)) - doc: add snap to the release checklist ([ipfs/go-ipfs#7253](https://github.com/ipfs/go-ipfs/pull/7253)) - Welcome message for users opening their first issue ([ipfs/go-ipfs#7247](https://github.com/ipfs/go-ipfs/pull/7247)) - feat: bump to 0.6.0-dev ([ipfs/go-ipfs#7249](https://github.com/ipfs/go-ipfs/pull/7249)) - github.com/ipfs/go-bitswap (v0.2.13 -> v0.2.15): - Improve peer manager performance ([ipfs/go-bitswap#395](https://github.com/ipfs/go-bitswap/pull/395)) - fix: return wants from engine.WantlistForPeer() ([ipfs/go-bitswap#390](https://github.com/ipfs/go-bitswap/pull/390)) - Add autocomment configuration - calculate message latency ([ipfs/go-bitswap#386](https://github.com/ipfs/go-bitswap/pull/386)) - fix: use one less go-routine per session (#377) ([ipfs/go-bitswap#377](https://github.com/ipfs/go-bitswap/pull/377)) - Add standard issue template - github.com/ipfs/go-ipfs-cmds (v0.2.2 -> v0.2.9): - Fix go-ipfs#7242: Remove "HEAD" from Allow methods ([ipfs/go-ipfs-cmds#195](https://github.com/ipfs/go-ipfs-cmds/pull/195)) - Staticcheck fixes (#196) ([ipfs/go-ipfs-cmds#196](https://github.com/ipfs/go-ipfs-cmds/pull/196)) - doc: update docs for interface changes ([ipfs/go-ipfs-cmds#197](https://github.com/ipfs/go-ipfs-cmds/pull/197)) - Add standard issue template - github.com/ipfs/go-ipfs-config (v0.5.3 -> v0.7.1): - feat: add an option for security transport experiments ([ipfs/go-ipfs-config#97](https://github.com/ipfs/go-ipfs-config/pull/97)) - feat: add peering service config section ([ipfs/go-ipfs-config#96](https://github.com/ipfs/go-ipfs-config/pull/96)) - fix: include key size in key init method ([ipfs/go-ipfs-config#95](https://github.com/ipfs/go-ipfs-config/pull/95)) - QUIC: remove experimental config option ([ipfs/go-ipfs-config#93](https://github.com/ipfs/go-ipfs-config/pull/93)) - fix boostrap peers ([ipfs/go-ipfs-config#94](https://github.com/ipfs/go-ipfs-config/pull/94)) - default config: add QUIC listening ports + quic to mars.i.ipfs.io ([ipfs/go-ipfs-config#91](https://github.com/ipfs/go-ipfs-config/pull/91)) - feat: remove strict signing pubsub option. ([ipfs/go-ipfs-config#90](https://github.com/ipfs/go-ipfs-config/pull/90)) - Add autocomment configuration - Add Init Alternative allowing specification of ED25519 key ([ipfs/go-ipfs-config#78](https://github.com/ipfs/go-ipfs-config/pull/78)) - github.com/ipfs/go-log/v2 (v2.0.5 -> v2.0.8): failed to fetch repo - github.com/ipfs/go-mfs (v0.1.1 -> v0.1.2): - Fix incorrect mutex unlock call in File.Open ([ipfs/go-mfs#82](https://github.com/ipfs/go-mfs/pull/82)) - Add autocomment configuration - Add standard issue template - test: add Directory.ListNames test ([ipfs/go-mfs#81](https://github.com/ipfs/go-mfs/pull/81)) - doc: add a lead maintainer - Update README.md with newer travis badge ([ipfs/go-mfs#78](https://github.com/ipfs/go-mfs/pull/78)) - github.com/ipfs/interface-go-ipfs-core (v0.2.7 -> v0.3.0): - add Pin.IsPinned(..) ([ipfs/interface-go-ipfs-core#50](https://github.com/ipfs/interface-go-ipfs-core/pull/50)) - Add autocomment configuration - Add standard issue template - extra time for dht spin-up ([ipfs/interface-go-ipfs-core#61](https://github.com/ipfs/interface-go-ipfs-core/pull/61)) - feat: make the CoreAPI expose a streaming pin interface ([ipfs/interface-go-ipfs-core#49](https://github.com/ipfs/interface-go-ipfs-core/pull/49)) - test: fail early on err to avoid an unrelated panic ([ipfs/interface-go-ipfs-core#57](https://github.com/ipfs/interface-go-ipfs-core/pull/57)) - github.com/jbenet/go-is-domain (v1.0.3 -> v1.0.5): - Add OpenNIC domains to extended TLDs. ([jbenet/go-is-domain#15](https://github.com/jbenet/go-is-domain/pull/15)) - feat: add .crypto and .zil from UnstoppableDomains ([jbenet/go-is-domain#17](https://github.com/jbenet/go-is-domain/pull/17)) - chore: update IANA TLDs to version 2020051300 ([jbenet/go-is-domain#18](https://github.com/jbenet/go-is-domain/pull/18)) - github.com/libp2p/go-addr-util (v0.0.1 -> v0.0.2): - farewell gx; thanks for serving us well. - lol, save the file first - fix discuss badge - add discuss link to readme - fix: fdcostly should take only the prefix into account ([libp2p/go-addr-util#5](https://github.com/libp2p/go-addr-util/pull/5)) - add gomod support // tag v0.0.1 ([libp2p/go-addr-util#17](https://github.com/libp2p/go-addr-util/pull/17)) - github.com/libp2p/go-libp2p (v0.8.3 -> v0.9.2): - fix: group observations by zeroing port ([libp2p/go-libp2p#949](https://github.com/libp2p/go-libp2p/pull/949)) - fix: fix connection gater in transport constructor ([libp2p/go-libp2p#948](https://github.com/libp2p/go-libp2p/pull/948)) - Fix potential flakiness in TestIDService ([libp2p/go-libp2p#945](https://github.com/libp2p/go-libp2p/pull/945)) - make the {F=>f}iltersConnectionGater private. (#946) ([libp2p/go-libp2p#946](https://github.com/libp2p/go-libp2p/pull/946)) - Filter observed addresses (#917) ([libp2p/go-libp2p#917](https://github.com/libp2p/go-libp2p/pull/917)) - fix: don't try to marshal a nil record ([libp2p/go-libp2p#943](https://github.com/libp2p/go-libp2p/pull/943)) - add test to demo missing peer records after listen ([libp2p/go-libp2p#941](https://github.com/libp2p/go-libp2p/pull/941)) - fix: don't leak a goroutine if a peer connects and immediately disconnects ([libp2p/go-libp2p#942](https://github.com/libp2p/go-libp2p/pull/942)) - no signed peer records for mocknets (#934) ([libp2p/go-libp2p#934](https://github.com/libp2p/go-libp2p/pull/934)) - implement connection gating at the top level (#881) ([libp2p/go-libp2p#881](https://github.com/libp2p/go-libp2p/pull/881)) - various identify fixes and nits (#922) ([libp2p/go-libp2p#922](https://github.com/libp2p/go-libp2p/pull/922)) - Remove race between ID, Push & Delta (#907) ([libp2p/go-libp2p#907](https://github.com/libp2p/go-libp2p/pull/907)) - fix a compilation error introduced in 077a818. (#919) ([libp2p/go-libp2p#919](https://github.com/libp2p/go-libp2p/pull/919)) - exchange signed routing records in identify (#747) ([libp2p/go-libp2p#747](https://github.com/libp2p/go-libp2p/pull/747)) - github.com/libp2p/go-libp2p-autonat (v0.2.2 -> v0.2.3): - react to incoming events ([libp2p/go-libp2p-autonat#65](https://github.com/libp2p/go-libp2p-autonat/pull/65)) - github.com/libp2p/go-libp2p-blankhost (v0.1.4 -> v0.1.6): - subscribe connmgr to net notifications ([libp2p/go-libp2p-blankhost#45](https://github.com/libp2p/go-libp2p-blankhost/pull/45)) - add WithConnectionManager option to blankhost ([libp2p/go-libp2p-blankhost#44](https://github.com/libp2p/go-libp2p-blankhost/pull/44)) - Blank host should support signed records ([libp2p/go-libp2p-blankhost#42](https://github.com/libp2p/go-libp2p-blankhost/pull/42)) - github.com/libp2p/go-libp2p-connmgr (v0.2.1 -> v0.2.3): - decaying tags: support removal and closure. (#72) ([libp2p/go-libp2p-connmgr#72](https://github.com/libp2p/go-libp2p-connmgr/pull/72)) - implement decaying tags. (#61) ([libp2p/go-libp2p-connmgr#61](https://github.com/libp2p/go-libp2p-connmgr/pull/61)) - github.com/libp2p/go-libp2p-core (v0.5.3 -> v0.5.6): - decaying tags: support removal and closure. (#151) ([libp2p/go-libp2p-core#151](https://github.com/libp2p/go-libp2p-core/pull/151)) - minor godoc fixes. - minor godoc fixes. - implement Stringer for network.{Direction,Connectedness,Reachability}. (#150) ([libp2p/go-libp2p-core#150](https://github.com/libp2p/go-libp2p-core/pull/150)) - connmgr: introduce abstractions and functions for decaying tags. (#104) ([libp2p/go-libp2p-core#104](https://github.com/libp2p/go-libp2p-core/pull/104)) - Interface to verify if a peer supports a protocol without making allocations. ([libp2p/go-libp2p-core#148](https://github.com/libp2p/go-libp2p-core/pull/148)) - add connection gating interfaces and types. (#139) ([libp2p/go-libp2p-core#139](https://github.com/libp2p/go-libp2p-core/pull/139)) - github.com/libp2p/go-libp2p-kad-dht (v0.7.11 -> v0.8.0): - fix: protect useful peers in low buckets ([libp2p/go-libp2p-kad-dht#634](https://github.com/libp2p/go-libp2p-kad-dht/pull/634)) - Double the usefulness interval for peers in the Routing Table (#651) ([libp2p/go-libp2p-kad-dht#651](https://github.com/libp2p/go-libp2p-kad-dht/pull/651)) - enhancement/remove-unused-variable ([libp2p/go-libp2p-kad-dht#633](https://github.com/libp2p/go-libp2p-kad-dht/pull/633)) - Put back TestSelfWalkOnAddressChange ([libp2p/go-libp2p-kad-dht#648](https://github.com/libp2p/go-libp2p-kad-dht/pull/648)) - Routing Table Refresh manager (#601) ([libp2p/go-libp2p-kad-dht#601](https://github.com/libp2p/go-libp2p-kad-dht/pull/601)) - Boostrap empty RT and Optimize allocs when we discover new peers (#631) ([libp2p/go-libp2p-kad-dht#631](https://github.com/libp2p/go-libp2p-kad-dht/pull/631)) - fix all flaky tests ([libp2p/go-libp2p-kad-dht#628](https://github.com/libp2p/go-libp2p-kad-dht/pull/628)) - Update default concurrency parameter ([libp2p/go-libp2p-kad-dht#605](https://github.com/libp2p/go-libp2p-kad-dht/pull/605)) - clean up a channel that was dangling ([libp2p/go-libp2p-kad-dht#620](https://github.com/libp2p/go-libp2p-kad-dht/pull/620)) - github.com/libp2p/go-libp2p-kbucket (v0.4.1 -> v0.4.2): - Reduce allocs in AddPeer (#81) ([libp2p/go-libp2p-kbucket#81](https://github.com/libp2p/go-libp2p-kbucket/pull/81)) - NPeersForCpl and collapse empty buckets (#77) ([libp2p/go-libp2p-kbucket#77](https://github.com/libp2p/go-libp2p-kbucket/pull/77)) - github.com/libp2p/go-libp2p-noise (null -> v0.1.1): - Further Optimizations and refactor benchmarking code. ([libp2p/go-libp2p-noise#81](https://github.com/libp2p/go-libp2p-noise/pull/81)) - github.com/libp2p/go-libp2p-peerstore (v0.2.3 -> v0.2.4): - Implement the FirstSupportedProtocol API. ([libp2p/go-libp2p-peerstore#147](https://github.com/libp2p/go-libp2p-peerstore/pull/147)) - github.com/libp2p/go-libp2p-pubsub (v0.2.7 -> v0.3.0): - add doc comment for GossipSubDout - fix typo - add note about opp. graft threshold - stop using term "gossip mesh" - add doc comments to gossipsub constants - sleep longer for travis - sleep in test after bumping fake clock - rm empty slice alloc - better range check for expected value in test - loosen test expectations to appease travis - this is getting a bit silly - play with test timing on travis - add delay in TestTagTracerDeliveryTags - even fewer peers in connmgr test - use fewer peers in connmgr test - inline decayingDeliveryTag method into addDeliveryTag - close decaying tags when leaving topic - rm misleading comment - sort imports - check reject reason before deleting near-first state - update dependencies and use preset decay fns - use map to track near-first deliveries in tagTracer - free up drec.peers when we're done with it - add delivery tags for "near-first" msg deliveries - bump blankhost to 0.1.6 - lock for reading in bumpDeliveryTag - add nil check for decayer - increase decaying tag interval - test delivery tags vs sybil storm - bump connmgr dep to latest PR commit - unit tests for tagTracer - add tagTracer to apply connmgr tags - temporarily depend on PR commits - track discovery API change - use the proto we already have in AddPeer to determine whether the connection houses the stream - only count an outbound connection if it has a pubsub stream - cache scores throughout the heartbeat - maintain outboud peer quota - precompute outbound peers on AddPeer, add Dout - only allow GRAFT from outbound peers when we are at or over Dhi - check if already in the mesh in handleGraft to support concurrent grafting - [blacklist] Use .Has check before add in time cached version - adjust signature - [validation] Use label for break - [validation] break not used correct - need return or give label, i choose return - deflake TestGossipsubAttackSpamIHAVE - update tests for quadratic p4 - make p4 quadratic - update IHAVE spam test to test for behavioural tests because of broken promises - add test for gossip_tracer - update godoc for P7 - gossip tracer implementation - gossip tracer preliminaries - extended internal tracing for multiple objects, not just score - try to deflake MeshFailure test; travis managed to fail on this - godoc - better backoff handling - add backoff to PRUNE messages - extend behaviour penalty test to ensure coverage - fix test for behaviour penalty validation - fix typo - incorporate p7 into score - add p7 configuration to score parameters - factor out doSendRPC and doDropRPC methods - only call fragmentRPC if we're over the size limit - close stream in test - split large IWANT / IHAVE messages, add unit test - fix prune message in test - try to send control messages in one RPC when fragmenting - fix race condition in rpc fragmentation test - rewrite test for rpc fragmentation - fragment large RPCs in sendRPC - downgrade graylist Warn log to Info - downgrade tracer Error logs to Warn - use log.Warn instead of Warning - add defensive checks for potentially duplicate traces - add some more validator tests - add comment about validationThrottled suerpseding ValidationIgnore - fix apparent flakiness in test - fix typo - add test exercising score integration with extended validation - add test for multitopic validation with extended validation interface - add test for rejections with ignore validator decision - implement extended validator support - Prevent multiple invocations of relay cancel function - Move counter incrementing before announce - Fix race in the TestTopicRelayReuse - Merge stop announcing conditions - Tests for topic relay - Relay incoming messages - Topic relay discovery - Topic relay cancelling - Topic relay support - add defaults for DecayInterval and DecayToZero - add utility function for computing score parameter decays - fix typo - add tests for message rejection paths - fix bug in penalties from message rejection - add test for peer score inspection - be less dependent on accurate time calculations in test - relax message delivery requirements for randomsub tests - make randomsub test reliable - add network size argument to randomsub - add randomsub test - add sqrt behaviour to randomsub - Score tests (#288) ([libp2p/go-libp2p-pubsub#288](https://github.com/libp2p/go-libp2p-pubsub/pull/288)) - reify fanoutTTL in the GossipSubRouter struct - add test for invalid message author - increase delay for TestPeerDisconnect - increase travis friendliness for new tests - add test for multi-topic validation - add test for assorted validation options that were not covered - add test for assorted pubsub options that were not covered - fix broken WithMessageAuthor option - add test for announce retry - improve reliability of star topology tests - pull overlay parameters into the GossipSubRouter struct - make star topology tests more reliable - import grouping - add signed peer records only in the center of the star for signed peer record test - add test with signed peer records - restore coverage for score_params.go - validate that TimeInMeshQuantum is non-zero - fix the global variable mutation races - increase time to disconnect in floodsub disconnection test - a little bit more time to avoid races with restoring mutated config variables - add a heartbeat's worth of delay before restoring mutated globals - reduce prune backoff times for opportunistic grafting test - increase wait time for IWANT spam test to 1s - add test for opportunistic grafting - make initial delay for direct peer connections a variable - add direct test for gossipsub router's piggybackControl - rename tests for consistency - add test for negative scores below the gossip threshold - remove unnecessary length check - add test for gossipsub enough peers - add test for gossipsub flood publishing - add test for gossipsub direct peers - docs: one more badger - docs: fix badgers - close the remote tracer in test - fix close race in basicTracer - add test for tracers - add test for peer score parameter validation - update go-libp2p-discovery, go mod tidy - move validation tests to their own file - deflake TestValidatorOverload - truncate per peer message id lists - log counts in IHAVE flood protection measures - add backoff penalty for GRAFT floods - make maximum number of IHAVE messages accepted per heartbeat configurable - improve handling of IHAVE floods - fix data races in spam tests - rename gossipsub_attack_test to gossipsub_spam_test - fix GRAFT spam test - fix IHAVE spam test - Add IHAVE spam test - test: add test for invalid message spam - test: GRAFT spam and GRAFT backoff tests - test: add IWANT spam test - add protections from IHAVE floods - add protections from GRAFT floods - gomod tidy - fix IPv4 -vs- IPv6 address handling in peer address tracking - more rigorously defeat unspecified address gremlins - fix issue with unspecific IPv6 addrs in conn IP tracking - gomod: update go-multiaddr-net - fix bug in peerScore.setIPs - disable flaky and irrelevant tests - configurable number of peers for opportunistic grafting - fix typo - fix typo - opportunistic grafting - use a configurable variable for direct peer reconnect ticks - eagerly send gossip at the end heartbeat - comments and nits. - support direct peering agreements - update REAMDE - correctly attribute invalid message deliveries due to self origin rejection - better handling of message originating from self but forwarded by another peer - move self-publish check to pubsub - rename delivery_ constants to satisfy the camelcase worshipers and nitpickers - use named constants for tracing message rejection reasons - add IP whitelist for colocation factor penalty - use manet.ToIP instead of hand-rolling multiaddr to IP conversion - gomod: use go-multiaddr-net v0.1.3 - don't forward messages with source ourselves but not published by ourselves - increase connection manager weight for mesh peers - only retain negative scores - topic score cap - godoc nits for parameter weights. - wrap topic param validation error - rename score debug option, address some comments around inspection - reraise GossipSubPrunePeers to 16 - use a struct for aggregating score thresholds - make PX optional and disabled by default, gate by acceptPXThreshold - move score parameter types to their own file - gomod tidy - fix rebase artifacts - track heartbeat ticks, only clear backoff every 15. - check exact expiration date of backoff when pruning because of backoff - add backoff when pruning because of negative score - refresh backoff when pruning during the backoff period - shuffle IWANT list - add option to periodically inspect peer scores for debugging purposes - validate the threshold values - move score starting nil check inside the score implementation - short-circuit score computation in peer selection filters - distinguish throttled validation because of a full worker queue. - fix bug in WithEventTracer - peer score parameter validation - implement periodic maintenance - implement getIPs and refreshIPs - implement message delivery struct details - double check delivery to aboid duplicate counts - finetune sticky mesh failure penalty parameter - sticky mesh failure penalty parameter - clean up the logic of mesh delivery window - more logical code ordering - simplify duplicate mesh delivery tracking - comment cosmetics - track message delivery time within the validation pipeline - peer tracking - mesh tracking - minimize hash table accesses - message delivery tracking - respect the time in mesh cap - respec decay to zero parameter in counter decay - add nil check in Score function - fix peer IP tracking structure - peer score refresh: counter decay and mesh activation - peer score calculation - explicit Start for peer scoring to inject the router - peer score parameters - limit message retransmissions through IWANT requests - track peer (re)transmissions in message cache - adaptive gossip dissemination - select good peers for PX, tune the px related limits - allow gossipsub to graylist peers based on score - router.Publish doesn't need the source, it takes the whole message now - backoff when we prune too, refuse to reGRAFT backed off peers - increase D_score to 4 - correctly implement flood publishing - implement flood publishing option - keep D peer when pruning for oversubscription - remove peers with negative score from fanout - hook score into heartbeat maintenance - heartbeat doPX is a more complex decision - filter peers with negative score in initial mesh on join - hook peer score into control message handlers - peer score scaffolding - hook the tracer for peer scoring - gomod: use go-libp2p-core@peer-records and go-libp2p-peerstore@certified-addrs - update PR branch dependencies - track changes to peer records in -core - gomod: use go-libp2p-core@peer-records and go-libp2p-peerstore@certified-addrs - fix: fix a busy loop while offline (#278) ([libp2p/go-libp2p-pubsub#278](https://github.com/libp2p/go-libp2p-pubsub/pull/278)) - docs: update readme welcome message - docs: fix CI badges - docs: Tidy up (#271) ([libp2p/go-libp2p-pubsub#271](https://github.com/libp2p/go-libp2p-pubsub/pull/271)) - Update label-syncer.yml - Delete label-syncer.yml - gomod tidy - fix rebase artifacts - don't spawn a goroutine for scheduling connections - shuffle peers when limiting px set - add limit to the number of peers to connect to from px - connect to peers obtained through px - backoff grafting to peers that have pruned us - peer exchange on prune - protocol ID for gossipsub v1.1 - gomod: use go-libp2p-core@peer-records and go-libp2p-peerstore@certified-addrs - renaming in error messages & local var - add missing continue to error case - fix import & var naming - update PR branch dependencies - track changes to peer records in -core - don't spawn a goroutine for scheduling connections - shuffle peers when limiting px set - add limit to the number of peers to connect to from px - make connection timeout a variable, set for 30s (instead of 10s) - extend star topology test to assert that no peer is left with a single connection - trace peer exchange - test prune px with a star topology - connect to peers obtained through px - backoff grafting to peers that have pruned us - peer exchange on prune - protocol ID for gossipsub v1.1 - extend ControlPrune with peer exchange information - gomod: use go-libp2p-core@peer-records and go-libp2p-peerstore@certified-addrs - set sendTo to remote peer id in trace events ([libp2p/go-libp2p-pubsub#268](https://github.com/libp2p/go-libp2p-pubsub/pull/268)) - make wire protocol message size configurable. (#261) ([libp2p/go-libp2p-pubsub#261](https://github.com/libp2p/go-libp2p-pubsub/pull/261)) - github.com/libp2p/go-libp2p-pubsub-router (v0.2.1 -> v0.3.0): - feat: update pubsub ([libp2p/go-libp2p-pubsub-router#76](https://github.com/libp2p/go-libp2p-pubsub-router/pull/76)) - github.com/libp2p/go-libp2p-quic-transport (v0.3.7 -> v0.4.1): - update quic-go to v0.15.8 ([libp2p/go-libp2p-quic-transport#160](https://github.com/libp2p/go-libp2p-quic-transport/pull/160)) - Revert "fix a potential nil-pointer panic." (#159) ([libp2p/go-libp2p-quic-transport#159](https://github.com/libp2p/go-libp2p-quic-transport/pull/159)) - fix a potential nil-pointer panic. (#158) ([libp2p/go-libp2p-quic-transport#158](https://github.com/libp2p/go-libp2p-quic-transport/pull/158)) - close the connection when it is refused by InterceptSecured ([libp2p/go-libp2p-quic-transport#157](https://github.com/libp2p/go-libp2p-quic-transport/pull/157)) - gate QUIC connections via new ConnectionGater (#152) ([libp2p/go-libp2p-quic-transport#152](https://github.com/libp2p/go-libp2p-quic-transport/pull/152)) - github.com/libp2p/go-libp2p-record (v0.1.2 -> v0.1.3): - feat: add a better record error ([libp2p/go-libp2p-record#39](https://github.com/libp2p/go-libp2p-record/pull/39)) - github.com/libp2p/go-libp2p-swarm (v0.2.3 -> v0.2.5): - Rank Dial addresses (#212) ([libp2p/go-libp2p-swarm#212](https://github.com/libp2p/go-libp2p-swarm/pull/212)) - implement connection gating support: intercept peer, address dials, upgraded conns (#201) ([libp2p/go-libp2p-swarm#201](https://github.com/libp2p/go-libp2p-swarm/pull/201)) - fix: avoid calling AddChild after the process may shutdown. ([libp2p/go-libp2p-swarm#207](https://github.com/libp2p/go-libp2p-swarm/pull/207)) - github.com/libp2p/go-libp2p-transport-upgrader (v0.2.0 -> v0.3.0): - call the connection gater when accepting connections and after crypto handshake (#55) ([libp2p/go-libp2p-transport-upgrader#55](https://github.com/libp2p/go-libp2p-transport-upgrader/pull/55)) - remove leftover ginkgo test file. (#61) ([libp2p/go-libp2p-transport-upgrader#61](https://github.com/libp2p/go-libp2p-transport-upgrader/pull/61)) - simplify tests by removing ginkgo/gomega. (#60) ([libp2p/go-libp2p-transport-upgrader#60](https://github.com/libp2p/go-libp2p-transport-upgrader/pull/60)) - github.com/libp2p/go-openssl (v0.0.4 -> v0.0.5): - add binding for OBJ_create ([libp2p/go-openssl#5](https://github.com/libp2p/go-openssl/pull/5)) - github.com/libp2p/go-yamux (v1.3.5 -> v1.3.7): - tighten lock around appending new chunks of read data in stream ([libp2p/go-yamux#28](https://github.com/libp2p/go-yamux/pull/28)) - fix: unlock recvLock in all cases. ([libp2p/go-yamux#25](https://github.com/libp2p/go-yamux/pull/25)) - github.com/lucas-clemente/quic-go (v0.15.7 -> v0.15.8): - create a milestone version for v0.15.x - rewrite the frame sorter ([lucas-clemente/quic-go#2561](https://github.com/lucas-clemente/quic-go/pull/2561)) - retire conn IDs with sequence numbers smaller than the currently active ([lucas-clemente/quic-go#2563](https://github.com/lucas-clemente/quic-go/pull/2563)) - remove unused readOffset member variable in receiveStream ([lucas-clemente/quic-go#2559](https://github.com/lucas-clemente/quic-go/pull/2559)) - fix int overflow when parsing the transport parameters ([lucas-clemente/quic-go#2564](https://github.com/lucas-clemente/quic-go/pull/2564)) - use struct{} instead of bool in window update queue ([lucas-clemente/quic-go#2555](https://github.com/lucas-clemente/quic-go/pull/2555)) - update the protobuf library to google.golang.org/protobuf/proto ([lucas-clemente/quic-go#2554](https://github.com/lucas-clemente/quic-go/pull/2554)) - use the correct error code for crypto stream errors ([lucas-clemente/quic-go#2546](https://github.com/lucas-clemente/quic-go/pull/2546)) - bundle small writes on streams ([lucas-clemente/quic-go#2538](https://github.com/lucas-clemente/quic-go/pull/2538)) - reduce the length of the unprocessed packet chan in the session ([lucas-clemente/quic-go#2534](https://github.com/lucas-clemente/quic-go/pull/2534)) - fix flaky session unit test ([lucas-clemente/quic-go#2537](https://github.com/lucas-clemente/quic-go/pull/2537)) - add a send stream test that randomly acknowledges and loses data ([lucas-clemente/quic-go#2535](https://github.com/lucas-clemente/quic-go/pull/2535)) - fix size calculation for version negotiation packets ([lucas-clemente/quic-go#2542](https://github.com/lucas-clemente/quic-go/pull/2542)) - run all unit tests with race detector ([lucas-clemente/quic-go#2528](https://github.com/lucas-clemente/quic-go/pull/2528)) - add support for the ChaCha20 interop test case ([lucas-clemente/quic-go#2517](https://github.com/lucas-clemente/quic-go/pull/2517)) - fix buffer use after it was released when sending an INVALID_TOKEN error ([lucas-clemente/quic-go#2524](https://github.com/lucas-clemente/quic-go/pull/2524)) - run the internal and http3 tests with race detector on Travis ([lucas-clemente/quic-go#2385](https://github.com/lucas-clemente/quic-go/pull/2385)) - reset the PTO when dropping a packet number space ([lucas-clemente/quic-go#2527](https://github.com/lucas-clemente/quic-go/pull/2527)) - stop the deadline timer in Stream.Read and Write ([lucas-clemente/quic-go#2519](https://github.com/lucas-clemente/quic-go/pull/2519)) - don't reset pto_count on Initial ACKs ([lucas-clemente/quic-go#2513](https://github.com/lucas-clemente/quic-go/pull/2513)) - fix all race conditions in the session tests ([lucas-clemente/quic-go#2525](https://github.com/lucas-clemente/quic-go/pull/2525)) - make sure that the server's run loop returned when closing ([lucas-clemente/quic-go#2526](https://github.com/lucas-clemente/quic-go/pull/2526)) - fix flaky proxy test ([lucas-clemente/quic-go#2522](https://github.com/lucas-clemente/quic-go/pull/2522)) - stop the timer when the session's run loop returns ([lucas-clemente/quic-go#2516](https://github.com/lucas-clemente/quic-go/pull/2516)) - make it more likely that a STREAM frame is bundled with the FIN ([lucas-clemente/quic-go#2504](https://github.com/lucas-clemente/quic-go/pull/2504)) - github.com/multiformats/go-multiaddr (v0.2.1 -> v0.2.2): - absorb go-maddr-filter; rm stale Makefile targets; upgrade deps (#124) ([multiformats/go-multiaddr#124](https://github.com/multiformats/go-multiaddr/pull/124)) - github.com/multiformats/go-multibase (v0.0.2 -> v0.0.3): - Base36 implementation ([multiformats/go-multibase#36](https://github.com/multiformats/go-multibase/pull/36)) - Even more tests/benchmarks, less repetition in-code ([multiformats/go-multibase#34](https://github.com/multiformats/go-multibase/pull/34)) - Beef up tests before adding new codec ([multiformats/go-multibase#32](https://github.com/multiformats/go-multibase/pull/32)) - Remove GX, bump spec submodule, fix tests ([multiformats/go-multibase#31](https://github.com/multiformats/go-multibase/pull/31))✅ Release Checklist
For each RC published in each stage:
version.gohas been updatedChecklist:
make test)make test_go_lint)./bin/mkreleaselogto generate a nice starter listversion.gohas been updatedgit merge vX.Y.Z).version.gotovX.(Y+1).0-dev.❤️ Contributors
Would you like to contribute to the IPFS project and don't know how? Well, there are a few places you can get started:
help wantedlabel in the go-ipfs repo⁉️ Do you have questions?
The best place to ask your questions about IPFS, how it works and what you can do with it is at discuss.ipfs.io. We are also available at the
#ipfschannel on Freenode, which is also accessible through our Matrix bridge.