feat: aggregate filter subscriptions to do bulk subs with peer

chaitanyaprem commented 3 days ago

Instead of subscribing for each application filter that is added, filters are queued and subscribed in batches so that number of subscriptions reduces towards peers and locally as well.

Application filter is the filterID and contentFilter provided by status-go whereas aggregatedFilter is the one that is subscribed with a Filter Peer.

https://github.com/waku-org/go-waku/pull/1144 needs to be merged before merging this.

Closes # https://github.com/status-im/status-go/issues/5262

status-im-auto commented 3 days ago

Jenkins Builds

Click to see older builds (28)

| :grey_question: | Commit | :hash: | Finished (UTC) | Duration | Platform | Result | |-|-|-|-|-|-|-| | :heavy_multiplication_x: | 92c10f8f | [#1](https://ci.status.im/job/status-go/job/prs/job/tests/job/PR-5440/1/) | 2024-06-28 06:11:18 | ~2 min | `tests` | [:page_facing_up:`log`](https://ci.status.im/job/status-go/job/prs/job/tests/job/PR-5440/1//consoleText) | | :heavy_check_mark: | 92c10f8f | [#1](https://ci.status.im/job/status-go/job/prs/job/linux/job/PR-5440/1/) | 2024-06-28 06:12:32 | ~3 min | `linux` | [:package:`zip`](https://status-im-prs.ams3.cdn.digitaloceanspaces.com/status-go-desktop-240628-060838-92c10f-pr5440.zip) | | :heavy_check_mark: | 92c10f8f | [#1](https://ci.status.im/job/status-go/job/prs/job/ios/job/PR-5440/1/) | 2024-06-28 06:12:43 | ~4 min | `ios` | [:package:`zip`](https://status-im-prs.ams3.cdn.digitaloceanspaces.com/status-go-ios-240628-060837-92c10f-pr5440.zip) | | :heavy_check_mark: | 92c10f8f | [#1](https://ci.status.im/job/status-go/job/prs/job/android/job/PR-5440/1/) | 2024-06-28 06:14:15 | ~5 min | `android` | [:package:`aar`](https://status-im-prs.ams3.cdn.digitaloceanspaces.com/status-go-android-240628-060837-92c10f-pr5440.aar) | | :heavy_check_mark: | 92c10f8f | [#2](https://ci.status.im/job/status-go/job/prs/job/android/job/PR-5440/2/) | 2024-06-28 08:28:47 | ~1 min | `android` | [:package:`aar`](https://status-im-prs.ams3.cdn.digitaloceanspaces.com/status-go-android-240628-082714-92c10f-pr5440.aar) | | :heavy_multiplication_x: | 92c10f8f | [#2](https://ci.status.im/job/status-go/job/prs/job/tests/job/PR-5440/2/) | 2024-06-28 08:29:40 | ~2 min | `tests` | [:page_facing_up:`log`](https://ci.status.im/job/status-go/job/prs/job/tests/job/PR-5440/2//consoleText) | | :heavy_check_mark: | 92c10f8f | [#2](https://ci.status.im/job/status-go/job/prs/job/ios/job/PR-5440/2/) | 2024-06-28 08:29:51 | ~2 min | `ios` | [:package:`zip`](https://status-im-prs.ams3.cdn.digitaloceanspaces.com/status-go-ios-240628-082714-92c10f-pr5440.zip) | | :heavy_check_mark: | 92c10f8f | [#2](https://ci.status.im/job/status-go/job/prs/job/linux/job/PR-5440/2/) | 2024-06-28 08:30:57 | ~3 min | `linux` | [:package:`zip`](https://status-im-prs.ams3.cdn.digitaloceanspaces.com/status-go-desktop-240628-082714-92c10f-pr5440.zip) | | | | | | | | | | :heavy_multiplication_x: | d571935c | [#3](https://ci.status.im/job/status-go/job/prs/job/tests/job/PR-5440/3/) | 2024-06-28 08:46:32 | ~1 min | `tests` | [:page_facing_up:`log`](https://ci.status.im/job/status-go/job/prs/job/tests/job/PR-5440/3//consoleText) | | :heavy_check_mark: | d571935c | [#3](https://ci.status.im/job/status-go/job/prs/job/ios/job/PR-5440/3/) | 2024-06-28 08:47:53 | ~2 min | `ios` | [:package:`zip`](https://status-im-prs.ams3.cdn.digitaloceanspaces.com/status-go-ios-240628-084504-d57193-pr5440.zip) | | :heavy_check_mark: | d571935c | [#3](https://ci.status.im/job/status-go/job/prs/job/linux/job/PR-5440/3/) | 2024-06-28 08:48:49 | ~3 min | `linux` | [:package:`zip`](https://status-im-prs.ams3.cdn.digitaloceanspaces.com/status-go-desktop-240628-084508-d57193-pr5440.zip) | | :heavy_check_mark: | d571935c | [#3](https://ci.status.im/job/status-go/job/prs/job/android/job/PR-5440/3/) | 2024-06-28 08:50:50 | ~5 min | `android` | [:package:`aar`](https://status-im-prs.ams3.cdn.digitaloceanspaces.com/status-go-android-240628-084504-d57193-pr5440.aar) | | | | | | | | | | :heavy_multiplication_x: | 6b9f571a | [#4](https://ci.status.im/job/status-go/job/prs/job/tests/job/PR-5440/4/) | 2024-06-28 08:49:37 | ~1 min | `tests` | [:page_facing_up:`log`](https://ci.status.im/job/status-go/job/prs/job/tests/job/PR-5440/4//consoleText) | | :heavy_check_mark: | 6b9f571a | [#4](https://ci.status.im/job/status-go/job/prs/job/ios/job/PR-5440/4/) | 2024-06-28 08:50:57 | ~2 min | `ios` | [:package:`zip`](https://status-im-prs.ams3.cdn.digitaloceanspaces.com/status-go-ios-240628-084819-6b9f57-pr5440.zip) | | :heavy_check_mark: | 6b9f571a | [#4](https://ci.status.im/job/status-go/job/prs/job/linux/job/PR-5440/4/) | 2024-06-28 08:51:26 | ~2 min | `linux` | [:package:`zip`](https://status-im-prs.ams3.cdn.digitaloceanspaces.com/status-go-desktop-240628-084905-6b9f57-pr5440.zip) | | :heavy_check_mark: | 6b9f571a | [#4](https://ci.status.im/job/status-go/job/prs/job/android/job/PR-5440/4/) | 2024-06-28 08:53:22 | ~2 min | `android` | [:package:`aar`](https://status-im-prs.ams3.cdn.digitaloceanspaces.com/status-go-android-240628-085107-6b9f57-pr5440.aar) | | | | | | | | | | :heavy_check_mark: | 097c04e8 | [#5](https://ci.status.im/job/status-go/job/prs/job/ios/job/PR-5440/5/) | 2024-06-28 13:03:57 | ~2 min | `ios` | [:package:`zip`](https://status-im-prs.ams3.cdn.digitaloceanspaces.com/status-go-ios-240628-130100-097c04-pr5440.zip) | | :heavy_check_mark: | 097c04e8 | [#5](https://ci.status.im/job/status-go/job/prs/job/linux/job/PR-5440/5/) | 2024-06-28 13:04:24 | ~3 min | `linux` | [:package:`zip`](https://status-im-prs.ams3.cdn.digitaloceanspaces.com/status-go-desktop-240628-130100-097c04-pr5440.zip) | | :heavy_multiplication_x: | 097c04e8 | [#5](https://ci.status.im/job/status-go/job/prs/job/tests/job/PR-5440/5/) | 2024-06-28 13:04:48 | ~3 min | `tests` | [:page_facing_up:`log`](https://ci.status.im/job/status-go/job/prs/job/tests/job/PR-5440/5//consoleText) | | :heavy_check_mark: | 097c04e8 | [#5](https://ci.status.im/job/status-go/job/prs/job/android/job/PR-5440/5/) | 2024-06-28 13:07:56 | ~6 min | `android` | [:package:`aar`](https://status-im-prs.ams3.cdn.digitaloceanspaces.com/status-go-android-240628-130057-097c04-pr5440.aar) | | | | | | | | | | :heavy_multiplication_x: | 31d8d3b9 | [#6](https://ci.status.im/job/status-go/job/prs/job/tests/job/PR-5440/6/) | 2024-06-29 02:33:49 | ~1 min | `tests` | [:page_facing_up:`log`](https://ci.status.im/job/status-go/job/prs/job/tests/job/PR-5440/6//consoleText) | | :heavy_check_mark: | 31d8d3b9 | [#6](https://ci.status.im/job/status-go/job/prs/job/linux/job/PR-5440/6/) | 2024-06-29 02:34:33 | ~2 min | `linux` | [:package:`zip`](https://status-im-prs.ams3.cdn.digitaloceanspaces.com/status-go-desktop-240629-023204-31d8d3-pr5440.zip) | | :heavy_check_mark: | 31d8d3b9 | [#6](https://ci.status.im/job/status-go/job/prs/job/android/job/PR-5440/6/) | 2024-06-29 02:34:41 | ~2 min | `android` | [:package:`aar`](https://status-im-prs.ams3.cdn.digitaloceanspaces.com/status-go-android-240629-023159-31d8d3-pr5440.aar) | | :heavy_check_mark: | 31d8d3b9 | [#6](https://ci.status.im/job/status-go/job/prs/job/ios/job/PR-5440/6/) | 2024-06-29 02:35:05 | ~3 min | `ios` | [:package:`zip`](https://status-im-prs.ams3.cdn.digitaloceanspaces.com/status-go-ios-240629-023159-31d8d3-pr5440.zip) | | | | | | | | | | :heavy_multiplication_x: | 279c2e28 | [#7](https://ci.status.im/job/status-go/job/prs/job/tests/job/PR-5440/7/) | 2024-06-29 02:38:25 | ~1 min | `tests` | [:page_facing_up:`log`](https://ci.status.im/job/status-go/job/prs/job/tests/job/PR-5440/7//consoleText) | | :heavy_check_mark: | 279c2e28 | [#7](https://ci.status.im/job/status-go/job/prs/job/android/job/PR-5440/7/) | 2024-06-29 02:39:18 | ~2 min | `android` | [:package:`aar`](https://status-im-prs.ams3.cdn.digitaloceanspaces.com/status-go-android-240629-023654-279c2e-pr5440.aar) | | :heavy_check_mark: | 279c2e28 | [#7](https://ci.status.im/job/status-go/job/prs/job/linux/job/PR-5440/7/) | 2024-06-29 02:39:19 | ~2 min | `linux` | [:package:`zip`](https://status-im-prs.ams3.cdn.digitaloceanspaces.com/status-go-desktop-240629-023654-279c2e-pr5440.zip) | | :heavy_check_mark: | 279c2e28 | [#7](https://ci.status.im/job/status-go/job/prs/job/ios/job/PR-5440/7/) | 2024-06-29 02:39:44 | ~2 min | `ios` | [:package:`zip`](https://status-im-prs.ams3.cdn.digitaloceanspaces.com/status-go-ios-240629-023654-279c2e-pr5440.zip) |

:grey_question:	Commit	:hash:	Finished (UTC)	Duration	Platform	Result
:heavy_multiplication_x:	c6703455	#8	2024-07-01 02:01:48	~1 min	`tests`	:page_facing_up:`log`
:heavy_check_mark:	c6703455	#8	2024-07-01 02:02:37	~2 min	`linux`	:package:`zip`
:heavy_check_mark:	c6703455	#8	2024-07-01 02:02:43	~2 min	`android`	:package:`aar`
:heavy_check_mark:	c6703455	#8	2024-07-01 02:03:22	~3 min	`ios`	:package:`zip`

:heavy_check_mark:	2ec58730	#9	2024-07-01 14:26:47	~3 min	`ios`	:package:`zip`
:heavy_check_mark:	2ec58730	#9	2024-07-01 14:31:46	~8 min	`linux`	:package:`zip`
:heavy_check_mark:	2ec58730	#9	2024-07-01 14:32:42	~9 min	`android`	:package:`aar`

chaitanyaprem commented 10 hours ago

any idea why i am getting following error in CI ? @richard-ramos @cammellos I saw that the uuid import is there in some other files in status-go already, so not sure what is causing this.

[2024-07-01T02:01:45.757Z] wakuv2/filter_manager.go:8: File is not `goimports`-ed with -local github.com/ethereum/go-ethereum,github.com/status-im/status-go,github.com/waku-org/go-waku (goimports)

[2024-07-01T02:01:45.757Z]  "github.com/google/uuid"

[2024-07-01T02:01:45.757Z] make: *** [Makefile:395: lint] Error 1

status-im / status-go

feat: aggregate filter subscriptions to do bulk subs with peer #5440

Jenkins Builds