Weird timeline API behaviour

[ara4n]

I just had a go at implementing the timeline API in my experimental rust-sdk on Web test jig (aurora). I'm seeing very strange behaviour, which maps onto reported bugs on Element X. If I use the API via:

    let Ok(ui_room) = room_list_service.room(&id).await else {
        return Err(Error::Other(anyhow!("couldn't get room")));
    };

    let builder = match ui_room.default_room_timeline_builder().await {
        Ok(builder) => builder,
        Err(err) => {
            return Err(Error::Other(anyhow!("error when getting default timeline builder: {err}")));
        }
    };

    if let Err(err) = ui_room.init_timeline_with_builder(builder).await {
        return Err(Error::Other(anyhow!("error when creating default timeline: {err}")));
    }

    let (items, stream) = ui_room.timeline().unwrap().subscribe().await;

    // and then keep calling:
    let diff = stream.next().await.ok_or(anyhow!("no diffs"))?;

Then I get the following behaviour:

• items returns an up-to-date array of date sep + the most recent ~6 items in the timeline, with unique_ids 0..6
• diff then yields:
  • Clear (which deletes them all)
  • PushBack of the date sep (with unique_id 7)
  • PushBack of the most recent item in the timeline (with a new unique_id of 8, rather than 6 as it was before the clear)
  • then PushBacks of the previous 10 messages (meaning the timeline is now out of order), mixed with lots of Set as the items are gradually built up with RRs
  • then RemoveItem at index 1 to remove the out-of-order item
  • then PushBack to add it back (with unique_id 8) at the right point at the end.

In other words, the bugs are:

• [ ] We clear the timeline immediately after subscribing to it, despite having received a valid set of items. This is presumably the cause of https://github.com/element-hq/element-x-ios/issues/1234
• [x] We assign new unique_ids to the same items (probably not a disaster given we just Cleared the previous ones), but it's unsetlling to see the same items turning up with different ids.
• [ ] We replay the history as if it were live events, rather than just yielding merged items (which we must already have, given we returned them them in items). This is presumably the cause of https://github.com/element-hq/element-x-ios/issues/1895
• [ ] We put the most recent message at the top of the list of items, and then move it to its right place at the bottom once the items have loaded. This feels like it's trying to do an optimisation where it we show the most recent message asap and then prepend the scrollback later... but has it completely wrong. This probably also contributes to the dancing bubble animation symptoms in https://github.com/element-hq/element-x-ios/issues/1234. could also cause https://github.com/element-hq/element-x-android/issues/2491

c.f. https://github.com/matrix-org/matrix-rust-sdk/issues/1103 for known limitations of the timeline API, which doesn't look to include these.

[Hywan]

The Clear is “normal“ because the server replies with a limited flag. With EventCache coming, we won't need to clear the timeline anymore.

[bnjbvr]

We assign new unique_ids to the same items (probably not a disaster given we just Cleared the previous ones), but it's unsetlling to see the same items turning up with different ids.

I've checked this one because this shouldn't happen anymore. If it does, please provide a code example showing how to trigger it. The unique IDs are about to gain in importance with #4093, so we need them to be super-stable.