`ProviderContainer.pump()` doesn't seem to actually pump entirely

[timcreatedit]

Describe the bug When writing tests for a project I couldn't get tests to pass until I called .pump() twice in a row. This took me a long time to find, I tried to build the simplest example of what we're trying to do below, I think it's easier to see in code than whatever explanation I would come up with.

To Reproduce

import 'dart:async';

import 'package:flutter_test/flutter_test.dart';
import 'package:mocktail/mocktail.dart';
import 'package:riverpod/riverpod.dart';

/// For overriding in the tests
final tupleProvider = StreamProvider.autoDispose<(int? a, int? b)>(
  (ref) => throw UnimplementedError(),
);

final testProvider =
    StreamNotifierProvider.autoDispose<TestNotifier, int>(TestNotifier.new);

class TestNotifier extends AutoDisposeStreamNotifier<int> {
  @override
  Stream<int> build() async* {
    final (first, second) = await ref.watch(tupleProvider.future);

    if (first != null) yield first;
    if (second != null) yield second;
  }
}

void main() {
  group('testNotifier', () {
    late ProviderContainer container;
    late MockListener<AsyncValue<int>> listener;
    late StreamController<(int? a, int? b)> streamController;

    setUp(() {
      streamController = StreamController.broadcast();
      addTearDown(streamController.close);
      container = createContainer(
        overrides: [
          tupleProvider.overrideWith((ref) => streamController.stream),
        ],
      );
      listener = MockListener();
      container.listen(
        testProvider,
        listener.call,
        fireImmediately: true,
      );
    });

    test('this fails', () async {
      streamController.add((42, null));
      await container.pump();

      verifyInOrder([
        () => listener(null, const AsyncLoading()),
        () => listener(const AsyncLoading(), const AsyncData(42)),
      ]);
      verifyNoMoreInteractions(listener);
    });

    test('this works for some reason', () async {
      streamController.add((42, null));
      await container.pump();
      // FIXME it is unclear why a second pump is needed
      await container.pump();

      verifyInOrder([
        () => listener(null, const AsyncLoading()),
        () => listener(const AsyncLoading(), const AsyncData(42)),
      ]);
      verifyNoMoreInteractions(listener);
    });
  });
}

/// taken from https://riverpod.dev/docs/essentials/testing
ProviderContainer createContainer({
  ProviderContainer? parent,
  List<Override> overrides = const [],
  List<ProviderObserver>? observers,
}) {
  // Create a ProviderContainer, and optionally allow specifying parameters.
  final container = ProviderContainer(
    parent: parent,
    overrides: overrides,
    observers: observers,
  );

  // When the test ends, dispose the container.
  addTearDown(container.dispose);

  return container;
}

/// Can be used to listen to providers and verify interactions using mocktail.
class MockListener<T extends Object> extends Mock {
  /// Should be called by `ref.listen` and can then be verified using mocktail
  /// syntax.
  void call(T? previous, T next);
}

Expected behavior Calling container.pump() after adding a new value to the Stream should consider that new value. Furthermore, the way container.pump() is explained, calling it multiple times in a row should never be different from calling it once in my opinion.

[rrousselGit]

This is normal.

container.pump() waits for providers to rebuild and get disposed. It doesn't wait for Streams/Futures to complete if you have any pending task.

StreamController.add emits its value asynchronously. When you first call container.pump(), the value isn't emitted yet. So no provider rebuilds initially.
The second container.pump() works because due to the first await, your Stream finally emitted something, so a provider correctly has to rebuild now.

You can verify this quickly by changing:

-      streamController = StreamController.broadcast();
+      streamController = StreamController.broadcast(sync: true);

With this, both tests now pass because the emit is now synchronous.

So ultimately the issue isn't with container.pump, but your stream and streamController.add.