RemoteMediator PREPEND is getting called multiple times when not needed

[vlad-dialpad]

I'm facing an issue with paging library requesting 'too much' data when requesting the information from the middle of the 'data source': For my scenario I can request both previous and next pages (Remote mediator load types PREPEND and APPEND). After initial portion (Remote mediator loadType REFRESH) and one APPEND and one PREPEND (as far as I can see from the source code paging lib will always request one page before and one page after after initial REFRESH) the library call mediator's load with 'PREPEND' loadType 1-3 more times, doesn't matter how big my page (and initial load) is, 30 or 300 elements. So my network calls sequence looks like

---> REFRESH ---> PREPEND ---> APPEND ---> PREPEND ---> PREPEND

When I'm scrolling and reaching boundary conditions, paging lib often requests 2-3 pages, instead of 1.

This all happens only when both PREVIOUS and NEXT pages are available. If user can scroll one direction only, most of the time, only 1 page is requested when user hits boundary condition.

Is it possible to control how many pages does paging library request (in order to reduce payload on the backend)

For the testing purposes, I modified this sample app to get 'endless stream of network data': https://github.com/android/architecture-components-samples/tree/master/PagingWithNetworkSample

@OptIn(ExperimentalPagingApi::class)
class PageKeyedRemoteMediator(
    private val db: RedditDb,
    private val redditApi: RedditApi,
    private val subredditName: String
) : RemoteMediator<Int, RedditPost>() {
    private val postDao: RedditPostDao = db.posts()
    private val remoteKeyDao: SubredditRemoteKeyDao = db.remoteKeys()

    var minIndex = 0
    var maxIndex = 0

    override suspend fun initialize(): InitializeAction {
        // Require that remote REFRESH is launched on initial load and succeeds before launching
        // remote PREPEND / APPEND.
        return InitializeAction.LAUNCH_INITIAL_REFRESH
    }

    override suspend fun load(
            loadType: LoadType,
            state: PagingState<Int, RedditPost>
    ): MediatorResult {
        try {
            delay(1000)
            val size = if (loadType == REFRESH) state.config.initialLoadSize else state.config.pageSize

            println(">>>>>    $loadType: ${if (loadType == APPEND) minIndex else maxIndex}")

            val items = (0 until size).map {
                val value = if (loadType == APPEND) minIndex-- else maxIndex++
                RedditPost(
                        name = value.toString(),
                        title = value.toString(),
                        score = value,
                        author = value.toString(),
                        subreddit = subredditName,
                        num_comments = value,
                        created = System.currentTimeMillis(),
                        thumbnail = null,
                        url = null
                ).apply {
                    indexInResponse = value
                }
            }

            db.withTransaction {
                if (loadType == REFRESH) {
                    postDao.deleteBySubreddit(subredditName)
                }

                postDao.insertAll(items)
            }

            return MediatorResult.Success(endOfPaginationReached = items.isEmpty())
        } catch (e: IOException) {
            return MediatorResult.Error(e)
        } catch (e: HttpException) {
            return MediatorResult.Error(e)
        }
    }
}

@Dao
interface RedditPostDao {
    ...
    @Query("SELECT * FROM posts WHERE subreddit = :subreddit ORDER BY num_comments DESC")
    fun postsBySubreddit(subreddit: String): PagingSource<Int, RedditPost>
    ...
}

    ...
    override fun postsOfSubreddit(subReddit: String, pageSize: Int) = Pager(
        config = PagingConfig(pageSize),
        remoteMediator = PageKeyedRemoteMediator(db, redditApi, subReddit)
    ) {
        db.posts().postsBySubreddit(subReddit)
    }.flow
    ...
    @OptIn(ExperimentalCoroutinesApi::class, FlowPreview::class)
    val posts = flowOf(
        clearListCh.receiveAsFlow().map { PagingData.empty<RedditPost>() },
        savedStateHandle.getLiveData<String>(KEY_SUBREDDIT)
            .asFlow()
            .flatMapLatest { repository.postsOfSubreddit(it, 100) }
            // cachedIn() shares the paging state across multiple consumers of posts,
            // e.g. different generations of UI across rotation config change
            .cachedIn(viewModelScope)
    ).flattenMerge(2)
    ...
    lifecycleScope.launchWhenCreated {
       model.posts.collectLatest {
           adapter.submitData(it)
        }
    }

Paging infra setup:

1. Local db + Network (RemoteMediator)
2. Room at persistence layer
3. Coroutines
4. Libs versions:
   • paging = "3.1.0-rc01"
   • room = "2.4.0-beta01"
   • kotlin = "1.4.21"
   • recyclerview = "1.2.1"