dscottboggs / mastodon-async

Mastodon Client for Rust
Other
34 stars 12 forks source link

fix: next/prev page handling in Page struct #99

Closed joshka closed 1 year ago

joshka commented 1 year ago

I had trouble getting paging to work in a situation where the user should be able to load an initial page and then later move to the next page and the previous pages. If the app hits the prev_page() call when there is no data in the previous page, it effectively is a dead end for the app. Instead, it should just be a noop.

(Commit message body) Improve the handling of next and previous pages in the Page struct by fixing a bug that caused the URLs to be set to None when a page with no data was loaded. This change allows for the next and previous page URLs to be preserved for future retrieval even when there are no results in the current page.

I tested this change using a small example app that I've been using to explore the mastodon-async functionality in depth (this could probably be used as a full example perhaps later, but for now it's just for proving out bits and pieces)

https://github.com/joshka/spike-mastodon/blob/fix-paging/src/main.rs#L188-L273

#[instrument(name = "home", skip_all, err)]
async fn show_timeline(client: &Mastodon) -> Result<()> {
    let mut timeline = load_home_timeline(client).await?;
    // log the initial page links
    log_page_links(&timeline);

    // intentionally load the previous page while we're at the first page to
    // check that the behavior of the page object doesn't dead-end at the
    // beginning. This should not fail, but it also should not update the page
    // links
    load_prev_page(&mut timeline).await?;
    // this should log the same as the initial page links
    log_page_links(&timeline);

    // moving to the next page should load the next page and update the page
    // links
    load_next_page(&mut timeline).await?;
    // this should log two different links
    log_page_links(&timeline);

    // this should move back to the initial page
    load_prev_page(&mut timeline).await?;
    // this should log the same as the initial page links
    log_page_links(&timeline);

    Ok(())
}

#[instrument(name = "initial", skip_all, err)]
async fn load_home_timeline(client: &Mastodon) -> Result<Page<Status>> {
    let timeline = client
        .get_home_timeline()
        .await
        .context("Couldn't get timeline")?;
    info!("loaded initial page of home timeline");
    for item in &timeline.initial_items {
        debug!(uri = %item.uri);
    }
    Ok(timeline)
}

#[instrument(name = "next_page", skip_all, err)]
async fn load_next_page(timeline: &mut Page<Status>) -> Result<()> {
    let url = timeline.next.clone().context("no next page")?;
    let page = timeline
        .next_page()
        .await
        .context("Couldn't get next page")?;
    info!(%url, "loaded next page");
    log_page_items(page);
    Ok(())
}

#[instrument(name = "prev_page", skip_all, err)]
async fn load_prev_page(timeline: &mut Page<Status>) -> Result<()> {
    let url = timeline.prev.clone().context("no prev page")?;
    let page = timeline
        .prev_page()
        .await
        .context("Couldn't get prev page")?;
    info!(%url, "loaded prev page");
    log_page_items(page);
    Ok(())
}

fn log_page_items(page: Option<Vec<Status>>) {
    page.map_or_else(
        || warn!("the page loaded successfully, but there is no data"),
        |items| {
            for item in items {
                debug!(uri = %item.uri);
            }
        },
    );
}

/// This exists because there was an issue with the way that the previous and
/// next pages were loaded when going to the previous page at the beginning or
/// the next page at the end.
fn log_page_links(page: &Page<Status>) {
    debug!(
        prev = page.prev.as_ref().map_or("None", |u| u.as_str()),
        next = page.next.as_ref().map_or("None", |u| u.as_str()),
        "page links"
    );
}

This loads a home timeline, tries to go to the previous link, then tries to go to the next (which cannot succeed as the links are now both set to none. Here's the log before the change:

❯ cargo run
    Finished dev [unoptimized + debuginfo] target(s) in 0.28s
     Running `target/debug/spike-mastodon`
2023-04-27T01:44:27.458954Z  INFO spike_mastodon: Starting spike-mastodon
2023-04-27T01:44:27.459779Z  INFO run:load_credentials:config_folder: spike_mastodon: return="/Users/joshka/Library/Application Support/com.joshka.mastodon-async"
2023-04-27T01:44:28.284045Z  INFO run:verify_credentials: spike_mastodon: verified credentials acct="joshka" id=109296336002994872 name="Joshka"
2023-04-27T01:44:29.484265Z  INFO run:home:initial: spike_mastodon: loaded initial page of home timeline
2023-04-27T01:44:29.484378Z DEBUG run:home:initial: spike_mastodon: uri=https://hachyderm.io/users/shanselman/statuses/110268232305525919
2023-04-27T01:44:29.484474Z DEBUG run:home:initial: spike_mastodon: uri=https://aus.social/users/nathanaelcbr/statuses/110268198159793027/activity
2023-04-27T01:44:29.484569Z DEBUG run:home:initial: spike_mastodon: uri=https://aus.social/users/nathanaelcbr/statuses/110268183063081696/activity
2023-04-27T01:44:29.484701Z DEBUG run:home:initial: spike_mastodon: uri=https://mastodon.online/users/batkaren/statuses/110268162586723454/activity
2023-04-27T01:44:29.484883Z DEBUG run:home:initial: spike_mastodon: uri=https://mastodon.online/users/batkaren/statuses/110268161987973821/activity
2023-04-27T01:44:29.485054Z DEBUG run:home:initial: spike_mastodon: uri=https://mastodon.social/users/mjec/statuses/110268135831916095/activity
2023-04-27T01:44:29.485214Z DEBUG run:home:initial: spike_mastodon: uri=https://infosec.exchange/users/hacks4pancakes/statuses/110268100444233223
2023-04-27T01:44:29.485417Z DEBUG run:home:initial: spike_mastodon: uri=https://infosec.exchange/users/taosecurity/statuses/110268098866351100
2023-04-27T01:44:29.485593Z DEBUG run:home:initial: spike_mastodon: uri=https://hachyderm.io/users/molly0xfff/statuses/110268095363254922
2023-04-27T01:44:29.485784Z DEBUG run:home:initial: spike_mastodon: uri=https://eigenmagic.net/users/NewtonMark/statuses/110268085544910153/activity
2023-04-27T01:44:29.485949Z DEBUG run:home:initial: spike_mastodon: uri=https://aus.social/users/nathanaelcbr/statuses/110268084022167418
2023-04-27T01:44:29.486074Z DEBUG run:home:initial: spike_mastodon: uri=https://eigenmagic.net/users/abstractcode/statuses/110268078820158963
2023-04-27T01:44:29.486172Z DEBUG run:home:initial: spike_mastodon: uri=https://federate.social/users/mattblaze/statuses/110268013883676186
2023-04-27T01:44:29.486268Z DEBUG run:home:initial: spike_mastodon: uri=https://mas.to/users/carnage4life/statuses/110268019600692290
2023-04-27T01:44:29.486412Z DEBUG run:home:initial: spike_mastodon: uri=https://eigenmagic.net/users/NewtonMark/statuses/110268019494286269/activity
2023-04-27T01:44:29.486555Z DEBUG run:home:initial: spike_mastodon: uri=https://me.dm/users/anildash/statuses/110268002238353609/activity
2023-04-27T01:44:29.486698Z DEBUG run:home:initial: spike_mastodon: uri=https://defcon.social/users/deviantollam/statuses/110267998737064533/activity
2023-04-27T01:44:29.486840Z DEBUG run:home:initial: spike_mastodon: uri=https://me.dm/users/anildash/statuses/110267997627913442/activity
2023-04-27T01:44:29.486982Z DEBUG run:home:initial: spike_mastodon: uri=https://defcon.social/users/deviantollam/statuses/110267996232196550/activity
2023-04-27T01:44:29.487133Z DEBUG run:home:initial: spike_mastodon: uri=https://defcon.social/users/deviantollam/statuses/110267995142677575/activity
2023-04-27T01:44:29.487344Z DEBUG run:home: spike_mastodon: page links prev="https://hachyderm.io/api/v1/timelines/home?min_id=110268232305525919" next="https://hachyderm.io/api/v1/timelines/home?max_id=110267995499635406"
2023-04-27T01:44:30.124103Z  INFO run:home:prev_page: spike_mastodon: loaded prev page url=https://hachyderm.io/api/v1/timelines/home?min_id=110268232305525919
2023-04-27T01:44:30.124343Z DEBUG run:home: spike_mastodon: page links prev="None" next="None"
2023-04-27T01:44:30.124635Z ERROR run:home:next_page: spike_mastodon: error=no next page
2023-04-27T01:44:30.125162Z ERROR run:home: spike_mastodon: error=no next page
2023-04-27T01:44:30.125571Z ERROR run: spike_mastodon: error=no next page
2023-04-27T01:44:30.125929Z ERROR spike_mastodon: error err=no next page

(Note I made the next/prev urls public for the purposes of this test)

And after this change:

❯ cargo run
    Blocking waiting for file lock on build directory
    Finished dev [unoptimized + debuginfo] target(s) in 1.93s
     Running `target/debug/spike-mastodon`
2023-04-27T01:55:21.849410Z  INFO spike_mastodon: Starting spike-mastodon
2023-04-27T01:55:21.850522Z  INFO run:load_credentials:config_folder: spike_mastodon: return="/Users/joshka/Library/Application Support/com.joshka.mastodon-async"
2023-04-27T01:55:22.444006Z  INFO run:verify_credentials: spike_mastodon: verified credentials acct="joshka" id=109296336002994872 name="Joshka"
2023-04-27T01:55:23.541082Z  INFO run:home:initial: spike_mastodon: loaded initial page of home timeline
2023-04-27T01:55:23.541189Z DEBUG run:home:initial: spike_mastodon: uri=https://infosec.exchange/users/mttaggart/statuses/110268275348023071
2023-04-27T01:55:23.541322Z DEBUG run:home:initial: spike_mastodon: uri=https://hachyderm.io/users/shanselman/statuses/110268232305525919
2023-04-27T01:55:23.541461Z DEBUG run:home:initial: spike_mastodon: uri=https://aus.social/users/nathanaelcbr/statuses/110268198159793027/activity
2023-04-27T01:55:23.541648Z DEBUG run:home:initial: spike_mastodon: uri=https://aus.social/users/nathanaelcbr/statuses/110268183063081696/activity
2023-04-27T01:55:23.541782Z DEBUG run:home:initial: spike_mastodon: uri=https://mastodon.online/users/batkaren/statuses/110268162586723454/activity
2023-04-27T01:55:23.541910Z DEBUG run:home:initial: spike_mastodon: uri=https://mastodon.online/users/batkaren/statuses/110268161987973821/activity
2023-04-27T01:55:23.542043Z DEBUG run:home:initial: spike_mastodon: uri=https://mastodon.social/users/mjec/statuses/110268135831916095/activity
2023-04-27T01:55:23.542213Z DEBUG run:home:initial: spike_mastodon: uri=https://infosec.exchange/users/hacks4pancakes/statuses/110268100444233223
2023-04-27T01:55:23.542365Z DEBUG run:home:initial: spike_mastodon: uri=https://infosec.exchange/users/taosecurity/statuses/110268098866351100
2023-04-27T01:55:23.542584Z DEBUG run:home:initial: spike_mastodon: uri=https://hachyderm.io/users/molly0xfff/statuses/110268095363254922
2023-04-27T01:55:23.542814Z DEBUG run:home:initial: spike_mastodon: uri=https://eigenmagic.net/users/NewtonMark/statuses/110268085544910153/activity
2023-04-27T01:55:23.542946Z DEBUG run:home:initial: spike_mastodon: uri=https://aus.social/users/nathanaelcbr/statuses/110268084022167418
2023-04-27T01:55:23.543102Z DEBUG run:home:initial: spike_mastodon: uri=https://eigenmagic.net/users/abstractcode/statuses/110268078820158963
2023-04-27T01:55:23.543250Z DEBUG run:home:initial: spike_mastodon: uri=https://federate.social/users/mattblaze/statuses/110268013883676186
2023-04-27T01:55:23.543396Z DEBUG run:home:initial: spike_mastodon: uri=https://mas.to/users/carnage4life/statuses/110268019600692290
2023-04-27T01:55:23.543547Z DEBUG run:home:initial: spike_mastodon: uri=https://eigenmagic.net/users/NewtonMark/statuses/110268019494286269/activity
2023-04-27T01:55:23.543696Z DEBUG run:home:initial: spike_mastodon: uri=https://me.dm/users/anildash/statuses/110268002238353609/activity
2023-04-27T01:55:23.543851Z DEBUG run:home:initial: spike_mastodon: uri=https://defcon.social/users/deviantollam/statuses/110267998737064533/activity
2023-04-27T01:55:23.544041Z DEBUG run:home:initial: spike_mastodon: uri=https://me.dm/users/anildash/statuses/110267997627913442/activity
2023-04-27T01:55:23.544204Z DEBUG run:home:initial: spike_mastodon: uri=https://defcon.social/users/deviantollam/statuses/110267996232196550/activity
2023-04-27T01:55:23.544391Z DEBUG run:home: spike_mastodon: page links prev="https://hachyderm.io/api/v1/timelines/home?min_id=110268275368557316" next="https://hachyderm.io/api/v1/timelines/home?max_id=110267996439747983"
2023-04-27T01:55:23.942326Z  INFO run:home:prev_page: spike_mastodon: loaded prev page url=https://hachyderm.io/api/v1/timelines/home?min_id=110268275368557316
2023-04-27T01:55:23.942486Z  WARN run:home:prev_page: spike_mastodon: the page loaded successfully, but there is no data
2023-04-27T01:55:23.942703Z DEBUG run:home: spike_mastodon: page links prev="https://hachyderm.io/api/v1/timelines/home?min_id=110268275368557316" next="https://hachyderm.io/api/v1/timelines/home?max_id=110267996439747983"
2023-04-27T01:55:25.145491Z  INFO run:home:next_page: spike_mastodon: loaded next page url=https://hachyderm.io/api/v1/timelines/home?max_id=110267996439747983
2023-04-27T01:55:25.145637Z DEBUG run:home:next_page: spike_mastodon: uri=https://defcon.social/users/deviantollam/statuses/110267995142677575/activity
2023-04-27T01:55:25.145730Z DEBUG run:home:next_page: spike_mastodon: uri=https://me.dm/users/anildash/statuses/110267993325024162
2023-04-27T01:55:25.145810Z DEBUG run:home:next_page: spike_mastodon: uri=https://eigenmagic.net/users/NewtonMark/statuses/110267992752364002
2023-04-27T01:55:25.145889Z DEBUG run:home:next_page: spike_mastodon: uri=https://hachyderm.io/users/molly0xfff/statuses/110267962281429565
2023-04-27T01:55:25.151031Z DEBUG run:home:next_page: spike_mastodon: uri=https://hachyderm.io/users/molly0xfff/statuses/110267960470755209
2023-04-27T01:55:25.151198Z DEBUG run:home:next_page: spike_mastodon: uri=https://hachyderm.io/users/molly0xfff/statuses/110267958305946394
2023-04-27T01:55:25.151392Z DEBUG run:home:next_page: spike_mastodon: uri=https://hachyderm.io/users/molly0xfff/statuses/110267954441141833
2023-04-27T01:55:25.151548Z DEBUG run:home:next_page: spike_mastodon: uri=https://eigenmagic.net/users/stilgherrian/statuses/110267943499722846
2023-04-27T01:55:25.151639Z DEBUG run:home:next_page: spike_mastodon: uri=https://mastodon.social/users/jwz/statuses/110267938513538131/activity
2023-04-27T01:55:25.151765Z DEBUG run:home:next_page: spike_mastodon: uri=https://infosec.exchange/users/malwaretech/statuses/110267927471977064
2023-04-27T01:55:25.151967Z DEBUG run:home:next_page: spike_mastodon: uri=https://mastodon.social/users/jwz/statuses/110267925975733898
2023-04-27T01:55:25.152199Z DEBUG run:home:next_page: spike_mastodon: uri=https://mastodon.social/users/kocienda/statuses/110267906318316430
2023-04-27T01:55:25.152366Z DEBUG run:home:next_page: spike_mastodon: uri=https://mastodon.social/users/kocienda/statuses/110267898124685259
2023-04-27T01:55:25.152568Z DEBUG run:home:next_page: spike_mastodon: uri=https://mastodon.social/users/VaughnVernon/statuses/110267854632882750
2023-04-27T01:55:25.152718Z DEBUG run:home:next_page: spike_mastodon: uri=https://ohai.social/users/dr2chase/statuses/110267800556449295/activity
2023-04-27T01:55:25.152892Z DEBUG run:home:next_page: spike_mastodon: uri=https://infosec.exchange/users/mttaggart/statuses/110267782984070348/activity
2023-04-27T01:55:25.152986Z DEBUG run:home:next_page: spike_mastodon: uri=https://mastodon.social/users/rust_discussions/statuses/110267780722806492/activity
2023-04-27T01:55:25.153118Z DEBUG run:home:next_page: spike_mastodon: uri=https://infosec.exchange/users/mttaggart/statuses/110267777386715369
2023-04-27T01:55:25.153297Z DEBUG run:home:next_page: spike_mastodon: uri=https://hachyderm.io/users/shanselman/statuses/110267749484262117
2023-04-27T01:55:25.153490Z DEBUG run:home: spike_mastodon: page links prev="https://hachyderm.io/api/v1/timelines/home?min_id=110267995499635406" next="https://hachyderm.io/api/v1/timelines/home?max_id=110267749484262117"
2023-04-27T01:55:26.490659Z  INFO run:home:prev_page: spike_mastodon: loaded prev page url=https://hachyderm.io/api/v1/timelines/home?min_id=110267995499635406
2023-04-27T01:55:26.490753Z DEBUG run:home:prev_page: spike_mastodon: uri=https://infosec.exchange/users/mttaggart/statuses/110268275348023071
2023-04-27T01:55:26.490840Z DEBUG run:home:prev_page: spike_mastodon: uri=https://hachyderm.io/users/shanselman/statuses/110268232305525919
2023-04-27T01:55:26.490927Z DEBUG run:home:prev_page: spike_mastodon: uri=https://aus.social/users/nathanaelcbr/statuses/110268198159793027/activity
2023-04-27T01:55:26.491057Z DEBUG run:home:prev_page: spike_mastodon: uri=https://aus.social/users/nathanaelcbr/statuses/110268183063081696/activity
2023-04-27T01:55:26.491272Z DEBUG run:home:prev_page: spike_mastodon: uri=https://mastodon.online/users/batkaren/statuses/110268162586723454/activity
2023-04-27T01:55:26.491433Z DEBUG run:home:prev_page: spike_mastodon: uri=https://mastodon.online/users/batkaren/statuses/110268161987973821/activity
2023-04-27T01:55:26.491584Z DEBUG run:home:prev_page: spike_mastodon: uri=https://mastodon.social/users/mjec/statuses/110268135831916095/activity
2023-04-27T01:55:26.491744Z DEBUG run:home:prev_page: spike_mastodon: uri=https://infosec.exchange/users/hacks4pancakes/statuses/110268100444233223
2023-04-27T01:55:26.491883Z DEBUG run:home:prev_page: spike_mastodon: uri=https://infosec.exchange/users/taosecurity/statuses/110268098866351100
2023-04-27T01:55:26.492030Z DEBUG run:home:prev_page: spike_mastodon: uri=https://hachyderm.io/users/molly0xfff/statuses/110268095363254922
2023-04-27T01:55:26.492144Z DEBUG run:home:prev_page: spike_mastodon: uri=https://eigenmagic.net/users/NewtonMark/statuses/110268085544910153/activity
2023-04-27T01:55:26.492311Z DEBUG run:home:prev_page: spike_mastodon: uri=https://aus.social/users/nathanaelcbr/statuses/110268084022167418
2023-04-27T01:55:26.492458Z DEBUG run:home:prev_page: spike_mastodon: uri=https://eigenmagic.net/users/abstractcode/statuses/110268078820158963
2023-04-27T01:55:26.492598Z DEBUG run:home:prev_page: spike_mastodon: uri=https://federate.social/users/mattblaze/statuses/110268013883676186
2023-04-27T01:55:26.492690Z DEBUG run:home:prev_page: spike_mastodon: uri=https://mas.to/users/carnage4life/statuses/110268019600692290
2023-04-27T01:55:26.492779Z DEBUG run:home:prev_page: spike_mastodon: uri=https://eigenmagic.net/users/NewtonMark/statuses/110268019494286269/activity
2023-04-27T01:55:26.492884Z DEBUG run:home:prev_page: spike_mastodon: uri=https://me.dm/users/anildash/statuses/110268002238353609/activity
2023-04-27T01:55:26.493010Z DEBUG run:home:prev_page: spike_mastodon: uri=https://defcon.social/users/deviantollam/statuses/110267998737064533/activity
2023-04-27T01:55:26.493129Z DEBUG run:home:prev_page: spike_mastodon: uri=https://me.dm/users/anildash/statuses/110267997627913442/activity
2023-04-27T01:55:26.493289Z DEBUG run:home:prev_page: spike_mastodon: uri=https://defcon.social/users/deviantollam/statuses/110267996232196550/activity
2023-04-27T01:55:26.493470Z DEBUG run:home: spike_mastodon: page links prev="https://hachyderm.io/api/v1/timelines/home?min_id=110268275368557316" next="https://hachyderm.io/api/v1/timelines/home?max_id=110267996439747983"