Add Pagination

[178inaba]

Related issue: #52

This PR provides pagination.

Usage:

var maxID *int64
var total []*mastodon.Account
for {
    accounts, pg, err := c.GetAccountFollowers(ctx, 12, &mastodon.Pagination{
        MaxID: maxID,
        Limit: mastodon.Int64(80),
    })
    if err != nil {
        log.Fatal(err)
    }

    total = append(total, accounts...)
    if pg.MaxID == nil {
        break
    }
    maxID = pg.MaxID
    time.Sleep(10 * time.Second)
}

fmt.Println(len(total))

[coveralls]

Coverage increased (+0.7%) to 83.626% when pulling dd0b467062ba3da72262e0dd537e1cef683139b8 on 178inaba:pagination into 759ce4a33160acab411081d5b6e9c856cfe11f7d on mattn:master.

[mattn]

Why you return new Pagenation?

[178inaba]

Because I wanted to be able to get pagination even if pagination is nil.

// Pagination is nil
total, pg, err := c.GetAccountFollowers(ctx, 12, nil)
if err != nil {
    log.Fatal(err)
}

maxID := pg.MaxID
for {
    accounts, pg, err := c.GetAccountFollowers(ctx, 12, &mastodon.Pagination{
        MaxID: maxID,
        Limit: mastodon.Int64(80),
    })
    if err != nil {
        log.Fatal(err)
    }

    total = append(total, accounts...)
    fmt.Println(len(total))
    if pg.MaxID == nil {
        break
    }
    maxID = pg.MaxID
    time.Sleep(10 * time.Second)
}

fmt.Println(len(total))

Do you like the following one?

var pg mastodon.Pagination
total, err := c.GetAccountFollowers(ctx, 12, &pg)
if err != nil {
    log.Fatal(err)
}

maxID := pg.MaxID

In this case, what kind of action should be taken if Pagination is nil?

[mattn]

I'm thinking below is enoguh to work for two use-cases.

Get followers (not all. the number of followers is depend on mastodon)

total, err := c.GetAccountFollowers(ctx, nil)
if err != nil {
    log.Fatal(err)
}

Get all followers

var pg mastodon.Pagination
for {
    accounts, err := c.GetAccountFollowers(ctx, &pg)
    if err != nil {
        log.Fatal(err)
    }

    total = append(total, accounts...)
    fmt.Println(len(total))
    if !pg.Next {
        break
    }
    time.Sleep(10 * time.Second)
}
fmt.Println(len(total))

This way should modify pg it-self.

[178inaba]

Thank you so much for review! I will start fixing!

[coveralls]

Coverage increased (+0.6%) to 83.568% when pulling 1c0e37928e581ceff697e88a1caf250b6951925a on 178inaba:pagination into 759ce4a33160acab411081d5b6e9c856cfe11f7d on mattn:master.

[178inaba]

Has completed! Please check when there is time!

[178inaba]

It is not possible to specify both max_id and since_id (When both are specified, the same value is repeatedly get) correction: The same value is not returned. An empty array will be returned.

Therefore, it is necessary to once extract the max_id and define the Pagination again.

Usage:

var maxID *int64
var total []*mastodon.Account
for {
    pg := mastodon.Pagination{
        MaxID: maxID,
        Limit: mastodon.Int64(80),
    }
    accounts, err := c.GetAccountFollowers(ctx, 12, &pg)
    if err != nil {
        log.Fatal(err)
    }

    total = append(total, accounts...)
    if pg.MaxID == nil {
        break
    }
    maxID = pg.MaxID
    time.Sleep(10 * time.Second)
}

fmt.Println(len(total))

[mattn]

MaxID/Limit should be greater than 0, I think. So those must not be pointer. Sorry I'm not sure.

[178inaba]

OK. I will fix it now!

[178inaba]

I think that SinceID is also larger than 0, so I will fix that.

[coveralls]

Coverage increased (+0.6%) to 83.557% when pulling f283f056710f363cb6ba99d2b5425f178fa93501 on 178inaba:pagination into 759ce4a33160acab411081d5b6e9c856cfe11f7d on mattn:master.

[178inaba]

Has completed! Please check when there is time!

[mattn]

I'm thinking pg should not be re-created. Right?

    var followers []*mastodon.Account
    pg := mastodon.Pagination{MaxID: 0}
    for {
        fs, err := client.GetAccountFollowers(context.Background(), account.ID, &pg)
        if err != nil {
            return err
        }
        followers = append(followers, fs...)
        if pg.MaxID == 0 {
            break
        }
        time.Sleep(10 * time.Second)
    }

[178inaba]

Thank you so much for review!

If both SinceID and MaxID are set, the empty value is returned each time and it does not end. So, I resetting it by substituting 0 for SinceID.

[coveralls]

Coverage increased (+3.06%) to 85.999% when pulling 9e07d8951e695a03a7852f20cfb51b2eefac0537 on 178inaba:pagination into 759ce4a33160acab411081d5b6e9c856cfe11f7d on mattn:master.

[mattn]

How about to add trick to ignore SinceID with MaxID like below?

if pg.MaxID > 0 && pg.SinceID > 0 {
    pg.SinceID = 0
}

[178inaba]

I think there is a use case that monitors the follower list and displays a new follower.

    var pg mastodon.Pagination
    for {
        fs, err := c.GetAccountFollowers(context.Background(), 1, &pg)
        if err != nil {
            log.Fatal(err)
        }
        if len(fs) > 0 {
            for _, f := range fs {
                // New followers.
                fmt.Println(f.DisplayName)
            }
        }
        pg.MaxID = 0
        time.Sleep(10 * time.Second)
    }

In this case, can not monitor the follower list without SinceID.

[mattn]

Yes, My idea should work both, I think. Also in your use-case too.

list all followers

var pg mastodon.Pagination
    for {
        fs, err := c.GetAccountFollowers(context.Background(), 1, &pg)
        if err != nil {
            log.Fatal(err)
        }
        if len(fs) > 0 {
            for _, f := range fs {
                // New followers.
                fmt.Println(f.DisplayName)
            }
        }
        //pg.MaxID = 0
        time.Sleep(10 * time.Second)
    }

find new followers

var pg mastodon.Pagination
    for {
        fs, err := c.GetAccountFollowers(context.Background(), 1, &pg)
        if err != nil {
            log.Fatal(err)
        }
        if len(fs) > 0 {
            for _, f := range fs {
                // New followers.
                fmt.Println(f.DisplayName)
            }
        }
        pg.MaxID = 0
        time.Sleep(10 * time.Second)
    }

[178inaba]

Ah! I was misunderstanding where to use tricks.

[coveralls]

Coverage increased (+3.1%) to 86.05% when pulling 403343617540f6180908679665e1c0e7ed96cbf5 on 178inaba:pagination into 759ce4a33160acab411081d5b6e9c856cfe11f7d on mattn:master.

[178inaba]

Fixed. A trick has been applied to the place where MaxID / SinceID is set to url.Values.

[mattn]

:+1: :+1: :+1: thanks for you great work!

[178inaba]

Likewise!