search

CXuesong / WikiClientLibrary

/*🌻*/ Wiki Client Library is an asynchronous MediaWiki API client library targeting modern .NET platforms
https://github.com/CXuesong/WikiClientLibrary/wiki
Apache License 2.0
80 stars 16 forks source link

Failed to enumerate category members on enwarriorswiki:Category:Spotfur's Rebellion #88

Open CXuesong opened 3 years ago

CXuesong commented 3 years ago

Example code:

[Fact]
public async Task Test1()
{
    var client = new WikiClient();
    var site = new WikiSite(client, "https://warriors.fandom.com/api.php");
    await site.Initialization;

    var result = await new CategoryMembersGenerator(site, "Category:Spotfur's Rebellion") { PaginationSize = 50, }.EnumPagesAsync().Take(100).ToListAsync();
}

Failure


System.ArgumentNullException
Value cannot be null. (Parameter 'value')
   at Newtonsoft.Json.Linq.JToken.EnsureValue(JToken value)
   at Newtonsoft.Json.Linq.JToken.op_Explicit(JToken value)
   at WikiClientLibrary.Pages.Queries.Properties.PageInfoPropertyGroup..ctor(JObject jPage) in ~\WikiClientLibrary\WikiClientLibrary\Pages\Queries\Properties\PageInfoPropertyProvider.cs:line 54
   at WikiClientLibrary.Pages.Queries.Properties.PageInfoPropertyProvider.ParsePropertyGroup(JObject json) in ~\WikiClientLibrary\WikiClientLibrary\Pages\Queries\Properties\PageInfoPropertyProvider.cs:line 28
   at WikiClientLibrary.Pages.Queries.WikiPageQueryProvider.ParsePropertyGroups(JObject json)+MoveNext() in ~\WikiClientLibrary\WikiClientLibrary\Pages\Queries\WikiPageQueryProvider.cs:line 150
   at WikiClientLibrary.Pages.WikiPage.OnLoadPageInfo(JObject jpage, IWikiPageQueryProvider options) in ~\WikiClientLibrary\WikiClientLibrary\Pages\WikiPage.cs:line 324
   at WikiClientLibrary.Infrastructures.MediaWikiHelper.PopulatePageFromJson(WikiPage page, JObject json, IWikiPageQueryProvider options) in ~\WikiClientLibrary\WikiClientLibrary\Infrastructures\MediaWikiHelper.cs:line 356
   at WikiClientLibrary.Pages.WikiPage.<>c__DisplayClass0_0.<FromJsonQueryResult>b__1(JProperty page) in ~\WikiClientLibrary\WikiClientLibrary\Pages\PageFactory.cs:line 35
   at System.Linq.Enumerable.SelectIPartitionIterator`2.ToList()
   at System.Linq.Enumerable.ToList[TSource](IEnumerable`1 source)
   at WikiClientLibrary.Pages.WikiPage.FromJsonQueryResult(WikiSite site, JObject jpages, IWikiPageQueryProvider options) in ~\WikiClientLibrary\WikiClientLibrary\Pages\PageFactory.cs:line 31
   at WikiClientLibrary.Generators.Primitive.WikiPageGenerator`1.<>c__DisplayClass8_0.<EnumPagesAsync>b__1(JObject jquery) in ~\WikiClientLibrary\WikiClientLibrary\Generators\Primitive\WikiPageGenerator.cs:line 117
   at System.Linq.AsyncEnumerable.SelectManyAsyncIterator`2.MoveNextCore() in /_/Ix.NET/Source/System.Linq.Async/System/Linq/Operators/SelectMany.cs:line 463
   at System.Linq.AsyncIteratorBase`1.MoveNextAsync() in /_/Ix.NET/Source/System.Linq.Async/System/Linq/AsyncIterator.cs:line 70
   at System.Linq.AsyncIteratorBase`1.MoveNextAsync() in /_/Ix.NET/Source/System.Linq.Async/System/Linq/AsyncIterator.cs:line 75
   at System.Linq.AsyncEnumerablePartition`1.ToListAsync(CancellationToken cancellationToken) in /_/Ix.NET/Source/System.Linq.Async/System/Linq/AsyncEnumerablePartition.cs:line 348
   at System.Linq.AsyncEnumerablePartition`1.ToListAsync(CancellationToken cancellationToken) in /_/Ix.NET/Source/System.Linq.Async/System/Linq/AsyncEnumerablePartition.cs:line 353
   at WikiClientLibrary.Tests.UnitTestProject1.Playground.Test1() in ~\WikiClientLibrary\UnitTestProject1\_private\Playground.cs:line 53
   at Xunit.Sdk.TestInvoker`1.<>c__DisplayClass48_1.<<InvokeTestMethodAsync>b__1>d.MoveNext() in C:\Dev\xunit\xunit\src\xunit.execution\Sdk\Frameworks\Runners\TestInvoker.cs:line 264
--- End of stack trace from previous location where exception was thrown ---
   at Xunit.Sdk.ExecutionTimer.AggregateAsync(Func`1 asyncAction) in C:\Dev\xunit\xunit\src\xunit.execution\Sdk\Frameworks\ExecutionTimer.cs:line 48
   at Xunit.Sdk.ExceptionAggregator.RunAsync(Func`1 code) in C:\Dev\xunit\xunit\src\xunit.core\Sdk\ExceptionAggregator.cs:line 90
CXuesong commented 3 years ago

This is caused by a file (title=File:NO-20.jpg) in the first batch of pages being split into 2 parts. and the first continuation returns something that WikiPageGenerator.EnumPagesAsync didn't expect.

1st request

POST https://warriors.fandom.com/api.php HTTP/1.1

format=json&action=query&maxlag=5&inprop=protection&rvprop=ids%7ctimestamp%7cflags%7ccomment%7cuser%7cuserid%7ccontentmodel%7csha1%7ctags%7csize&rvslots=main&iiprop=timestamp%7cuser%7ccomment%7curl%7csize%7csha1&prop=info%7crevisions%7ccategoryinfo%7cpageprops%7cimageinfo&generator=categorymembers&gcmtitle=Category%3aSpotfur%27s+Rebellion&gcmlimit=50&gcmtype=page%7csubcat%7cfile

response

{
    "continue": {
        "iistart": "2020-03-08T03:16:46Z",
        "continue": "||info|revisions|categoryinfo|pageprops"
    },
    "query": {
        "pages": {
            "145387": {
                "pageid": 145387,
                "ns": 0,
                "title": "Spotfur's Rebellion",
                "contentmodel": "wikitext",
                "pagelanguage": "en",
                "pagelanguagehtmlcode": "en",
                "pagelanguagedir": "ltr",
                "touched": "2021-06-12T08:43:04Z",
                "lastrevid": 1573944,
                "length": 2978,
                "protection": [],
                "restrictiontypes": ["edit", "move"],
                "revisions": [
                    {
                        "revid": 1573944,
                        "parentid": 1573942,
                        "minor": "",
                        "user": "Vector Sigma",
                        "userid": 27042031,
                        "timestamp": "2021-06-01T13:40:18Z",
                        "size": 2978,
                        "sha1": "5b46e32ddb57d627993e54a0dd537deceb146e01",
                        "slots": {
                            "main": {
                                "contentmodel": "wikitext"
                            }
                        },
                        "comment": "Reverted edits by [[Special:Contributions/Aroundtheworldguy|Aroundtheworldguy]] ([[User talk:Aroundtheworldguy|talk]]) to last revision by [[User:Diablo.|Diablo.]]",
                        "tags": ["mw-rollback"]
                    }
                ], "pageprops": {
                    "displaytitle": "<i>Spotfur&#39;s Rebellion</i>",
                    "infoboxes": "[{\"parser_tag_version\":2,\"data\":[{\"type\":\"title\",\"data\":{\"value\":\"<i>Spotfur&#39;s Rebellion<\\/i>\",\"item-name\":null,\"source\":\"name\"}},{\"type\":\"image\",\"data\":[{\"url\":\"https:\\/\\/static.wikia.nocookie.net\\/warriors\\/images\\/d\\/da\\/NOC-7.jpg\\/revision\\/latest?cb=20201202031148\",\"name\":\"NOC-7.jpg\",\"key\":\"NOC-7.jpg\",\"alt\":\"Front\",\"caption\":\"Front\",\"isVideo\":false,\"item-name\":null,\"source\":null},{\"url\":\"https:\\/\\/static.wikia.nocookie.net\\/warriors\\/images\\/0\\/06\\/BK-NOC-7.png\\/revision\\/latest?cb=20210331035558\",\"name\":\"BK-NOC-7.png\",\"key\":\"BK-NOC-7.png\",\"alt\":\"Back\",\"caption\":\"Back\",\"isVideo\":false,\"item-name\":null,\"source\":null}]},{\"type\":\"group\",\"data\":{\"value\":[{\"type\":\"header\",\"data\":{\"value\":\"General\",\"item-name\":null,\"source\":null}},{\"type\":\"data\",\"data\":{\"label\":\"Author:\",\"value\":\"<!--LINK'\\\" 0:0--><sup id=\\\"cite_ref-srded_1-0\\\" class=\\\"reference\\\"><a href=\\\"#cite_note-srded-1\\\">&#91;1&#93;<\\/a><\\/sup>\",\"span\":1,\"layout\":null,\"item-name\":null,\"source\":\"author\"}},{\"type\":\"data\",\"data\":{\"label\":\"Cover artist:\",\"value\":\"<!--LINK'\\\" 0:5--><sup id=\\\"cite_ref-choicebcover_2-0\\\" class=\\\"reference\\\"><a href=\\\"#cite_note-choicebcover-2\\\">&#91;2&#93;<\\/a><\\/sup>\",\"span\":1,\"layout\":null,\"item-name\":null,\"source\":\"cover artist\"}},{\"type\":\"data\",\"data\":{\"label\":\"Audiobook narrator:\",\"value\":\"<!--LINK'\\\" 0:7--><sup id=\\\"cite_ref-3\\\" class=\\\"reference\\\"><a href=\\\"#cite_note-3\\\">&#91;3&#93;<\\/a><\\/sup>\",\"span\":1,\"layout\":null,\"item-name\":null,\"source\":\"audiobook narrator\"}}],\"layout\":\"default\",\"collapse\":null,\"row-items\":null,\"item-name\":null}},{\"type\":\"group\",\"data\":{\"value\":[{\"type\":\"header\",\"data\":{\"value\":\"Details\",\"item-name\":null,\"source\":null}},{\"type\":\"data\",\"data\":{\"label\":\"Publish date:\",\"value\":\"6 <!--LINK'\\\" 0:1--><sup id=\\\"cite_ref-edelweiss_4-0\\\" class=\\\"reference\\\"><a href=\\\"#cite_note-edelweiss-4\\\">&#91;4&#93;<\\/a><\\/sup>\",\"span\":1,\"layout\":null,\"item-name\":null,\"source\":\"publish date\"}},{\"type\":\"data\",\"data\":{\"label\":\"Identification:\",\"value\":\"9780062857446\",\"span\":1,\"layout\":null,\"item-name\":null,\"source\":\"isbn\"}},{\"type\":\"data\",\"data\":{\"label\":\"Editions:\",\"value\":\"Paperback, E-book, audiobook\",\"span\":1,\"layout\":null,\"item-name\":null,\"source\":\"editions\"}}],\"layout\":\"default\",\"collapse\":null,\"row-items\":null,\"item-name\":null}},{\"type\":\"group\",\"data\":{\"value\":[{\"type\":\"header\",\"data\":{\"value\":\"Summary\",\"item-name\":null,\"source\":null}},{\"type\":\"data\",\"data\":{\"label\":\"\",\"value\":\"<i>What made Spotfur risk everything to defy her Clan's deputy?<\\/i>\",\"span\":1,\"layout\":null,\"item-name\":null,\"source\":\"summary\"}}],\"layout\":\"default\",\"collapse\":null,\"row-items\":null,\"item-name\":null}},{\"type\":\"group\",\"data\":{\"value\":[{\"type\":\"header\",\"data\":{\"value\":\"Chapter subpages\",\"item-name\":null,\"source\":null}},{\"type\":\"navigation\",\"data\":{\"value\":\"<p style=\\\"padding-left:10px; padding-right:10px; text-align:center;\\\"><\\/p>\",\"item-name\":null,\"source\":null}}],\"layout\":\"default\",\"collapse\":null,\"row-items\":null,\"item-name\":null}},{\"type\":\"group\",\"data\":{\"value\":[{\"type\":\"header\",\"data\":{\"value\":\"Chronology\",\"item-name\":null,\"source\":null}},{\"type\":\"data\",\"data\":{\"label\":\"Preceded by:\",\"value\":\"<i><!--LINK'\\\" 0:3--><\\/i>\",\"span\":1,\"layout\":null,\"item-name\":null,\"source\":\"preceded\"}},{\"type\":\"data\",\"data\":{\"label\":\"Concurrent with:\",\"value\":\"<i><!--LINK'\\\" 0:11--><\\/i>, <i><!--LINK'\\\" 0:12--><\\/i>, <i><!--LINK'\\\" 0:13--><\\/i>\",\"span\":1,\"layout\":null,\"item-name\":null,\"source\":\"concurrent\"}},{\"type\":\"data\",\"data\":{\"label\":\"Followed by:\",\"value\":\"<i><!--LINK'\\\" 0:15--><\\/i>\",\"span\":1,\"layout\":null,\"item-name\":null,\"source\":\"followed\"}}],\"layout\":\"default\",\"collapse\":null,\"row-items\":null,\"item-name\":null}}],\"metadata\":[{\"type\":\"title\",\"sources\":{\"name\":{\"label\":\"\",\"primary\":true}}},{\"type\":\"image\",\"sources\":{\"image2\":{\"label\":\"\"},\"image\":{\"label\":\"\"},\"option1\":{\"label\":\"\"},\"option2\":{\"label\":\"\"}}},{\"type\":\"group\",\"metadata\":[{\"type\":\"header\",\"sources\":[]},{\"type\":\"data\",\"sources\":{\"author\":{\"label\":\"Author:\",\"primary\":true}}},{\"type\":\"data\",\"sources\":{\"artist\":{\"label\":\"Artist:\",\"primary\":true}}},{\"type\":\"data\",\"sources\":{\"cover artist\":{\"label\":\"Cover artist:\",\"primary\":true}}},{\"type\":\"data\",\"sources\":{\"jacket designer\":{\"label\":\"Jacket designer:\",\"primary\":true}}},{\"type\":\"data\",\"sources\":{\"cover designer\":{\"label\":\"Cover designer:\",\"primary\":true}}},{\"type\":\"data\",\"sources\":{\"audiobook narrator\":{\"label\":\"Audiobook narrator:\",\"primary\":true}}}]},{\"type\":\"group\",\"metadata\":[{\"type\":\"header\",\"sources\":[]},{\"type\":\"data\",\"sources\":{\"publish date\":{\"label\":\"Publish date:\",\"primary\":true}}},{\"type\":\"data\",\"sources\":{\"isbn\":{\"label\":\"Identification:\",\"primary\":true}}},{\"type\":\"data\",\"sources\":{\"editions\":{\"label\":\"Editions:\",\"primary\":true}}}]},{\"type\":\"group\",\"metadata\":[{\"type\":\"header\",\"sources\":[]},{\"type\":\"data\",\"sources\":{\"summary\":{\"label\":\"\",\"primary\":true}}}]},{\"type\":\"group\",\"metadata\":[{\"type\":\"header\",\"sources\":[]},{\"type\":\"navigation\",\"sources\":[]}]},{\"type\":\"group\",\"metadata\":[{\"type\":\"header\",\"sources\":[]},{\"type\":\"navigation\",\"sources\":[]}]},{\"type\":\"group\",\"metadata\":[{\"type\":\"header\",\"sources\":[]},{\"type\":\"data\",\"sources\":{\"preceded\":{\"label\":\"Preceded by:\",\"primary\":true}}},{\"type\":\"data\",\"sources\":{\"concurrent\":{\"label\":\"Concurrent with:\",\"primary\":true}}},{\"type\":\"data\",\"sources\":{\"followed\":{\"label\":\"Followed by:\",\"primary\":true}}}]}]}]"
                }
            },
            "152171": {
                "pageid": 152171,
                "ns": 0,
                "title": "Spotfur's Rebellion/Allegiances",
                "contentmodel": "wikitext",
                "pagelanguage": "en",
                "pagelanguagehtmlcode": "en",
                "pagelanguagedir": "ltr",
                "touched": "2021-06-07T00:43:12Z",
                "lastrevid": 1573053,
                "length": 8238,
                "protection": [],
                "restrictiontypes": ["edit", "move"],
                "revisions": [
                    {
                        "revid": 1573053,
                        "parentid": 1570867,
                        "user": "Zweibeiner",
                        "userid": 46200245,
                        "timestamp": "2021-05-19T17:10:57Z",
                        "size": 8238,
                        "sha1": "d9e1ec49c86575d0adb5771c029355e6b943f216",
                        "slots": {
                            "main": {
                                "contentmodel": "wikitext"
                            }
                        },
                        "comment": "",
                        "tags": ["visualeditor-wikitext"]
                    }
                ]
            },
            "152970": {
                "pageid": 152970,
                "ns": 0,
                "title": "Spotfur's Rebellion/Chapter 1",
                "contentmodel": "wikitext",
                "pagelanguage": "en",
                "pagelanguagehtmlcode": "en",
                "pagelanguagedir": "ltr",
                "touched": "2021-06-12T11:15:27Z",
                "lastrevid": 1575676,
                "length": 5132,
                "protection": [],
                "restrictiontypes": ["edit", "move"],
                "revisions": [
                    {
                        "revid": 1575676,
                        "parentid": 1575657,
                        "user": "Amazingshaded",
                        "userid": 48421147,
                        "timestamp": "2021-06-12T11:15:27Z",
                        "size": 5132,
                        "sha1": "a09d5be9bb38b9afbc9967c06e1b1e1dd456f700",
                        "slots": {
                            "main": {
                                "contentmodel": "wikitext"
                            }
                        },
                        "comment": "",
                        "tags": []
                    }
                ], "pageprops": {
                    "forcetoc": "",
                    "infoboxes": "[{\"parser_tag_version\":2,\"data\":[{\"type\":\"title\",\"data\":{\"value\":\"<i>Spotfur&#39;s Rebellion<\\/i>\",\"item-name\":null,\"source\":null}},{\"type\":\"group\",\"data\":{\"value\":[{\"type\":\"data\",\"data\":{\"label\":\"Chapter number:\",\"value\":\"1\\u00a0(of 10)\",\"span\":1,\"layout\":null,\"item-name\":null,\"source\":\"chap\"}},{\"type\":\"data\",\"data\":{\"label\":\"Page numbers:\",\"value\":\"N\\/A\",\"span\":1,\"layout\":null,\"item-name\":null,\"source\":\"pages\"}},{\"type\":\"data\",\"data\":{\"label\":\"Point of View:\",\"value\":\"<!--LINK'\\\" 0:3-->\",\"span\":1,\"layout\":null,\"item-name\":null,\"source\":\"pov\"}}],\"layout\":\"default\",\"collapse\":null,\"row-items\":null,\"item-name\":null}},{\"type\":\"group\",\"data\":{\"value\":[{\"type\":\"header\",\"data\":{\"value\":\"Chronology\",\"item-name\":null,\"source\":null}},{\"type\":\"navigation\",\"data\":{\"value\":\"<p style=\\\"padding-left:10px; padding-right:10px; text-align:center;\\\"><strong class=\\\"mw-selflink selflink\\\"> \\u2022 1<\\/strong><!--LINK'\\\" 0:0--><!--LINK'\\\" 0:1--><\\/p>\",\"item-name\":null,\"source\":null}}],\"layout\":\"default\",\"collapse\":null,\"row-items\":null,\"item-name\":null}}],\"metadata\":[{\"type\":\"title\",\"sources\":[]},{\"type\":\"image\",\"sources\":{\"image2\":{\"label\":\"\"},\"image\":{\"label\":\"\"},\"option1\":{\"label\":\"\"},\"option2\":{\"label\":\"\"}}},{\"type\":\"group\",\"metadata\":[{\"type\":\"data\",\"sources\":{\"chap\":{\"label\":\"Chapter number: (chap)\",\"primary\":true},\"chapall\":{\"label\":\"Chapter number: (chapall)\"}}},{\"type\":\"data\",\"sources\":{\"pages\":{\"label\":\"Page numbers: (pages)\",\"primary\":true},\"pagee\":{\"label\":\"Page numbers: (pagee)\"}}},{\"type\":\"data\",\"sources\":{\"pov\":{\"label\":\"Point of View: (pov)\",\"primary\":true},\"povname\":{\"label\":\"Point of View: (povname)\"}}}]},{\"type\":\"group\",\"metadata\":[{\"type\":\"header\",\"sources\":[]},{\"type\":\"navigation\",\"sources\":[]}]}]}]"
                }
            },
            "152988": {
                "pageid": 152988,
                "ns": 0,
                "title": "Spotfur's Rebellion/Chapter 2",
                "contentmodel": "wikitext",
                "pagelanguage": "en",
                "pagelanguagehtmlcode": "en",
                "pagelanguagedir": "ltr",
                "touched": "2021-06-12T08:43:04Z",
                "lastrevid": 1575528,
                "length": 460,
                "new": "",
                "protection": [],
                "restrictiontypes": ["edit", "move"],
                "revisions": [
                    {
                        "revid": 1575528,
                        "parentid": 0,
                        "user": "Amazingshaded",
                        "userid": 48421147,
                        "timestamp": "2021-06-11T06:54:55Z",
                        "size": 460,
                        "sha1": "dca4d295fa38cbb65a591f9823de1ae4c67cca2e",
                        "slots": {
                            "main": {
                                "contentmodel": "wikitext"
                            }
                        },
                        "comment": "Created page with \"{{Startspoiler}} {{Inuse|[[User:Amazingshaded|Shaded]]}}  {{Chapter  |image= |chap=2 |chapall=10 |pages=N/A |pov=[[Spotfur|Spotpaw]] }}  {{chapintro|2|2|10|N/A}}  ==Chapter de...\"",
                        "tags": []
                    }
                ], "pageprops": {
                    "forcetoc": "",
                    "infoboxes": "[{\"parser_tag_version\":2,\"data\":[{\"type\":\"title\",\"data\":{\"value\":\"<i>Spotfur&#39;s Rebellion<\\/i>\",\"item-name\":null,\"source\":null}},{\"type\":\"group\",\"data\":{\"value\":[{\"type\":\"data\",\"data\":{\"label\":\"Chapter number:\",\"value\":\"2\\u00a0(of 10)\",\"span\":1,\"layout\":null,\"item-name\":null,\"source\":\"chap\"}},{\"type\":\"data\",\"data\":{\"label\":\"Page numbers:\",\"value\":\"N\\/A\",\"span\":1,\"layout\":null,\"item-name\":null,\"source\":\"pages\"}},{\"type\":\"data\",\"data\":{\"label\":\"Point of View:\",\"value\":\"<!--LINK'\\\" 0:2-->\",\"span\":1,\"layout\":null,\"item-name\":null,\"source\":\"pov\"}}],\"layout\":\"default\",\"collapse\":null,\"row-items\":null,\"item-name\":null}},{\"type\":\"group\",\"data\":{\"value\":[{\"type\":\"header\",\"data\":{\"value\":\"Chronology\",\"item-name\":null,\"source\":null}},{\"type\":\"navigation\",\"data\":{\"value\":\"<p style=\\\"padding-left:10px; padding-right:10px; text-align:center;\\\"><!--LINK'\\\" 0:0--><\\/p>\",\"item-name\":null,\"source\":null}}],\"layout\":\"default\",\"collapse\":null,\"row-items\":null,\"item-name\":null}}],\"metadata\":[{\"type\":\"title\",\"sources\":[]},{\"type\":\"image\",\"sources\":{\"image2\":{\"label\":\"\"},\"image\":{\"label\":\"\"},\"option1\":{\"label\":\"\"},\"option2\":{\"label\":\"\"}}},{\"type\":\"group\",\"metadata\":[{\"type\":\"data\",\"sources\":{\"chap\":{\"label\":\"Chapter number: (chap)\",\"primary\":true},\"chapall\":{\"label\":\"Chapter number: (chapall)\"}}},{\"type\":\"data\",\"sources\":{\"pages\":{\"label\":\"Page numbers: (pages)\",\"primary\":true},\"pagee\":{\"label\":\"Page numbers: (pagee)\"}}},{\"type\":\"data\",\"sources\":{\"pov\":{\"label\":\"Point of View: (pov)\",\"primary\":true},\"povname\":{\"label\":\"Point of View: (povname)\"}}}]},{\"type\":\"group\",\"metadata\":[{\"type\":\"header\",\"sources\":[]},{\"type\":\"navigation\",\"sources\":[]}]}]}]"
                }
            },
            "152997": {
                "pageid": 152997,
                "ns": 0,
                "title": "Spotfur's Rebellion/Chapter 3",
                "contentmodel": "wikitext",
                "pagelanguage": "en",
                "pagelanguagehtmlcode": "en",
                "pagelanguagedir": "ltr",
                "touched": "2021-06-12T08:43:04Z",
                "lastrevid": 1575661,
                "length": 460,
                "new": "",
                "protection": [],
                "restrictiontypes": ["edit", "move"],
                "revisions": [
                    {
                        "revid": 1575661,
                        "parentid": 0,
                        "user": "Amazingshaded",
                        "userid": 48421147,
                        "timestamp": "2021-06-12T08:43:04Z",
                        "size": 460,
                        "sha1": "f45ee8f1c22eba0538635d6fc33ca8158ff72dc0",
                        "slots": {
                            "main": {
                                "contentmodel": "wikitext"
                            }
                        },
                        "comment": "Created page with \"{{Startspoiler}} {{Inuse|[[User:Amazingshaded|Shaded]]}}  {{Chapter  |image= |chap=3 |chapall=10 |pages=N/A |pov=[[Spotfur|Spotpaw]] }}  {{chapintro|3|3|10|N/A}}  ==Chapter de...\"",
                        "tags": []
                    }
                ], "pageprops": {
                    "forcetoc": "",
                    "infoboxes": "[{\"parser_tag_version\":2,\"data\":[{\"type\":\"title\",\"data\":{\"value\":\"<i>Spotfur&#39;s Rebellion<\\/i>\",\"item-name\":null,\"source\":null}},{\"type\":\"group\",\"data\":{\"value\":[{\"type\":\"data\",\"data\":{\"label\":\"Chapter number:\",\"value\":\"3\\u00a0(of 10)\",\"span\":1,\"layout\":null,\"item-name\":null,\"source\":\"chap\"}},{\"type\":\"data\",\"data\":{\"label\":\"Page numbers:\",\"value\":\"N\\/A\",\"span\":1,\"layout\":null,\"item-name\":null,\"source\":\"pages\"}},{\"type\":\"data\",\"data\":{\"label\":\"Point of View:\",\"value\":\"<!--LINK'\\\" 0:3-->\",\"span\":1,\"layout\":null,\"item-name\":null,\"source\":\"pov\"}}],\"layout\":\"default\",\"collapse\":null,\"row-items\":null,\"item-name\":null}},{\"type\":\"group\",\"data\":{\"value\":[{\"type\":\"header\",\"data\":{\"value\":\"Chronology\",\"item-name\":null,\"source\":null}},{\"type\":\"navigation\",\"data\":{\"value\":\"<p style=\\\"padding-left:10px; padding-right:10px; text-align:center;\\\"><!--LINK'\\\" 0:0--><!--LINK'\\\" 0:1--><\\/p>\",\"item-name\":null,\"source\":null}}],\"layout\":\"default\",\"collapse\":null,\"row-items\":null,\"item-name\":null}}],\"metadata\":[{\"type\":\"title\",\"sources\":[]},{\"type\":\"image\",\"sources\":{\"image2\":{\"label\":\"\"},\"image\":{\"label\":\"\"},\"option1\":{\"label\":\"\"},\"option2\":{\"label\":\"\"}}},{\"type\":\"group\",\"metadata\":[{\"type\":\"data\",\"sources\":{\"chap\":{\"label\":\"Chapter number: (chap)\",\"primary\":true},\"chapall\":{\"label\":\"Chapter number: (chapall)\"}}},{\"type\":\"data\",\"sources\":{\"pages\":{\"label\":\"Page numbers: (pages)\",\"primary\":true},\"pagee\":{\"label\":\"Page numbers: (pagee)\"}}},{\"type\":\"data\",\"sources\":{\"pov\":{\"label\":\"Point of View: (pov)\",\"primary\":true},\"povname\":{\"label\":\"Point of View: (povname)\"}}}]},{\"type\":\"group\",\"metadata\":[{\"type\":\"header\",\"sources\":[]},{\"type\":\"navigation\",\"sources\":[]}]}]}]"
                }
            },
            "152158": {
                "pageid": 152158,
                "ns": 14,
                "title": "Category:Spotfur's Rebellion characters",
                "contentmodel": "wikitext",
                "pagelanguage": "en",
                "pagelanguagehtmlcode": "en",
                "pagelanguagedir": "ltr",
                "touched": "2021-05-17T03:27:53Z",
                "lastrevid": 1565970,
                "length": 191,
                "new": "",
                "protection": [],
                "restrictiontypes": ["edit", "move"],
                "revisions": [
                    {
                        "revid": 1565970,
                        "parentid": 0,
                        "user": "Vector Sigma",
                        "userid": 27042031,
                        "timestamp": "2021-04-05T16:01:31Z",
                        "size": 191,
                        "sha1": "2bcc7ae3263d650c97c0c559de40cf37517263f7",
                        "slots": {
                            "main": {
                                "contentmodel": "wikitext"
                            }
                        },
                        "comment": "Created page with \"This category lists the characters from ''[[Spotfur's Rebellion]]'' in the [[Novellas]] of ''Warriors''. [[Category:Novellas]] [[Category:Characters by book]] Category:Spotf...\"",
                        "tags": []
                    }
                ], "categoryinfo": {
                    "size": 165,
                    "pages": 165,
                    "files": 0,
                    "subcats": 0
                }
            },
            "145414": {
                "pageid": 145414,
                "ns": 6,
                "title": "File:NO-20.jpg",
                "contentmodel": "wikitext",
                "pagelanguage": "en",
                "pagelanguagehtmlcode": "en",
                "pagelanguagedir": "ltr",
                "touched": "2020-12-16T02:15:08Z",
                "lastrevid": 1542779,
                "length": 120,
                "protection": [],
                "restrictiontypes": ["edit", "move", "upload"],
                "revisions": [
                    {
                        "revid": 1542779,
                        "parentid": 1451816,
                        "user": "Vector Sigma",
                        "userid": 27042031,
                        "timestamp": "2020-12-02T03:15:02Z",
                        "size": 120,
                        "sha1": "68cb61dda4c1547b0c59e0e0a9a3345f4c4928ba",
                        "slots": {
                            "main": {
                                "contentmodel": "wikitext"
                            }
                        },
                        "comment": "VectorSigma101 uploaded a new version of [[File:NO-20.jpg]]",
                        "tags": []
                    }
                ], "pageprops": {
                    "words_count": "1"
                },
                "imagerepository": "local",
                "imageinfo": [
                    {
                        "timestamp": "2020-12-02T03:15:02Z",
                        "user": "Vector Sigma",
                        "size": 175471,
                        "width": 435,
                        "height": 648,
                        "comment": "",
                        "url": "https://static.wikia.nocookie.net/warriors/images/1/19/NO-20.jpg/revision/latest?cb=20201202031502",
                        "descriptionurl": "https://warriors.fandom.com/wiki/File:NO-20.jpg",
                        "descriptionshorturl": "https://warriors.fandom.com/index.php?curid=145414",
                        "sha1": "81a55e538cbdf13e0f6231e01b51f65cf204ad01"
                    }
                ]
            }
        }
    }
}

2nd request

POST https://warriors.fandom.com/api.php HTTP/1.1

format=json&action=query&maxlag=5&inprop=protection&rvprop=ids%7ctimestamp%7cflags%7ccomment%7cuser%7cuserid%7ccontentmodel%7csha1%7ctags%7csize&rvslots=main&iiprop=timestamp%7cuser%7ccomment%7curl%7csize%7csha1&prop=info%7crevisions%7ccategoryinfo%7cpageprops%7cimageinfo&generator=categorymembers&gcmtitle=Category%3aSpotfur%27s+Rebellion&gcmlimit=50&gcmtype=page%7csubcat%7cfile&iistart=2020-03-08T03%3a16%3a46Z&continue=%7c%7cinfo%7crevisions%7ccategoryinfo%7cpageprops

Response

{
    "batchcomplete": "",
    "query": {
        "pages": {
            "145387": {
                "pageid": 145387,
                "ns": 0,
                "title": "Spotfur's Rebellion"
            },
            "152171": {
                "pageid": 152171,
                "ns": 0,
                "title": "Spotfur's Rebellion/Allegiances"
            },
            "152970": {
                "pageid": 152970,
                "ns": 0,
                "title": "Spotfur's Rebellion/Chapter 1"
            },
            "152988": {
                "pageid": 152988,
                "ns": 0,
                "title": "Spotfur's Rebellion/Chapter 2"
            },
            "152997": {
                "pageid": 152997,
                "ns": 0,
                "title": "Spotfur's Rebellion/Chapter 3"
            },
            "152158": {
                "pageid": 152158,
                "ns": 14,
                "title": "Category:Spotfur's Rebellion characters"
            },
            "145414": {
                "pageid": 145414,
                "ns": 6,
                "title": "File:NO-20.jpg",    // <-- This page gets split across the continuation.
                "imagerepository": "local",
                "imageinfo": [
                    {
                        "timestamp": "2020-03-08T03:16:46Z",
                        "user": "Potato Flakin",
                        "size": 497651,
                        "width": 1280,
                        "height": 1904,
                        "comment": "",
                        "url": "https://static.wikia.nocookie.net/warriors/images/1/19/NO-20.jpg/revision/20201202031501",
                        "descriptionurl": "https://warriors.fandom.com/wiki/File:NO-20.jpg",
                        "descriptionshorturl": "https://warriors.fandom.com/index.php?curid=145414",
                        "sha1": "e46e6c32b79a349edf25dccfdc5679e08ea8d5f4"
                    }
                ]
            }
        }
    }
}
CXuesong commented 3 years ago

Hint: we may leverage batchcomplete since MW 1.25 to decide whether we have retrieved all the information needed for a batch of pages. If there is no such property, we should enumerate to the next page, and merge the response before further processing.

CXuesong commented 3 years ago

Not reproduceable on MW Commons [1.37.0-wmf.16 (cbd846f)]. Considering FANDOM is on MW 1.33.3, we may wait for a while and revisit this issue.