rwv37
commented
2 years ago If not, is there a workaround for this?
I can just catch the System.InvalidOperationException and return, but I mean something more natural.
Closed rwv37 closed 2 years ago
rwv37
commented
2 years ago If not, is there a workaround for this?
I can just catch the System.InvalidOperationException and return, but I mean something more natural.
CXuesong
commented
2 years ago I think this error occurs because page.Value is a JValue on L31.
I'm not sure how it happens, but perhaps can you share me the code you've used to return the IAsyncEnumerable<WikiPage> in itemsToAdd delegate? Which wiki site are you querying against? Is it a public wiki?
rwv37
commented
2 years ago Sorry for the late reply. Unfortunately, the exact code that was giving this exact error is gone. However, I later worked around a seemingly similar error in the same way as I previously worked around the earlier error, and I can show you the new one:
The difference in behavior is that whereas in the previous case, an exception was being thrown, in this case, it behaves as if the await foreach on the IAsyncEnumerable never finishes. To be clear, just like in the original report, the buggy behavior happens if and only if there are no pages to be returned.
I don't know exactly what the difference in my code is that led to this change in behavior, but I suspect that it's this: Previously (when the problem was that an exception was being thrown), I was doing something like...
// generator is a WikiClientLibrary.Generators.QueryPageGenerator
return generator.EnumPagesAsync().Take(100);... then doing an await on that return value, and then an await foreach on the awaited return. Whereas the new problem is occurring while I am instead doing:
// generator is still a WikiClientLibrary.Generators.QueryPageGenerator
var pages = new KludgyWikiPageEnumerable(generator.EnumPagesAsync());
await foreach (var page in pages.ConfigureAwait(false).WithCancellation(cancellationToken))
{
yield return page;
}... and then doing an await foreach directly on the awaited return. So I'm guessing that maybe the change in behavior comes down to using Take() versus using yield return. I don't know, though.
I worked around both this error and the previous one by creating the following two classes:
using System.Collections.Generic;
using System.Threading;
using WikiClientLibrary.Pages;
namespace Rwv37.MediaWiki.Api
{
/// <summary>
/// <para>
/// A kludgy class to deal with an apparent issue in WikiClientLibrary
/// that happens when you attempt to retrieve a list that has no members
/// (like, you try to get the items that are on Special:WantedFiles, but
/// there are no such items).
/// </para><para>
/// See: <a href="https://github.com/CXuesong/WikiClientLibrary/issues/92">Problem getting list of "wanted" items when there are none</a>
/// </para>
/// </summary>
internal class KludgyWikiPageEnumerable : IAsyncEnumerable<WikiPage>
{
private IAsyncEnumerable<WikiPage> WikiPages { get; init; }
/// <summary>
/// Initializes a new instance of the <see cref="T:Rwv37.MediaWiki.Api.KludgyWikiPageEnumerable" /> class.
/// </summary>
/// <param name="wikiPages">
/// The wiki pages.
/// </param>
internal KludgyWikiPageEnumerable(IAsyncEnumerable<WikiPage> wikiPages)
{
this.WikiPages = wikiPages;
}
/// <summary>
/// Returns an enumerator that iterates asynchronously through the collection.
/// </summary>
/// <param name="cancellationToken">
/// A <see cref="T:System.Threading.CancellationToken">CancellationToken</see>
/// that may be used to cancel the asynchronous iteration.
/// </param>
/// <returns>
/// An enumerator that can be used to iterate asynchronously through the collection.
/// </returns>
public IAsyncEnumerator<WikiPage> GetAsyncEnumerator(
CancellationToken cancellationToken = default)
{
return new KludgyWikiPageEnumerator(
this.WikiPages.GetAsyncEnumerator(cancellationToken),
this.WikiPages.GetAsyncEnumerator(cancellationToken));
}
}
}using System;
using System.Collections.Generic;
using System.Threading;
using System.Threading.Tasks;
using WikiClientLibrary.Pages;
namespace Rwv37.MediaWiki.Api
{
/// <summary>
/// <para>
/// A kludgy class to deal with an apparent issue in WikiClientLibrary
/// that happens when you attempt to retrieve a list that has no members
/// (like, you try to get the items that are on Special:WantedFiles, but
/// there are no such items).
/// </para><para>
/// See: <a href="https://github.com/CXuesong/WikiClientLibrary/issues/92">Problem getting list of "wanted" items when there are none</a>
/// </para>
/// </summary>
internal class KludgyWikiPageEnumerator : IAsyncEnumerator<WikiPage>
{
private IAsyncEnumerator<WikiPage> UnderlyingReal { get; init; }
private IAsyncEnumerator<WikiPage> UnderlyingTester { get; init; }
private bool WackinessTested { get; set; } = false;
private bool IsWacky { get; set; } = false;
/// <summary>
/// Initializes a new instance of the <see cref="T:Rwv37.MediaWiki.Api.KludgyWikiPageEnumerator" /> class.
/// </summary>
/// <param name="underlyingReal">
/// The underlying enumerator to "really" use.
/// </param>
/// <param name="underlyingTester">
/// The underlying enumerator to use to test if we're gonna have a problem.
/// </param>
internal KludgyWikiPageEnumerator(IAsyncEnumerator<WikiPage> underlyingReal,
IAsyncEnumerator<WikiPage> underlyingTester)
{
this.UnderlyingReal = underlyingReal;
this.UnderlyingTester = underlyingTester;
}
private async ValueTask<bool> TestWackinessAsync(CancellationToken cancellationToken = default)
{
try
{
if (!this.WackinessTested)
{
_ = await this.UnderlyingTester.MoveNextAsync(cancellationToken).ConfigureAwait(false);
this.IsWacky = false;
}
}
catch (InvalidOperationException)
{
this.IsWacky = true;
}
finally
{
this.WackinessTested = true;
}
return this.IsWacky;
}
/// <summary>
/// Gets the element in the collection at the current position of the enumerator.
/// </summary>
public WikiPage Current
{
get
{
return this.UnderlyingReal.Current;
}
}
/// <summary>
/// Dispose as an asynchronous operation.
/// </summary>
/// <returns>
/// A Task<ValueTask> representing the asynchronous operation.
/// </returns>
[System.Diagnostics.CodeAnalysis.SuppressMessage(
"IDisposableAnalyzers.Correctness",
"IDISP007:Don't dispose injected",
Justification = "Uh... I think this is cool? Errr... or \"hope\", at least?")]
public async ValueTask DisposeAsync()
{
await this.UnderlyingReal.DisposeAsync().ConfigureAwait(false);
await this.UnderlyingTester.DisposeAsync().ConfigureAwait(false);
}
/// <summary>
/// Move next as an asynchronous operation.
/// </summary>
/// <returns>
/// A Task<System.Boolean> representing the asynchronous operation.
/// </returns>
public async ValueTask<bool> MoveNextAsync()
{
if (!this.WackinessTested)
{
_ = await this.TestWackinessAsync().ConfigureAwait(false);
}
return !this.IsWacky && await this.UnderlyingReal.MoveNextAsync().ConfigureAwait(false);
}
}
}Usage is just wrapping the IAsyncEnumerable returned by WikiClientLibrary in a KludgyWikiPageEnumerable. For example, the following code works fine, but before I wrapped the generator.EnumPagesAsync() within a new KludgyWikiPageEnumerable(), my program was acting as if the await foreach on the IAsyncEnumerable never completed:
public async IAsyncEnumerable<WikiPage> QueryAsync(
int paginationSize = 100,
[EnumeratorCancellation] CancellationToken cancellationToken = default)
{
// TODO: Think about pagination size
var generator = new QueryPageGenerator(this.Site, this.QueryPageName)
{
PaginationSize = paginationSize,
};
var pages = new KludgyWikiPageEnumerable(generator.EnumPagesAsync());
await foreach (var page in pages.ConfigureAwait(false).WithCancellation(cancellationToken))
{
yield return page;
}
}Actually, QueryPageGenerator.EnumItemsAsync haven't ever been working before... Please try the latest release instead.
Released v0.8.0-int.6.
I'm trying to get Wantedtemplates, Wantedpages, etc. It's working fine for me for some of them, but an exception is being thrown by others (from within WikiClientLibrary). I believe I have narrowed it down to "works fine" = "there are some such wanted things" and "exception thrown" = "there are no such wanted things".
I'm not sure if this is because I'm doing something wrong or if it's perhaps a bug in WCL or in something at a deeper level than that. If it's not simply because I'm doing something wrong, I suspect it might have something to do with the issue outlined here: Apparently "HasValues" should be called on a Newtonsoft object before trying to access a child object. Disclaimer: I know nothing about Newtonsoft beyond what it is and that it's popular.
Am I doing something wrong? If not, is there a workaround for this? Any help would be appreciated.
I am getting the error when I try to do an "await foreach" on the items, and also if I simply try to get the count of them.
Here's the part of my code that's failing:
And here's the exception (in the "CountAsync" case):