NoppesTheFolf / E621Client

.NET Standard wrapper for the e621 API.
MIT License
9 stars 1 forks source link
csharp dotnet-standard e621 e621-api

E621Client

Build Status Nuget Codacy Badge

E621Client

E621Client is an unofficial .NET Standard 2.1 library for interacting with the e621 and e926 API maintained by me, Noppes. It has support for nullable and non-nullable reference types.

Table of Contents

Completeness

This project is far from a complete wrapper around the e621 API. A table can be seen below to give you a quick idea of how complete this library is. I might never implement most of the API because I'd never use those endpoints myself. However, feel free to implement those areas yourself if you need them! Just make sure the check out how to contribute first.

Legend

Symbol Meaning
:x: Not implemented
:heavy_minus_sign: Partially implemented
:heavy_check_mark: Fully implemented

Cover per API area

Area Complete Comment
Authentication :heavy_check_mark:
Posts :heavy_minus_sign: Only the retrieval of posts
Tags :heavy_minus_sign: Only the retrieval of tags
Tag aliases :x:
Tag implications :x:
Notes :x:
Pools :heavy_minus_sign: Only the retrieval of pools
Users :heavy_minus_sign: Only the retrieval of a user by name
Favorites :heavy_check_mark:
Artists :heavy_minus_sign: Only the retrieval of artists, not documented by e621
IQDB :heavy_check_mark: Not yet documented by e621 at the moment of writing
DB export :heavy_minus_sign: All except wiki pages

Installation

E621Client is available as a NuGet package listed as Noppes.E621Client. You can easily install it using either the Package Manager Console or the .NET CLI.

Package Manager Console

Install-Package Noppes.E621Client -Version 0.10.0

.NET CLI

dotnet add package Noppes.E621Client --version 0.10.0

Getting started

You will need a IE621Client instance in order to interact with the API. These instances can only be created using the E621ClientBuilder class. The builder will allow you to create your very own personalized IE621Client instance in a fluent manner based on the specific needs of your application. Just make sure you at least specify User-Agent information, as e621 requires it. Not specifying it will cause an exception to be thrown.

Bare minimum example

var e621Client = new E621ClientBuilder()
    .WithUserAgent("MyApplicationName", "MyApplicationVersion", "MyTwitterUsername", "Twitter")
    .Build();

However, you might need something a little more suited for your application. The default IE621Client instance built above uses settings tuned to make sure the load on e621's side is kept to a minimum. This is not desirable if you're, for example, developing an interactive tool and therefore want it to be as snappy as possible.

Example for an interactive application

var e621Client = new E621ClientBuilder()
    .WithUserAgent("MyApplicationName", "MyApplicationVersion", "MyTwitterUsername", "Twitter")
    .WithMaximumConnections(E621Constants.MaximumConnectionsLimit)
    .WithRequestInterval(E621Constants.MinimumRequestInterval)
    .Build();

We've also added support to use this client for any website that is running the open source E621ng framework and thus is sharing the same API. For example (E926 or E6AI). In order to change which image board the client is using you can use the WithBaseUrl(Uri) method on the E621ClientBuilder like below:

Example for use with a different E621ng image board

var e926Client = new E621ClientBuilder()
    .WithUserAgent("MyApplicationName", "MyApplicationVersion", "MyTwitterUsername", "Twitter")
    .WithBaseUrl(new Uri("https://e926.net"))
    .Build();

Since E621 and E926 are officially supported you can also use E621Constants.E621BaseUrl or E621Constants.E926BaseUrl instead of creating a new Uri directly

IE621Client instances can be disposed of, but you generally want to treat an IE621Client instance as a singleton. It uses a HttpClient behind the scenes which gets disposed when you dispose the associated IE621Client. You can read more about why that's bad at You're using HttpClient wrong and it is destabilizing your software if you're interested.

Authentication

Authentication will have to take place in order to be gain access to certain protected resources. E621Client makes this convenient by mimicking a login- and logout-based system.

Logging in

Logging in can only be done based on a combination of username and API key.

Log in using a username and API key

bool success = await e621Client.LogInAsync("MyUsername", "MyApiKey");

The method returns a boolean that indicates whether not the login attempt was a success. It does this by requesting the user's favorites. This method executing successfully, implies that the user entered valid credentials. In case the method return false, it may also mean that the user doesn't have API access enabled in their account. There is no way of determining if that's the case though.

Logging out

You need to log out in order for another log in to be allowed to happen. You'll most likely don't need this this for most applications, but here is a method that does so anyway.

Log the currently logged-in user out

e621Client.Logout();

Functionality per API area

The main point of this section is to show how the e621 API endpoints map to E621Client methods. The methods themselves are sometimes more elaborately documented than they are here. So don't worry if things may still seem kind off vague after reading this, check the method documentation out too!

Posts

Retrieving a post

A post can be retrieved using either its ID or its image's MD5 hash by calling the GetPostAsync method.

Retrieving a post by its ID

var postById = await e621Client.GetPostAsync(546281);

Retrieving a post by its image's MD5 hash

var postByMd5 = await e621Client.GetPostAsync("900e98af5b512ba1a5f8a1a9885c1ef1");

Retrieving posts

A collection of posts can be retrieved using the GetPostsAsync method. There is a limit on how much posts can be retrieved in a single call to this method. This limited is defined at E621Constants.PostsMaximumLimit. Therefore, you need to be able to navigate through e621's post collection. There are two ways to do this: pagination and relative positioning.

Without navigation

You can request a bunch of posts without navigation, but the use cases are very limited of course.

Retrieve a collection of posts tagged "canine" with the maximum number of posts retrievable in a single call without using any navigation

var posts = await e621Client.GetPostsAsync("canine", limit: E621Constants.PostsMaximumLimit);
Navigation using pagination

You're probably already familiar with the concept of pagination: a collection of something, in this case posts, is split into parts of equal size. Each of those parts is assigned a number and these parts can then be requested using that number. That number is which we call a "page".

Be aware that there is a limit on the maximum allowed page number. This limit is defined at E621Constants.PostsMaximumPage. Exceeding this number will cause an exception to be thrown.

Retrieve the fifteenth page of a collection of posts tagged "canine"

var posts = await e621Client.GetPostsAsync("canine", 15);

Retrieve the fifteenth page of a collection of posts tagged "canine" with the maximum number of posts retrievable in a single call

var posts = await e621Client.GetPostsAsync("canine", 15, E621Constants.PostsMaximumLimit);
Navigation using relative positioning

This may sound a bit scary at first, but it really isn't. All you need to do is specify both an post ID and a position. The position parameter defines the position of the returned posts relative to the given post ID.

Let's take post with ID 1000 as an example. Passing this ID in combination with Position.Before will cause the posts 999, 998, 997, etc. to be retrieved. Using Position.After will retrieve the posts 1001, 1002, 1003, etc. You should use this method if you don't need pagination or need to avoid the limit pagination comes with. Moreover, this is the most efficient way to navigate through posts.

Retrieve a collection of posts with an ID less than 200

var posts = await e621Client.GetPostsAsync(200, Position.Before);

Retrieve a collection of posts with an ID greater than 200

var posts = await e621Client.GetPostsAsync(200, Position.After);

Retrieve a collection of posts tagged "canine" with an ID greater than 200 using the maximum limit of posts retrieved in a single call

var posts = await e621Client.GetPostsAsync(200, Position.After, "canine", E621Constants.PostsMaximumLimit);

Tags

Retrieving a tag

A tag can either be retrieved by either its name or ID by using the GetTagAsync method.

Retrieve a tag by ID

var tag = await e621Client.GetTagAsync(813847);

Retrieve a tag by name

var tag = await e621Client.GetTagAsync("noppes");

Retrieving tags

There are a couple of different ways a collection of tags can be retrieved: without using any filter, using the names of the tags and using a query.

Searching for tags without using filter, opens up the usage of pagination with relative positioning. This works in exactly the same way as it does for posts.. All of the other available overloads make use of pagination as you're used to.. The maximum allowed page number is defined at E621Constants.TagsMaximumPage.

Note that there is a limit on the number of tags that can be retrieved in a single call to any of the overloads. This limit is defined at E621Constants.TagsMaximumLimit.

Without filter

The first way of retrieving tags is without using any specific name filter with the GetTagsAsync method.

Get the maximum possible number of tags after ID 1000

var tags = await e621Client.GetTagsAsync(1000, Position.After, E621Constants.TagsMaximumLimit);

Get the 10th page using the maximum possible number of tags ordered by the number of posts making use of the tag

var tags = await e621Client.GetTagsAsync(10, E621Constants.TagsMaximumLimit, order: TagOrder.Count);
Using tags and their names

A collection of tags can also be retrieved using their names using the GetTagsUsingNamesAsync method.

Get tags using their names

var tagNames = new []
{
    "noppes",
    "blep",
    "fur"
};
var tags = await e621Client.GetTagsByNamesAsync(tagNames);
Using a search query

Tags can also be retrieved using a search query on the tag their names using a wildcards, for example, with an overload of the GetTagsUsingNamesAsync method.

Get the first page of tags that start with 'wolf' in the species category

var tags = await e621Client.GetTagsByNamesAsync("wolf*", category: TagCategory.Species);

Pools

Retrieving a pool

You can retrieve the information of a single pool by its ID. If there exists no pool with the given ID, a null value will be returned. If you need to retrieve more than one pool, you should take a look at the next section of the documentation. There is a more efficient way to do that than to call this method multiple times.

Retrieve information about pool with ID 621

var pool = await e621Client.GetPoolAsync(621);

Retrieving pools

As an expansion upon the previous section, you can retrieve multiple pools by their IDs. IDs to which there is no pool associated, will be left out of the result.

Retrieve pool with ID 621 and pool with ID 926

var poolIds = new[]
{
    621,
    926
};
var pools = await e621Client.GetPoolsAsync(ids: poolIds);

You can also get a listing of all the pools on e621. You can navigate through all of the available pools using either pagination or relative position. Both of these concepts have already been explained in the posts section of the documentation and therefore won't be further elaborated on here. Using relative positioning won't allow you to specify an order in which pools should be sorted.

Get the third page of pools with the response containing as much pools as allowed

var pools = await e621Client.GetPoolsAsync(3, limit: E621Constants.PoolsMaximumLimit);

To narrow down the pools returned by e621, you can filter based on the following attributes of a pool: name, description, by who it was created, whether its still being actively updated, whether it has been deleted and its category.

Get (the first page of) pools of which the name ends in "cat" and which are not deleted, in descending order by the amount of posts contained in the pool.

var pools = await e621Client.GetPoolsAsync(name: "*cat", isDeleted: false, order: PoolOrder.PostCount)

Users

Currently E621Client doesn't support much of the users area of the API mainly due to there being no documentation on it whatsoever at the moment this was written.

Retrieving a user

You can retrieve a part of the info available about a user by searching for them using their username.

Get information about a user by searching by their username

var user = await e621Client.GetUserAsync("noppes");

Retrieving the logged-in user

You can also retrieve the same info for the user that is currently logged-in.

Get information about the currently logged-in user

var user = await e621Client.GetLoggedInUserAsync();

Favorites

Adding a post

Adding a post to the logged-in user their favorites, can be done with the AddFavoriteAsync method.

Adding a post using its ID to the logged-in user's favorites

await e621Client.AddFavoriteAsync(546281);

Removing a post

You can remove a post from the logged-in user their favorites using the RemoveFavoriteAsync method.

Removing a post by its ID from the logged-in user's favorites

await e621Client.RemoveFavoriteAsync(546281);

Retrieving favorites

We can retrieve the posts favorited by either the logged-in user or some other user by using their user ID.

Retrieving the fifth page of posts favorited by the logged-in user, retrieving as many posts as possible in a single call

var favorites = await e621Client.GetOwnFavoritesAsync(5, E621Constants.FavoritesMaximumLimit);

Retrieving the seventh page of posts favorited by the user with ID 11271 (SnowWolf)

var favorites = await e621Client.GetFavoritesAsync(11271, 7);

Artists

Retrieving an artist

You can retrieve a single artist by their ID.

Retrieve information about artist with ID 8695, which is the artist WagnerMutt

var wagnermutt = await e621Client.GetArtistAsync(8695);

Retrieving artists

You can also retrieve a list of artists and optionally filter the results by name, for example. Pagination works the same way as it does for other endpoints with normal pagination using numbers and relative positioning.

Retrieve a list of artists that contain the the name "wagner"

var wagners = await e621Client.GetArtistsAsync(name: "wagner")

Database exports

e621 provides exports of posts, pools, tags, tag aliases, tag implications and wiki pages.

E621Client supports downloading and the reading all of these exports except wiki pages. Due to this functionality being unlikely to be used in combination with it introducing a few dependencies, it has its own separate package: Noppes.E621Client.DbExport.

Package Manager Console

Install-Package Noppes.E621Client.DbExport -Version 0.10.0

.NET CLI

dotnet add package Noppes.E621Client.DbExport --version 0.10.0

After installing the package, you need to get a database export client. You can get one by calling the GetDbExportClient method on your IE621Client instance, as shown below.

Get a database export client

var dbExportClient = e621Client.GetDbExportClient();

The next step you want to take whenever you want to retrieve a database export, is get a list of all the database exports available on e621. This can be done using the GetDbExportsAsync method on your database export client. This list can also be viewed on e621.

Get a list of all the available database exports

var exports = await dbExportClient.GetDbExportsAsync();

Now that you have a list of all the database exports available on e621, it is time to choose which one you'd like to download and read. The retrieved exports contain a handy extension method named Latest. With this method, you can get the latest database export of a given type (posts, pools, tags, etc.).

After selecting the desired database export, you need to get the data as a stream using the GetDbExportStreamAsync method. This stream, depending on what export you're downloading, can then be read using one of the following methods on your database export client:

Posts: ReadStreamAsPostsDbExportAsync\ Pools: ReadStreamAsPoolsDbExportAsync\ Tags: ReadStreamAsTagsDbExportAsync\ Tag implications: ReadStreamAsTagImplicationsDbExportAsync\ Tag aliases: ReadStreamAsTagAliasesDbExportAsync

The code below shows a concrete example of how the use the interface described above.

Print the IDs of the posts in the latest post database export to the console

var export = exports.Latest(DbExportType.Post);
await using var stream = await dbExportClient.GetDbExportStreamAsync(export);
await foreach (var post in dbExportClient.ReadStreamAsPostsDbExportAsync(stream))
  Console.WriteLine(post.Id);

IQDB (Reverse image searching)

Note: Be sure to check out another project of mine, fluffle.xyz, if you're interested in this kind of functionality.

You can reverse search an image on e621 by either a locally stored image, a stream or a URL. In addition it is also possible to use another post as reference for the reverse search query (uses the image attached to the post). Reverse searching will return a collection of posts of which the images are similar to the submitted image/post. The returned posts have an additional property named IqdbScore which can be used to assess how similar the image is to the submitted one. E621Client will by default not return posts that have been deleted. However, if you'd like to include them, you can simply pass a boolean to any of the methods associated with querying IQDB.

In case you're using the URL method, note that e621 will download images only from domains whitelisted by them. Which domains are on the whitelist is unknown. You should test if the domains of the URLs you are planning to use are whitelisted or not.

Reverse image searching using a file, also returning deleted posts

var results = await e621Client.QueryIqdbByFileAsync("/my/path", false);

Reverse search the image of a post

var results = await e621Client.QueryIqdbByPostIdAsync(546281);

Reverse image searching using a URL

var results = await e621Client.QueryIqdbByUrlAsync("https://url.net");

Reverse image searching using a stream

// You can use any stream, a FileStream is simply used as an example here
await using var exampleStream = File.OpenRead("/my/path");
var results = await e621Client.QueryIqdbByStreamAsync(exampleStream);

Additional

Get response body as a stream

In case you need to get data from e621 that requires authorization, especially images, you can request said data as a Stream by using the GetStreamAsync method.

Get the response body as a stream from a given URL

await using var stream = await e621Client.GetStreamAsync("my/url");

Testing

E621Client supports testing by mocking. E621ClientBuilder.Build() will return an interface IE621Client which can be mocked using a mocking framework. You can use this to test your own logic with different responses of the IE621Client.

Report a bug

You can open an issue. Please make sure to include a piece of code that reproduces the bug. That will make troubleshooting a lot easier on my side! If you do not have a GitHub account, then also feel free to contact me.

Contributing

Contributions to this project are very welcome! Please open an issue where you describe what you'd like to contribute and why it is useful if that is not obvious. If you want to contact me personally, you can find places to contact me at on my website.