jikan-me / jikan-rest

The REST API for Jikan
https://docs.api.jikan.moe/
MIT License
435 stars 269 forks source link

💡 Synopsis in general /episodes endpoint #341

Open Fribb opened 1 year ago

Fribb commented 1 year ago

Is there an existing issue for this?

Is your feature request related to a problem?

Currently, to read the synopsis of an episode you have to request that specific episode. This leads to a lot of requests because each episode has to be requested individually.

Edit: Unlike initially observed, there are some differences between what the individual episode at /episode/1 is returning and what an episode element has in /episodes

field /episodes /episodes/X
mal_id x x
url x x
title x x
title_japanese x x
title_romanji x x
aired x x
score x
filler x x
recap x x
forum_url x
duration x
synopsis x

That would mean, to be able to request all information of each episode you would need to request each page of /episodes to get the score a forum_url but also each individual episode for the duration and synopsis

Describe the solution you'd like

If available, the synopsis should be available as well in the /episodes endpoint to cut down the potential requests to be made to the API.

It would also be fantastic if the /episodes endpoint could trigger the update of the episodes metadata

Describe alternatives you've considered

No response

Teachability, Documentation, Adoption, Migration Strategy

As described above, it would cut down the number of requests a user has to do if they don't have the ability to update individual episode metadata. You would be able to hit up the /episodes endpoint and go from there, reducing the number of requests made to the API by once for every 100 episodes instead of X times for all episodes.

Fribb commented 1 year ago

I updated the initial enhancement with further information.

Apparently, the /episodes and /episodes/X endpoints provide different sets of information as you can see in the provided table.

This would mean that to get a comprehensive result you would not only request the individual episodes but also the episode list with each page.

So an Anime like One Piece would require:

So 1076 requests in total, according to the v4 rate limit with 3 Requests each second this would take roughly 358 Seconds (roughly 6 Minutes) to request all of the data for the episodes.

Consolidating the Information on the /episodes endpoint would, as initially stated, cut down this time to 3 seconds.

pushrbx commented 1 year ago

The API is following what is visible to the user when yourself go on an anime's datasheet:

If Jikan would want to consolidate it into one, it would require to regularly scrape all anime data-sheets to have an up to date database about the episodes, which is not feasible to do. (Also it's not fair towards MAL)

If your request to /episode and /episodes is not yet cached, it translates to an http request to MAL and the scraper parses the page, then you get the scraper's result.

Fribb commented 1 year ago

I can understand that the update process could be a bit much, especially with data that hasn't been cached yet. With slow connections or anything like that queuing 100 requests to update the individual episodes might end more in a timeout.

But providing the information on an /episodes level would still be beneficial even if it is just outdated data maybe as a parameter.

Still, I don't really think that it is unfair towards MAL. An API should make things easier and faster to use for someone who wants to consume the data.

Correct me if I'm wrong but wasn't v4 intended to have a better data structure to not be as static as v3 was? I remember asking the same thing a while ago but at that time this was basically the answer that this wasn't possible with v3 and that it might be with v4 by joining the tables.

irfan-dahir commented 1 year ago

Correct me if I'm wrong but wasn't v4 intended to have a better data structure to not be as static as v3 was? I remember asking the same thing a while ago but at that time this was basically the answer that this wasn't possible with v3 and that it might be with v4 by joining the tables.

Yes, v4 took that step towards resources (Anime, Manga, etc) and search.

We can look into the feasibility of this in the next phase of things. At the moment it's not pushing the episodes+details into a table. It's just caching the JSON response into the DB - like a lot of the other requests which are parsed directly.

I understand the problem and do agree with you. But this would require some major overhaul to how things are processed, stored, and served so wouldn't say this would be something done in the near future.