jwueller
commented
1 year ago Great find, thank you! I was aware that an API exists, but as far as I was told, it's not final yet. But it might still be a better idea to use the unfinished API as opposed to scraping anyway. Luckily, the data fetching is pretty independent of the matching logic, so it could probably be swapped out without major interruptions. I'll look into it.
Hey, I think I might have a way to alleviate some of the manual website scraping required in the project. You can actually send GraphQL queries at https://onepace.net/api/graphql. The schema was a bit hard to determine, but using some previous work from Jaxel on the One Pace discord, I was able to flesh out a mostly complete GraphQL query for the arcs. The JSON response includes all the information about every single arc. This should make parsing extremely easy since the GraphQL API will return consistent results. The format for the query is schema.txt
You can send a minified version of this as the query parameter in either a GET or POST request. So something like this should give you all the information you need: https://onepace.net/api/graphql?query={databaseGetAllArcs{title,description,translations{title,description,language{code}},manga_chapters,anime_episodes,images{src},episodes{title,description,translations{title,description},part,manga_chapters,anime_episodes,released_date,images{src}}}}.
The schema.txt file is also a useful guide for parsing the JSON response, since all the fields in the response follow the format of the query. A quick note about the image links provided by the response. The src field under the images field only provides the last part of the full URL, e.g: "cover-romance-dawn-arc_270w.jpg". However the full URL is not too difficult to find. For arcs, the root URL for images is "https://onepace.net/images/arcs/" and for individual episodes within an arc, the root URL for images is "https://onepace.net/images/episodes/". So for the romance-dawn-arc example, the full URL would be: https://onepace.net/images/arcs/cover-romance-dawn-arc_270w.jpg.
Hope you find this useful.