[feature request] add 'comments' field to 'edit' function

[bradenbest]

The comments field is one I use quite extensively to remind myself of various things, like why I loved/hated a show so much, or what convinced me to add it to my on-hold queue (which I treat as a high-priority plan to watch, so explanations are necessary). It would be nice to have that here so I don't have to use the website.

I didn't see a way to tag the issue from the editor, so I put the tag in the title out of consideration.

[bradenbest]

I realize the dev is probably busy, so to help out somewhat, one suggestion to make it easy to handle multi-line is to use the \n escape sequence

comments: Look at that lil' kitty face.\n\nSo cute!

Will appear on MAL as:

...
Time Spent Watching: ...
comments: Look at that lil' kitty face.

So cute!

And vice versa. That way you won't have to come up with a special token or make huge changes to the existing code. Just add the escaping to the string sanitation step when importing, and convert it back when exporting.

I would try to implement it myself, but I'm not familiar with the site's API.

[ryukinix]

Yes, I'm pretty busy unfortunately, but this feature it's really cool. I don't have sure about the escape thing by \n, maybe we can find a better way later to add newlines. I'll keep that issue/feature as desirable

[bradenbest]

@ryukinix If it helps, this is what a quick experiment (python2.7) yielded for me:

>>> print "hello\nworld"
hello
world
>>> print "hello\nworld".replace("\n", "\\n") # importing (real newlines -> escape sequences)
hello\nworld
>>> print "hello\\nworld".replace("\\n", "\n") # exporting (escape sequences -> real newlines)
hello
world

It shouldn't be any different for 3.x, just replace "hello\nworld" with the hypothetical comment_field.value

[ryukinix]

This doesn't make sense to me, sorry. Why you want escape newlines as \\n? Where? For what? If you input \n on the desirable comment field of mal edit, the stored string with \n after print would work as expected.

Sorry if I'm missing your point.

[ryukinix]

There is a problem in implement that. I'm sorry to say that, but maybe this will not possible to implement.

Seems that mal API even saying that one of possible values has a comments field, however, there is no comments field response on XML when I retrieve it:

ref: https://myanimelist.net/malappinfo.php?u=lerax&status=all&type=anime

And yes, I have a comments field on Cowboy Bebop:

If none show that this way to fetch is wrong, I'll close this issue because we cannot do that.

[ryukinix]

Just to you don't being surprise again, yes, it's really what you think, MAL API is terrible!

[bradenbest]

@ryukinix

[on API not returning 'comments' field]

Weird. The documentation does explicitly say that comments is a field returned under "anime values".

MAL API is terrible!

Seems like it.

Why you want escape newlines as \n?

It's a measure of future-proofing. If you were to read to EOF, that would trap you into having comments be the very last field of the file, and complicate the addition of other multi-line fields. Escaping newlines is the simplest way to ensure that multi-line fields behave like single-line fields while preserving the freedom to use newlines. The user would be typing \n, and the app would be taking that escape sequence and transforming it into an actual newline (and to do that via a string replace, you have to represent these values in a string literal, and to do that, you have to escape it, hence \\n to represent \n (a literal \ followed by an n) and \n to represent a literal newline). This would reduce the complexity of the application, thus reducing the potential for bugs, and making for a better UI.

[bradenbest]

So I started playing around with curl requests based on the examples, and managed to get a manga to update with the comment "Hello World". I experimented with manga because I don't care about my tiny manga list and am okay with it getting destroyed/corrupted.

$ curl -u b1kmusic:(password) -d data="<?xml version='1.0' encoding='UTF-8'?><entry><comments>Hello World</comments></entry>" https://myanimelist.net/api/mangalist/update/24705.xml
Updated

Checking the page showed that the comments field did in fact update.

What I don't understand is how to retrieve data. The API documentation has nothing written about that. So I don't know how you figured out the malappinfo.php thing. That said, it's still possible to grab the data via web scraping. It's not pretty though. You have to authenticate the user through the site's login form, save the returned cookie, and send a new request on the user's "all anime/manga" page, find and execute the javascript that runs when "more" is clicked, and scrape out the text. Easy with a web browser and a pair of eyeballs, but extremely tedious when done procedurally. I've made HTML scrapers before, and it's an extremely ugly process. But it's something to keep in mind as a last resort until Xinil eventually decides to fix the API to allow reading data.

Though I fear he's running on Valve Time...

Perhaps it would be a llittle less tedious to scrape if you used the direct urls https://myanimelist.net/ownlist/manga/24705/edit. At that point you just need the auth token to be in the cookies, and then find and scrape out the text. I thought you'd need javascript, but I discovered it's just hidden with some CSS. I found this:

<table class="advanced" border="0" cellpadding="5" cellspacing="0" width="100%" style="display: none;">

Looking inside that element will give you this:

<tr>
    <td class="borderClass" valign="top">Comments</td>
        <td class="borderClass">
            <textarea id="add_manga_comments" name="add_manga[comments]" class="inputtext" rows="5" cols="45">Hello World</textarea>
...

Boom! There's the smoking gun! This is definitely possible.

Here's some helpful links for dealing with the two major issues (logging in and HTML scraping):

Requests library, for dealing with cookies: http://docs.python-requests.org/en/master/ BeautifulSoup4, a library for parsing HTML: https://readthedocs.org/projects/beautiful-soup-4/ A stackoverflow question about cookies: https://stackoverflow.com/questions/7164679/how-to-send-cookies-in-a-post-request-with-the-python-requests-library

A word of caution: don't hard code the exact location of the "comments" field in the HTML. Search for it every time. You never know when things will get moved around, making less assumptions will lead to less breakage the next time the site changes.

[ryukinix]

I don't have intention to add web crawling stuff because this means to relying on mal front-end which is a not reliable thing.

Thanks for all good intention here. You can freely fork this project and implement it for your own, I hope this package can be a good start for you.

But I don't have intention to merge this feature.

[lubien]

But it's something to keep in mind as a last resort until Xinil eventually decides to fix the API to allow reading data

Developers who use MAL API feel like