kaegi / MorphMan

Anki plugin that reorders language cards based on the words you know
Other
260 stars 66 forks source link

Last call for any interest in revival of this addon?? #255

Open nlovell1 opened 3 years ago

nlovell1 commented 3 years ago

Hello to anyone that is reading this. It seems this project has been abandoned and this is extremely frustrating for people that have put in work to improve this add-on, such as this one, that adds NLP framework support for more accurate parsing for MANY languages. It was largely functional but the pull has not been merged for months. @ianki @kaegi , is there any interest in maintaining? I think there are many people that would be interested in keeping up with this project, but it's impossible to contribute when no merges are being made.

Additionally, the pushes to AnkiWeb have been riddled with bugs rendering it, in cases, broken and unusable. I'd like to take the time to understand the code and try and troubleshoot but it's impossible to guarantee my changes would even be merged when maintainers have abandoned the project. Thanks.

kaegi commented 3 years ago

I'm not learning a language currently, so I'm also not using MorphMan/Anki. I hope that my code benefits other people (that's why I uploaded my changes to the original MorphMan), but maintaining the project, trying out merge requests, finding/fixing bugs, etc. is much time and work which I'd rather put into projects that I personally use.

A good compromise is that anyone that has sufficient interest in using MorphMan and maintaining the repository for a little bit (that is trying out whether merge requests work etc. when there is no one else available) can get the corresponding rights for it. This way this repository can be a stable place where people can get the latest changes.

I've been reading the titles of issues and merge requests. So if you (or anyone else reading this) feels up to the challenge, you can create an issue with MAINTAINER REQUEST I want to be maintainer of this project and I will give you the rights :)

nlovell1 commented 3 years ago

@kaegi Thank you very much for the reply and the information. To anyone reading this, I'd be happy to contribute, but it would probably make more sense for someone more experienced to take the lead. Thanks again

ghost commented 3 years ago

I've been using morphman for the last year and a half and it's been a real change in how I learn. Sadly I'm no programmer, but if I can offer any help whatsoever I'd be delighted to do so!

nlovell1 commented 3 years ago

@kaegi , as a novice, the code is a little hard to understand, and after reviewing some of the pull requests, it seems it might be a more difficult process to address them than I thought. Do you know of any past devs that might be able to spend a few moments helping me understand the code? This goes to anyone that might be reading, as well. Anything helps.

nlovell1 commented 3 years ago

@ianki @kaegi @rteabeault @yaoberh @landonepps is there any way someone could give me a quick write-up on how to set up for development? the read-me is a little hard to understand, and I don't know what the best way to debug would be (i.e, I want to see the value of a variable over time), or figure out what gets inputted into a function. I've made a few changes so far but it's hard to continue without debugging details

ghost commented 3 years ago

@thinkingbox12 Can't help you personally because I'm no programmer. There's a big community in reddit dedicated to japanese language learning with morphman where you could ask if no one answers you here. I've had so many problems with this addon to date and I've been fixing them manually after many hours battling with each of them. Totally worth it though.

ianki commented 3 years ago

Hi all, sorry I was pretty active last year on changes to MorphMan but I haven't had much time for this in the last few months, but I can take a look. Could you please summarize what are the top couple of things you want to fix / update?

There are steps for setting up the environment and building the UI files in the README.md

@thinkingbox12, the version on AnkiWeb should be working finde with Anki 2.1.35. Are you using a newer Anki version?

nlovell1 commented 3 years ago

@ianki thank you very much for the reply. To answer your questions,

Spacy:

  1. Spacy works pretty much fine, when it works. There are some issues with the readability analyzer and developing frequency lists because the output from SudachiPy is different than the output from MeCab. I forgot the details as I stopped trying to debug that a while ago, but I'd be happy to help if you have any questions.
  2. There are some problems with how the output of Spacy is passed back into MorphMan. I haven't spent enough time with the code, but if you take a look here and here, (which actually look like two separate problems but are actually not, the second only results because the terminal is not flushed after each press of the recalc button AFAIK), the mechanism in which output from Spacy is read from the terminal and parsed into a json breaks somewhere for whatever reason I haven't been able to figure out. I wonder if there is a better way, like to call Spacy as a subprocess rather than running another instance of a terminal and reading it from there. I would guess that this is also a slowdown for recalcs.

Edit- I'm about 80 percent sure this is due to an encoding issue, whether it be characters with accents in them for Spanish, or essentially any Japanese character. I believe I remember testing sentences specifically without non-ASCII glyphs and it seemed to work fine but the second I throw "más" or 「こんにちは」at it, it breaks.

  1. The version of spacy that reteaubault's PR works with is now outdated. Updating it to 3.0 breaks the way morphman reads available models, which was fixed with a trivial update to the regex that reads installed models from the terminal. Model links are also deprecated. There are no other problems as far as I can tell, because it works all the way up to the point where the results of the morphemizer is saved to a json, which I mentioned above.

Besides from the above problems, this PR should be fairly complete, unless @rteabeault has other input.

253 #252 #250 are some issues... I haven't really looked into whether they're actually bugs or user errors, but they're there, to say the least. Some users on AnkiWeb are reporting issues with 2021.2 release. I have been using 2.1.35 and 2021.1 without any major problems that I've been able to notice.

Going above and beyond, I'd most like to see support for being able to change the focus morph depending on the user's input. For example, say you know "I went to the park" with "park" as the original focus. Say you learned the word park somewhere, but for whatever reason "went" isn't sticking. I'd like to be able to change the focus morph based on user input that would take this into account when scheduling.

Secondarily, I'm working on an experimental way to detect when a word appears in a multi-word expression (thru Spacy), for example "floats your boat". I would like to see some way for Morphman to record this information and to have scheduling options.. perhaps "Only show figures of speech when all subwords are mature" or something like that. This would likely be handled in spacy, and with the joy of user-created models, you can basically develop a new 'type' and store words under a category. Then, all MMan has to do is, for example, extract the "MWE" category and store it in a db with special information with it.

The latest two pararaphs are undoubtedly difficult to implement but I'm interested in making it happen. Besides setting up pyenv, how does one go about debugging? I've just been raising exeptions to find the value of things, but to see the values of objects as they pass through the code, seeing the terminal that Anki uses to interact with Spacy, all these things would be vry helpful to see, but I'm unable to do that right now. What do you (and presumably other developers) do in your workflow to go about development other than what's in the readme?

Thank you very much. I want to be as helpful here as possible so please let me know how we can make this happen.

ashprice commented 3 years ago

I realize this might not be the best place to write this, but if there are plans to add the ability to change the morph, something I'd really appreciate is the ability to hide all cards that have more than a certain number of unlearned target morphs - so ie. if there are more than (say) 2 target morphs, the card is buried/doesn't come up with other new cards. (I know there aren't really 'plans,' but if other people are putting suggestions out there, I feel like I might too.)

This should be easier to implement than the morph-focus feature, I would say. Maybe I can even implement it myself given some time, but I too am not a programmer.

nlovell1 commented 3 years ago

@ianki , have you been able to take a look? I understand people's personal priorities might have strayed away from Morphman, but again, I'm happy to take on the load. I've tried debugging on my own for a few of the problems and it seems the issues are more deep than I thought.

RawToast commented 3 years ago

I'm not learning a language currently, so I'm also not using MorphMan/Anki. I hope that my code benefits other people (that's why I uploaded my changes to the original MorphMan), but maintaining the project, trying out merge requests, finding/fixing bugs, etc. is much time and work which I'd rather put into projects that I personally use.

A good compromise is that anyone that has sufficient interest in using MorphMan and maintaining the repository for a little bit (that is trying out whether merge requests work etc. when there is no one else available) can get the corresponding rights for it. This way this repository can be a stable place where people can get the latest changes.

It would be nice if one of the projects leeching off MorphMan would've actually committed back to this. There are many Japanese learners using Morphman and have been for years, so you might be able to get Patreon funding or something to at least maintain the add-on (no new features, just keep it running).

I could request maintainer rights, but all I would be capable of doing is: fixing crashes caused by version updates and reviewing the simple bug fix issues. I don't have the time to do more, such as go through the Spacey stuff, so IDK if that's helpful.

As an FYI, looks like you could safely merge #141 and #242

joestrong commented 2 years ago

I'm also happy to have a go at contributing, I did a few simple PRs last year and would be up for trying a few more. However, I also do not know the source code well enough to take any lead.

khanguyenwk commented 2 years ago

I'm a user of this add-on and I can say that this is a game changer. I really appreciate for people who have been spending time and effort to create it. I would love to donate and I hope someone will continue to maintain it (and add some new language like Korean, ...)

nlovell1 commented 2 years ago

Hello all, I've gotten more time in my schedule and am interested in doing what needs to be done to lift this out of the ditch. Unfortunately, I'm not experienced with projects of this size, so I'd need some catching up and tips on how to familiarize myself with the codebase, as I'm a pretty novice coder. @RawToast @joestrong , would either of you be willing or able to collaborate on this semi-regularly?

joestrong commented 2 years ago

@thinkingbox12 I don't know the codebase that well, or have a massive amount of spare time currently. I can probably chip in with minor fixes here and there however.

I would love to contribute a bit more substantially, but I can't promise anything yet. I'd also be interested in tips to catch up with the codebase, if anyone has any notes or tips.

ianki commented 2 years ago

The latest two pararaphs are undoubtedly difficult to implement but I'm interested in making it happen. Besides setting up pyenv, how does one go about debugging? I've just been raising exeptions to find the value of things, but to see the values of objects as they pass through the code, seeing the terminal that Anki uses to interact with Spacy, all these things would be vry helpful to see, but I'm unable to do that right now. What do you (and presumably other developers) do in your workflow to go about development other than what's in the readme?

First of all, the Readability Analyzer has performs "hot-reload" every time it's opened, so when hacking on code that's around the Analyzer it's quite a quick cycle to code and test.

To be able to debug, you can run Anki from a Python interpreter.

run_anki.py:

import aqt
aqt.run()

If you have a debugging environment e.g. Visual Studio Code you can debug starting with this script, and set breakpoints in your add-on code. Hope that helps.