Adding support for Cloze Overlapping cards

[RoseWebb]

It would be very useful to have support for cloze overlapping cards.

Card type can be defined in the property drawer. Perhaps by prompting the user to chose a card type when the heading have mutable items.

Here is an example based on this issue in a similar project

* title
:PROPERTIES:
:ANKI_NOTE_ID: 1675532440856
:ANKI_DECK: example_deck
:ANKI_CARDTYPE: cloze (overlapping)
:END:
- item 1
- item 1
- item 1

example jason for anki-connect:

{
    "action": "addNote",
    "version": 6,
    "params": {
        "note": {
            "deckName": "example_deck",
            "modelName": "Cloze (overlapping)",
            "tags": [],
            "fields": {
                "Title": "title",
                "Full": "<div>{{c21::Item 1}}</div><div>{{c21::Item 1}}</div><div>{{c21::Item 1}}</div>",
                "Text2": "<div>Item 1</div><div>{{c2::Item 1}}</div><div>...</div>",
                "Text3": "<div>...</div><div>Item 1</div><div>{{c3::Item 1}}</div>",
                "Text1": "<div>{{c1::Item 1}}</div><div>...</div><div>...</div>",
                "Settings": "1,1,0 | n,n,n,n",
                "Original": "Item 1<div>Item 1</div><div>Item 1</div>",
                "Sources": ""
            }
        }
    }
}

[RoseWebb]

I created a temporary solution by running a shell command on region usually bound to M-| from Emacs to a Python script

I will paste it here in case someone might find it useful. Most of it is from the original anki addon:

python code

```python #!/usr/bin/env python3 import sys import requests import json import orgparse import sys import generate_cloze from typing import Match import re from operator import itemgetter from itertools import groupby from html.parser import HTMLParser reComment = re.compile("(?s)") reStyle = re.compile("(?si).*?") reScript = re.compile("(?si).*?") reTag = re.compile("(?s)<.*?>") reEnts = re.compile(r"&#?\w+;") reMedia = re.compile("(?i)]+src=[\"']?([^\"'>]+)[\"']?[^>]*>") class HTMLFilter(HTMLParser): text = '' def handle_data(self, data): self.text += data def stripHTML(s: str) -> str: s = reComment.sub("", s) s = reStyle.sub("", s) s = reScript.sub("", s) s = reTag.sub("", s) s = entsToTxt(s) return s def entsToTxt(html: str) -> str: # entitydefs defines nbsp as \\xa0 instead of a standard space, so we # replace it first html = html.replace(" ", " ") def fixup(m: Match) -> str: text = m.group(0) if text[:2] == "&#": # character reference try: if text[:3] == "&#x": return chr(int(text[3:-1], 16)) else: return chr(int(text[2:-1])) except ValueError: pass else: # named entity try: text = chr(name2codepoint[text[1:-1]]) except KeyError: pass return text # leave as is return reEnts.sub(fixup, html) def parseNoteSettings(html): """Return note settings. Fall back to defaults if necessary.""" options, settings, opts, sets = None, None, None, None field = stripHTML(html) lines = field.replace(" ", "").split("|") #print(lines) if not lines: #print(dflt_set, dflt_opt) return (dflt_set, dflt_opt) settings = lines[0].split(",") if len(lines) > 1: options = lines[1].split(",") if not options and not settings: return (dflt_set, dflt_opt) if not settings: sets = dflt_set else: sets = [] for idx, item in enumerate(settings[:3]): try: sets.append(int(item)) except ValueError: sets.append(None) length = len(sets) if length == 3 and isinstance(sets[1], int): pass elif length == 2 and isinstance(sets[0], int): sets = [sets[1], sets[0], sets[1]] elif length == 1 and isinstance(sets[0], int): sets = [dflt_set[0], sets[0], dflt_set[2]] else: sets = dflt_set if not options: opts = dflt_opt else: opts = [] for i in range(4): try: if options[i] == "y": opts.append(True) else: opts.append(False) except IndexError: opts.append(dflt_opt[i]) #print(sets, opts) return (sets, opts) #def createNoteSettings(setopts): #"""Create plain text settings string""" #set_str = ",".join(str(i) if i is not None else "all" for i in setopts[0]) #opt_str = ",".join("y" if i else "n" for i in setopts[1]) #return set_str + " | " + opt_str class ClozeOverlapper(object): """Reads note, calls ClozeGenerator, writes results back to note""" creg = r"(?s)\[\[oc(\d+)::((.*?)(::(.*?))?)?\]\]" def __init__(self, setopts="1,1,0 | n,n,n,n", maxfields = 20, markup=False, silent=False): self.markup = markup self.silent = silent self.setopts = setopts self.maxfields = maxfields def add(self, original): """Add overlapping clozes to note""" matches = re.findall(self.creg, original) if matches: custom = True formstr = re.sub(self.creg, "{{\\1}}", original) items, keys = self.getClozeItems(matches) else: custom = False formstr = None items, keys = self.getLineItems(original) #setopts = parseNoteSettings("1,1,0 | n,n,n,n") gen = ClozeGenerator(parseNoteSettings(self.setopts), self.maxfields) fields, full, total = gen.generate(items, formstr, keys) return fields, full, total def getClozeItems(self, matches): """Returns a list of items that were clozed by the user""" matches.sort(key=lambda x: int(x[0])) groups = groupby(matches, itemgetter(0)) items = [] keys = [] for key, data in groups: phrases = tuple(item[1] for item in data) keys.append(key) if len(phrases) == 1: items.append(phrases[0]) else: items.append(phrases) return items, keys def getLineItems(self, html): """Detects HTML list markups and returns a list of plaintext lines""" f = HTMLFilter() f.feed(html) text = f.text # self.markup = "div" # remove empty lines: lines = re.sub(r"^( )+$", "", text, flags=re.MULTILINE).splitlines() items = [line for line in lines if line.strip() != ''] return items, None def processField(self, field): """Convert field contents back to HTML based on previous markup""" markup = self.markup if markup == "div": tag_start, tag_end = "", "" tag_items = "

{0}

" else: tag_start = '<{0}>'.format(markup) tag_end = ''.format(markup) tag_items = "

{0}

" lines = "".join(tag_items.format(line) for line in field) return tag_start + lines + tag_end class ClozeGenerator(object): """Cloze generator""" cformat = "{{c%i::%s}}" def __init__(self, setopts, maxfields): self.maxfields = maxfields self.before, self.prompt, self.after = setopts[0] self.options = setopts[1] self.start = None self.total = None def generate(self, items, original=None, keys=None): """Returns an array of lists with overlapping cloze deletions""" length = len(items) # print(self.before, self.prompt, self.after) if self.prompt > length: return 0, None, None if self.options[2]: self.total = length + self.prompt - 1 self.start = 1 else: self.total = length self.start = self.prompt if self.total > self.maxfields: return None, None, self.total fields = [] for idx in range(self.start, self.total+1): snippets = ["..."] * length start_c = self.getClozeStart(idx) start_b = self.getBeforeStart(idx, start_c) end_a = self.getAfterEnd(idx) if start_b is not None: snippets[start_b:start_c] = self.removeHints( items[start_b:start_c]) if end_a is not None: snippets[idx:end_a] = self.removeHints(items[idx:end_a]) snippets[start_c:idx] = self.formatCloze( items[start_c:idx], idx-self.start+1) field = self.formatSnippets(snippets, original, keys) fields.append(field) nr = len(fields) if self.maxfields > self.total: # delete contents of unused fields fields = fields + [""] * (self.maxfields - len(fields)) fullsnippet = self.formatCloze(items, self.maxfields + 1) full = self.formatSnippets(fullsnippet, original, keys) return fields, full, nr def formatCloze(self, items, nr): """Apply cloze deletion syntax to item""" res = [] for item in items: if not isinstance(item, (list, tuple)): res.append(self.cformat % (nr, item)) else: res.append([self.cformat % (nr, i) for i in item]) return res def removeHints(self, items): """Removes cloze hints from items""" res = [] for item in items: if not isinstance(item, (list, tuple)): res.append(item.split("::")[0]) else: res.append([i.split("::")[0] for i in item]) return res def formatSnippets(self, snippets, original, keys): """Insert snippets back into original text, if available""" html = original if not html: return snippets for nr, phrase in zip(keys, snippets): if phrase == "...": # placeholder, replace all instances html = html.replace("{{" + nr + "}}", phrase) elif not isinstance(phrase, (list, tuple)): html = html.replace("{{" + nr + "}}", phrase, 1) else: for item in phrase: html = html.replace("{{" + nr + "}}", item, 1) return html def getClozeStart(self, idx): """Determine start index of clozed items""" if idx < self.prompt or idx > self.total: return 0 return idx-self.prompt # looking back from current index def getBeforeStart(self, idx, start_c): """Determine start index of preceding context""" if (self.before == 0 or start_c < 1 or (self.before and self.options[1] and idx == self.total)): return None if self.before is None or self.before > start_c: return 0 return start_c-self.before def getAfterEnd(self, idx): """Determine end index of following context""" left = self.total - idx if (self.after == 0 or left < 1 or (self.after and self.options[0] and idx == self.start)): return None if self.after is None or self.after > left: return self.total return idx+self.after org_data = orgparse.loads('''* title :PROPERTIES: :ANKI_NOTE_ID: 1675532440856 :ANKI_DECK: Default :ANKI_CARDTYPE: Cloze (overlapping) :END: - item 1 - item 2 - item 3''') org_data = sys.stdin.read() org_data = orgparse.loads(org_data) data = { "action": "addNote", "version": 6, "params": { "note": { "deckName": "Default", "modelName": "Cloze (overlapping)", "tags": [], "fields": { "Title": "", "Settings": "1,1,0 | n,n,n,n", "Original": "Item 1

Item 1

Item 1

", } } } } data['params']['note']['fields']['Title'] = org_data.children[0].heading data['params']['note']['modelName'] = org_data.children[0].get_property('ANKI_CARDTYPE') data['params']['note']['deckName'] = org_data.children[0].get_property('ANKI_DECK') # cloze items pretify and add original = org_data.children[0].body.replace('- ','') data['params']['note']['fields']['Original'] = ''.join([f'

{i}

' for i in original.split('\n')]) generate_cloze = ClozeOverlapper('1,1,0 | n,n,n,n', 20) cloze, fullcloze,ignore = generate_cloze.add(original) data['params']['note']['fields'].update({k:''.join([f'

{i}

' for i in v]) for k,v in zip([f"Text{i}" for i in range(1, len(cloze)+1)],cloze)}) data['params']['note']['fields'].update({'Full':''.join([f'

{i}

' for i in fullcloze])}) r = requests.post('http://localhost:8765', data=json.dumps(data)) print(r.text + "\n") ```

[eyeinsky]

Thanks, will try to look into this soon! :) Have been swamped at work..

[eyeinsky]

Hi @RoseWebb, could you also add an example org entry that would be converted into an overlapping Cloze, such that I would understand what the org source text looks like? There is an org example in the very first comment, but it doesn't have any {{..}} in it -- how would we know which fields to hide?

[eyeinsky]

This is the Anki extension that this would support, right: https://ankiweb.net/shared/info/969733775 ?

[RoseWebb]

Hi @eyeinsky, sorry for the late reply and thanks for looking into this issue.

Yes, you are linking to the right Anki addon:

• Ankiweb: https://ankiweb.net/shared/info/969733775.
• Github: https://github.com/glutanimate/cloze-overlapper However, I couldn't install it straightaway from Ankiweb. Instead, I had to build it from the source following the GitHub instructions which I found to be straightforward: https://github.com/glutanimate/cloze-overlapper#building

This addon works by taking a list of items and user defined settings to generate mutable cloze cards. The default setting is to show one item before the cloze prompt for each list item and one card asking for the full list. Therefore, for a list of 3 items, it will create 4 cloze cards. Here is a video explanation from the addon developer:

• overview: https://youtu.be/ag4Hdnkig8U?t=8
• settings: https://youtu.be/ag4Hdnkig8U?t=685

for an org example using the default setting, I suggest something like:

* title
:PROPERTIES:
:ANKI_NOTE_ID: 1675532440856
:ANKI_DECK: example_deck
:ANKI_CARDTYPE: cloze (overlapping)
:ANKI_CLOZE: 1,1,0 | n,n,n,n
:END:
- item 1
- item 2
- item 3

which the addon will convert into 4 cards: card 1:

title
{{c1::- item 1}}
...
...

card 2

title
- item 1
{{c2::- item 2}}
...

card 3

title
...
- item 2
{{c3::- item 3}}

card 4

title
{{c20::- item 1}}
{{c20::- item 2}}
{{c20::- item 3}}

[RoseWebb]

I found a project implementing the cloze overlapping concept into the card template itself via JS. Therefore, there is no need to support the original cloze overlapper addon. https://github.com/michalrus/anki-simple-cloze-overlapper/blob/main/front.html

One could use multi line cloze in the org file, then change the note type from Anki. The note in the org file can also be updated with no issues note the card is set to this template in Anki.

[eyeinsky]

Is there any way this could be made more convenient? :)

Currently there is a hardcoded test org-anki--is-cloze which changes note type to Cloze if it finds cloze syntax in org entry's title or body. Maybe this test could be customized such that a user could define their own test-fn/note-type pairs -- this way you wouldn't need to go and manually change every single note.