Render math latex keyboard in Django template

bbelderbos commented 6 months ago

I think this will involve Javascript / some front-end work ...

(ChatGPT does not let you share a thread that contains images)

So maybe we look at this? http://mathquill.com

anschelburk commented 6 months ago

Appreciate this. I also did a search for API's that might support this, and came up with this web site. Do you think this could help?

https://cortexjs.io/mathlive/

bbelderbos commented 6 months ago

Oh interesting, that seems even an abstraction higher that can save us a lot of work, I will play with this a bit tomorrow ..

Reminds me of the codechalleng.es-platform-building days where we needed a robust code editor and I found Ace ->

Don't re-invent the wheel!

bbelderbos commented 6 months ago

@anschelburk oK cool, great find, that plugin works:

Template html:

<!DOCTYPE html>
<html>
<head>
    <title>Math Editor</title>
    <script defer src="//unpkg.com/mathlive"></script>

</head>
<body>

  <math-field id="mathfield"></math-field>

  <script src="https://unpkg.com/mathlive/dist/mathlive.min.js"></script>
  <script>
      document.addEventListener("DOMContentLoaded", function() {
          const mathFieldElement = document.getElementById('mathfield');
          mathFieldElement.addEventListener('input', () => {
              console.log(mathFieldElement.value); // This will log the LaTeX content
          });
      });
  </script>

</body>
</html>

ChatGPT thread: https://chat.openai.com/share/d1853701-1273-44a7-a208-0a69d42218e1 (here you also see how I use AI tools like this ...)

So now it's just a matter of sending mathFieldElement.value back to the back-end / Django to store it in the database (alongside other things related to the assignment form), still some stuff to figure out here, but this is a nice validation that we don't have to code this keyboard ourselves, nor have to deal with npm / JS packaging, we can just include the CDN library links like above (phew!)

anschelburk commented 5 months ago

I have no idea why, but the math keyboard seems to be working now! (Maybe it was missing a dependency, and it happened to be one of the ones we installed?)

Going to work on customizing this keyboard while you're taking a look at the Postgres issue. Appreciate everything.

bbelderbos commented 5 months ago

Excellent, because it did not make sense why it would not work, the only dependency really was the JS library we linked, so maybe a browser caching or temporary rendering issue. Great news!

bbelderbos commented 5 months ago

Now the next step is to integrate it into the form right so that what the user selects using the keyboard gets submitted in that form to the database, right?

anschelburk commented 5 months ago

Now the next step is to integrate it into the form right so that what the user selects using the keyboard gets submitted in that form to the database, right?

Yes - I want to wait to try and submit the LaTex/JSON code until we can get Postgres working, though in the meantime, now that the basic keyboard seems to be working, I definitely want to try and integrate it into the assignment_template.html template, and then to try and customize it a little.

As a first step, I've successfully refactored the code of the base_site and two child templates so that they extend the parent template correctly. (As part of this, I learned about using multiple {% block %} tags with different labels, so that the appropriate code from each child template is inserted into the <head> and <body> of the base_site.html template.)

Updated code is pushed to GitHub. Going to work on integrating the math keyboard into assignment_form.html now.

bbelderbos commented 5 months ago

Awesome, happy to see you're able to figure out more things and faster, great progress, keep going!

anschelburk commented 5 months ago

Did some research this morning into the Math Keyboard. Found some tools which I think will be useful once the math keyboard is successfully integrated into the assignment_form.html template:

Customizing the virtual (on-screen) keyboard
Customizing the keybindings associated with that virtual keyboard
For reference, also found their default keybindings

Separately, I also found this resource for adding horizontal and vertical spacing in LaTex, and these two resouces (#1, #2) for adding horizontal lines. Planning to add these later to the virtual keyboard, and as custom keybindings. (Intuitively, this may seem like a side feature, but it's actually a crucial feature for this app - if you're in a basic Algebra class, and need to subtract a constant from both sides of an equation, you need a way to align the numbers correctly that you're subtracting):

(It's an interesting validation of this that even in GitHub, I couldn't get even this simple equation to align correctly, and ended up needing to take a screenshot. That difficulty for many students - particularly dyslexic students - is why I'm creating this app. In my experience teaching, this is pretty representative of most word processing software out there, and is why dyslexic students - despite using word processing software for most assignments - almost never use it for math.)

For reference, here are GitHub's attempts to render what I wrote above - first without the three back apostraphes, then with them:

Mostly putting this here so we can find it later. In the meantime, I'm reading through a tutorial MathLive has to see how I can integrate the keyboard into the assignment_form.html template, and capture the LaTex output code. (I'm spending less time on this than the Django tutorial, since unlike the Django docs, I probably don't need everything that they're describing for my app. Instead, I'm selectively trying to find the things I need.)

Let me know if you have any other suggestions. Appreciate it.

bbelderbos commented 5 months ago

That's such a cool idea and noble cause, thanks for sharing that background, interesting!

bbelderbos commented 5 months ago

Thanks for that resource, I pasted it into ChatGPT and combined my experience to get some good hints how you can write a bit of Javascript code to populate the django form so you can save the entered input into the db.

As we don't want to necessarily repeat the keyboard input in the Django form (show twice to the user), I had it update the ModelForm code as well adding a widget to make the corresponding form field a "hidden field" so it does not show to the user, while it's still submitted to the back-end (views.py).

Hope this makes kind of sense and that it helps you progress this before we meet again.

Saved conversation: https://chat.openai.com/share/acc85ad7-e4d9-4b9f-8c57-e15ca0e6f839

anschelburk commented 5 months ago

Saved conversation: https://chat.openai.com/share/acc85ad7-e4d9-4b9f-8c57-e15ca0e6f839

Thank you. I've taken a look through the conversation, and am now going to try and execute the steps this afternoon. I'll post questions as I go. Appreciate it.

anschelburk commented 5 months ago

Going back to this - I spent yesterday going back through your tutorial and CortexJS's tutorial. The thing that's still a bit confusing for me is understanding how to implement a hidden field, which I've spent this morning looking into.

In theory, hidden fields seem pretty straightforward. In practice, I'm running into some questions around them. I've been researching them this morning, but am still running into some points of confusion, and think they would be a good topic for us to talk about on our call:

Are hidden fields separate from other fields? If so, why? (In our case, wouldn't that just double the information being sent to the server? It seems inefficient.)
If they are separate, does that mean I need to define two separate fields in my MathAssignment model? If so, how are those fields connected?
- In theory, your tutorial mentioned an event listener that helps with this, but from what I can tell, event listeners just seem to look for events and respond to them. (Inherently, they don't necessarily to mirror the input of one field.) Would love to talk about this on our call.
Also, one other question - so far, we've been adding the math keyboard as a <script> that loads after the end of the template's <body>. But that means, structurally, it's not integrated into the Django field we need. I'm having some trouble understanding how to integrate that <script> into the appropriate Django field. Maybe we could talk about this on our call?

anschelburk commented 5 months ago

Holy smokes - just found this. Playing with it now, but I wonder if it can help!

anschelburk commented 5 months ago

OK - tried installing and adding django-mathfield, but got an error. Just pushed my code for reference. Here's the traceback:

[19/Apr/2024 15:19:18] "GET /assignment/ HTTP/1.1" 200 2513
/Users/anschelburk/Documents/Pybites/Math-Homework/mysite/mysite/settings.py changed, reloading.
Watching for file changes with StatReloader
Exception in thread django-main-thread:
Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/threading.py", line 1073, in _bootstrap_inner
    self.run()
  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/threading.py", line 1010, in run
    self._target(*self._args, **self._kwargs)
  File "/Users/anschelburk/Documents/Pybites/Math-Homework/venv/lib/python3.12/site-packages/django/utils/autoreload.py", line 64, in wrapper
    fn(*args, **kwargs)
  File "/Users/anschelburk/Documents/Pybites/Math-Homework/venv/lib/python3.12/site-packages/django/core/management/commands/runserver.py", line 125, in inner_run
    autoreload.raise_last_exception()
  File "/Users/anschelburk/Documents/Pybites/Math-Homework/venv/lib/python3.12/site-packages/django/utils/autoreload.py", line 87, in raise_last_exception
    raise _exception[1]
  File "/Users/anschelburk/Documents/Pybites/Math-Homework/venv/lib/python3.12/site-packages/django/core/management/__init__.py", line 394, in execute
    autoreload.check_errors(django.setup)()
  File "/Users/anschelburk/Documents/Pybites/Math-Homework/venv/lib/python3.12/site-packages/django/utils/autoreload.py", line 64, in wrapper
    fn(*args, **kwargs)
  File "/Users/anschelburk/Documents/Pybites/Math-Homework/venv/lib/python3.12/site-packages/django/__init__.py", line 24, in setup
    apps.populate(settings.INSTALLED_APPS)
  File "/Users/anschelburk/Documents/Pybites/Math-Homework/venv/lib/python3.12/site-packages/django/apps/registry.py", line 91, in populate
    app_config = AppConfig.create(entry)
                 ^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/anschelburk/Documents/Pybites/Math-Homework/venv/lib/python3.12/site-packages/django/apps/config.py", line 193, in create
    import_module(entry)
  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/importlib/__init__.py", line 90, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<frozen importlib._bootstrap>", line 1387, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1360, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1331, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 935, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 995, in exec_module
  File "<frozen importlib._bootstrap>", line 488, in _call_with_frames_removed
  File "/Users/anschelburk/Documents/Pybites/Math-Homework/venv/lib/python3.12/site-packages/mathfield/__init__.py", line 1, in <module>
    from mathfield.api import *
  File "/Users/anschelburk/Documents/Pybites/Math-Homework/venv/lib/python3.12/site-packages/mathfield/api.py", line 7, in <module>
    from django.utils.encoding import smart_unicode
ImportError: cannot import name 'smart_unicode' from 'django.utils.encoding' (/Users/anschelburk/Documents/Pybites/Math-Homework/venv/lib/python3.12/site-packages/django/utils/encoding.py)

bbelderbos commented 5 months ago

Great session, that plugin was way too outdated, but happy we got our initial spiffy keyboard now hooked up with the django back-end :) https://github.com/anschelburk/Math-Homework/pull/30/files

anschelburk commented 5 months ago

Indeed! Really excited to have that working. As a next step, I spent the last few days modifying the `math_keyboard.html' template to render a custom keyboard, using this guide from CortexJS. It's not quite done yet - I want to play a little bit with tabs, and a "shift" button, and then to make sure that each key has a simple, dyslexia-friendly key binding - but I've made a pretty solid first draft. It's pushed to GitHub, under the assignment_template_with_latex_keyboard branch.

For our meeting tomorrow, here are a few things I would love to talk about, that would help me continue working on this app:

The pull you request you submitted has some "conflicts that must be resolved". Since this is likely to come up in a software job environment, I'd love to talk through what this means tomorrow, and what to do when I encounter something like this.
I'd love to talk about tools I can use to refactor Django template code. Can I, for example, abstract the keyboard to a different module, and import it somehow to assignment_form.html? Is there a tutorial you would recommend for creating a separate CSS template file? (And even if I did, would that be for the keyboard, the assigmment_form.html template page, or both?)
I'd love to begin talking about testing. I'm not sure if this is the right moment to begin writing Django tests, but given your code clinic two weeks ago, I want to just check in about when might be the right time to start doing this.

Looking forward to talking tomorrow!

bbelderbos commented 5 months ago

OK sounds good!

Yes this happens a lot so I let you show the scenario and we go through it.
I think you can keep one css template in static/ (I would make a css subfolder), which you link in the base.html parent template file. This way it's available to all future child templates.
Bit early yet, but yes I do want to cover this towards the end of this app, although we could also introduce it with app 3 because, assuming that's python only (so no django wrapper), it might be easier to get up2speed with it.

Talk later ...

Btw I bought a new macbook air this week, love it, very smooth!

bbelderbos commented 5 months ago

I think the math keyboard is done for now, just need to be able to render multiple on assignment submit page ...

anschelburk / math_homework

Render math latex keyboard in Django template #4