Move table template code to separate HTML files

[jeremystretch]

Proposed Changes

Move all Django template code used to render table columns into discrete HTML files. For example, the IP addresses table has an address column which references the following template code, stored in a variable:

IPADDRESS_LINK = """
{% if record.pk %}
    <a href="{{ record.get_absolute_url }}" id="ipaddress_{{ record.pk }}">{{ record.address }}</a>
{% elif perms.ipam.add_ipaddress %}
    <a href="{% url 'ipam:ipaddress_add' %}?address={{ record.1 }}{% if object.vrf %}&vrf={{ object.vrf.pk }}{% endif %}{% if object.tenant %}&tenant={{ object.tenant.pk }}{% endif %}" class="btn btn-sm btn-success">{% if record.0 <= 65536 %}{{ record.0 }}{% else %}Many{% endif %} IP{{ record.0|pluralize }} available</a>
{% else %}
    {% if record.0 <= 65536 %}{{ record.0 }}{% else %}Many{% endif %} IP{{ record.0|pluralize }} available
{% endif %}
"""

This code, and similar blobs, would be moved to its own HTML file to be included by the table.

Justification

This better separates template code from application logic, and provides a cleaner approach to working with the template content.

We have experimented with this approach in the past and encountered performance issues, although that was years ago. We'll need to do some testing to ensure that the proposal is in fact viable.

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. NetBox is governed by a small group of core maintainers which means not all opened issues may receive direct feedback. Do not attempt to circumvent this process by "bumping" the issue; doing so will result in its immediate closure and you may be barred from participating in any future discussions. Please see our contributing guide.

[arthanson]

Some code for timing these - bash script to call Django logged in page:

LOGIN_URL='http://localhost:8000/login/'
YOUR_USER='xxx'
YOUR_PASS='xxx'
COOKIES=cookies.txt
CURL_BIN="curl -s -c $COOKIES -b $COOKIES -e $LOGIN_URL"

echo -n "Django Auth: get csrftoken ..."
$CURL_BIN $LOGIN_URL > /dev/null
DJANGO_TOKEN="csrfmiddlewaretoken=$(grep csrftoken $COOKIES | sed 's/^.*csrftoken[[:space:]]*//')"

echo -n " perform login ..."
$CURL_BIN \
    -d "$DJANGO_TOKEN&username=$YOUR_USER&password=$YOUR_PASS" \
    -X POST $LOGIN_URL

echo -n " calling IP Address list ..."
$CURL_BIN \
    -w "@curl-format.txt" -o /dev/null \
    -d "$DJANGO_TOKEN&..." \
    -X GET http://localhost:8000/ipam/ip-addresses/?per_page=1000

echo " logout"
rm $COOKIES

curl-format.txt:

     time_namelookup:  %{time_namelookup}s\n
        time_connect:  %{time_connect}s\n
     time_appconnect:  %{time_appconnect}s\n
    time_pretransfer:  %{time_pretransfer}s\n
       time_redirect:  %{time_redirect}s\n
  time_starttransfer:  %{time_starttransfer}s\n
                     ----------\n
          time_total:  %{time_total}s\n

[jeremystretch]

This can be greatly simplified by adding 'ipam.ipaddress' to EXEMPT_VIEW_PERMISSIONS, which obviates the need for authentication.

Also, the IP addresses list is fairly unique in that it invoked additional logic to annotate available space among IPs, which induces additional overhead. We should select a different object for a cleaner evaluation of the change's impact.

[arthanson]

Interfaces has three templated columns - with inline code 200 records ~0.9s, with template files ~15.4s, also had some weird very long timings when I first switched to the template files but wasn't able to get those again even trying to clear any potential caches. Over 15x slower using the template files.

With per_page=1000 inline=3.8s template files=78s so it's not a fixed overhead as this is 20x slower. Looks like django-tables2 is reinitializing the template each row or something. So this is a no-go for now, will look at a repo scenario and file a bug/FR in django-tables2.

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. NetBox is governed by a small group of core maintainers which means not all opened issues may receive direct feedback. Do not attempt to circumvent this process by "bumping" the issue; doing so will result in its immediate closure and you may be barred from participating in any future discussions. Please see our contributing guide.

[arthanson]

Tried a demo program to show the issue, but actually the templated columns in separate HTML file is slightly faster. Will need to try one line at a time in the NB app to narrow down what is causing the slowness. mysite.zip