Problem: Transfer browser doesn't encode paths with UTF-8

Expected behaviour

Browsing a directory hierarchy in the transfer browser supports unicode characters.

Current behaviour

If a directory name in the directory hierarchy contains unicode characters the transfer browser doesn't show the contents of the directory and the dashboard service logs an error like this:

ERROR     2024-01-09 15:07:10  django.request:log:log_response:224:  Internal Server Error: /filesystem/children/location/55893dc3-e9b5-4dcb-bdc3-a65441a5f243/
Traceback (most recent call last):
  File "/pyenv/data/versions/3.9.18/lib/python3.9/site-packages/django/core/handlers/exception.py", line 47, in inner
    response = get_response(request)
  File "/pyenv/data/versions/3.9.18/lib/python3.9/site-packages/django/core/handlers/base.py", line 181, in _get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
  File "/src/src/dashboard/src/components/filesystem_ajax/views.py", line 118, in directory_children_proxy_to_storage_server
    path = path + b64decode_string(request.GET.get("path", ""))
  File "/src/src/archivematicaCommon/lib/archivematicaFunctions.py", line 118, in b64decode_string
    return base64.b64decode(data.encode("utf8")).decode("utf8")
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xe9 in position 66: invalid continuation byte

It looks like the browse.service of the transfer browser encodes unicode paths in this scenario using the latin1 encoding but the filesystem/children/location view expects them as utf-8. This limitation is explained in the Unicode strings section of the btoa function documentation which also includes potential solutions.

Steps to reproduce

Copy the SampleTransfers/DemoTransferCSV directory as SampleTransfers/Montréal in the default transfer source location.
From the Transfer tab in the dashboard browse the default transfer source location and try to expand the contents of the SampleTransfer/Montréaldirectory.
The contents of the directory are not displayed and you should get the above error in the dashboard logs.

Your environment (version of Archivematica, operating system, other relevant details)

https://github.com/artefactual/archivematica/commit/43328a1756ae1235b2e6c0610527141db272d6f9

For Artefactual use:

Before you close this issue, you must check off the following:

[ ] All pull requests related to this issue are properly linked
[ ] All pull requests related to this issue have been merged
[ ] A testing plan for this issue has been implemented and passed (testing plan information should be included in the issue body or comments)
[ ] Documentation regarding this issue has been written and merged (if applicable)
[ ] Details about this issue have been added to the release notes (if applicable)

archivematica / Issues

Problem: Transfer browser doesn't encode paths with UTF-8 #1652