search

mediacms-io / mediacms

MediaCMS is a modern, fully featured open source video and media CMS, written in Python/Django and React, featuring a REST API.
https://mediacms.io
GNU Affero General Public License v3.0
2.86k stars 533 forks source link

API Upload Issue with Russian Language #1079

Closed habibulilalbaab closed 1 month ago

habibulilalbaab commented 1 month ago

Describe the issue When uploading media with Russian language content (e.g., metadata or text), the server consistently returns a 500 Internal Server Error. This issue occurs specifically when the media contains Russian text, and normal uploads without Russian content are processed successfully.

To Reproduce Steps to reproduce the issue:

  1. Go to the media upload endpoint.
  2. Upload media with Russian title metadata.
  3. Internal Server Error 500.
Internal Server Error: /api/v1/media/
Traceback (most recent call last):
  File "/home/mediacms.io/lib/python3.10/site-packages/django/core/handlers/exception.py", line 55, in inner
    response = get_response(request)
  File "/home/mediacms.io/lib/python3.10/site-packages/django/core/handlers/base.py", line 197, in _get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
  File "/home/mediacms.io/lib/python3.10/site-packages/django/views/decorators/csrf.py", line 56, in wrapper_view
    return view_func(*args, **kwargs)
  File "/home/mediacms.io/lib/python3.10/site-packages/django/views/generic/base.py", line 104, in view
    return self.dispatch(request, *args, **kwargs)
  File "/home/mediacms.io/lib/python3.10/site-packages/rest_framework/views.py", line 509, in dispatch
    response = self.handle_exception(exc)
  File "/home/mediacms.io/lib/python3.10/site-packages/rest_framework/views.py", line 469, in handle_exception
    self.raise_uncaught_exception(exc)
  File "/home/mediacms.io/lib/python3.10/site-packages/rest_framework/views.py", line 480, in raise_uncaught_exception
    raise exc
  File "/home/mediacms.io/lib/python3.10/site-packages/rest_framework/views.py", line 506, in dispatch
    response = handler(request, *args, **kwargs)
  File "/home/mediacms.io/mediacms/files/views.py", line 452, in post
    serializer.save(user=request.user, media_file=media_file)
  File "/home/mediacms.io/lib/python3.10/site-packages/rest_framework/serializers.py", line 212, in save
    self.instance = self.create(validated_data)
  File "/home/mediacms.io/lib/python3.10/site-packages/rest_framework/serializers.py", line 962, in create
    instance = ModelClass._default_manager.create(**validated_data)
  File "/home/mediacms.io/lib/python3.10/site-packages/django/db/models/manager.py", line 87, in manager_method
    return getattr(self.get_queryset(), name)(*args, **kwargs)
  File "/home/mediacms.io/lib/python3.10/site-packages/django/db/models/query.py", line 658, in create
    obj.save(force_insert=True, using=self.db)
  File "/home/mediacms.io/mediacms/files/models.py", line 367, in save
    super(Media, self).save(*args, **kwargs)
  File "/home/mediacms.io/lib/python3.10/site-packages/django/db/models/base.py", line 814, in save
    self.save_base(
  File "/home/mediacms.io/lib/python3.10/site-packages/django/db/models/base.py", line 877, in save_base
    updated = self._save_table(
  File "/home/mediacms.io/lib/python3.10/site-packages/django/db/models/base.py", line 1020, in _save_table
    results = self._do_insert(
  File "/home/mediacms.io/lib/python3.10/site-packages/django/db/models/base.py", line 1061, in _do_insert
    return manager._insert(
  File "/home/mediacms.io/lib/python3.10/site-packages/django/db/models/manager.py", line 87, in manager_method
    return getattr(self.get_queryset(), name)(*args, **kwargs)
  File "/home/mediacms.io/lib/python3.10/site-packages/django/db/models/query.py", line 1805, in _insert
    return query.get_compiler(using=using).execute_sql(returning_fields)
  File "/home/mediacms.io/lib/python3.10/site-packages/django/db/models/sql/compiler.py", line 1821, in execute_sql
    for sql, params in self.as_sql():
  File "/home/mediacms.io/lib/python3.10/site-packages/django/db/models/sql/compiler.py", line 1745, in as_sql
    value_rows = [
  File "/home/mediacms.io/lib/python3.10/site-packages/django/db/models/sql/compiler.py", line 1746, in <listcomp>
    [
  File "/home/mediacms.io/lib/python3.10/site-packages/django/db/models/sql/compiler.py", line 1747, in <listcomp>
    self.prepare_value(field, self.pre_save_val(field, obj))
  File "/home/mediacms.io/lib/python3.10/site-packages/django/db/models/sql/compiler.py", line 1695, in pre_save_val
    return field.pre_save(obj, add=True)
  File "/home/mediacms.io/lib/python3.10/site-packages/django/db/models/fields/files.py", line 317, in pre_save
    file.save(file.name, file.file, save=False)
  File "/home/mediacms.io/lib/python3.10/site-packages/django/db/models/fields/files.py", line 93, in save
    self.name = self.storage.save(name, content, max_length=self.field.max_length)
  File "/home/mediacms.io/lib/python3.10/site-packages/django/core/files/storage/base.py", line 38, in save
    name = self._save(name, content)
  File "/home/mediacms.io/lib/python3.10/site-packages/django/core/files/storage/filesystem.py", line 101, in _save
    file_move_safe(content.temporary_file_path(), full_path)
  File "/home/mediacms.io/lib/python3.10/site-packages/django/core/files/move.py", line 43, in file_move_safe
    if _samefile(old_file_name, new_file_name):
  File "/home/mediacms.io/lib/python3.10/site-packages/django/core/files/move.py", line 20, in _samefile
    return os.path.samefile(src, dst)
  File "/usr/lib/python3.10/genericpath.py", line 101, in samefile
    s2 = os.stat(f2)
UnicodeEncodeError: 'ascii' codec can't encode characters in position 95-99: ordinal not in range(128)

Expected behavior The upload should proceed without 500 error.

Environment OS: Ubuntu Linux Installation method: Standalone

Additional context This issue affects Russian language content uploads only.

mgogoulos commented 1 month ago

I remember there was a similar issue that was fixed - https://github.com/mediacms-io/mediacms/issues/41 - are you uploading using the web or through the API? Does the problem happen when uploading a media with a Russian title too, is it in description, or somewhere else (eg tags/categories)?

habibulilalbaab commented 1 month ago

I uploaded via API, and when I tried to upload through the web interface just now, I encountered an error (attempt 2 of 2). Currently, I'm focusing on the title and description and have not yet tried creating tags or categories.

image

debug.log upload from web interface

Internal Server Error: /fu/upload/
Traceback (most recent call last):
  File "/home/mediacms.io/lib/python3.10/site-packages/django/core/handlers/exception.py", line 55, in inner
    response = get_response(request)
  File "/home/mediacms.io/lib/python3.10/site-packages/django/core/handlers/base.py", line 197, in _get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
  File "/home/mediacms.io/lib/python3.10/site-packages/django/views/generic/base.py", line 104, in view
    return self.dispatch(request, *args, **kwargs)
  File "/home/mediacms.io/mediacms/uploader/views.py", line 49, in dispatch
    return super(FineUploaderView, self).dispatch(request, *args, **kwargs)
  File "/home/mediacms.io/lib/python3.10/site-packages/django/views/generic/base.py", line 143, in dispatch
    return handler(request, *args, **kwargs)
  File "/home/mediacms.io/lib/python3.10/site-packages/django/views/generic/edit.py", line 153, in post
    return self.form_valid(form)
  File "/home/mediacms.io/mediacms/uploader/views.py", line 55, in form_valid
    self.upload.combine_chunks()
  File "/home/mediacms.io/mediacms/uploader/fineuploader.py", line 98, in combine_chunks
    self.real_path = self.storage.save(self._full_file_path, StringIO())
  File "/home/mediacms.io/lib/python3.10/site-packages/django/core/files/storage/base.py", line 38, in save
    name = self._save(name, content)
  File "/home/mediacms.io/lib/python3.10/site-packages/django/core/files/storage/filesystem.py", line 106, in _save
    fd = os.open(full_path, self.OS_OPEN_FLAGS, 0o666)
UnicodeEncodeError: 'ascii' codec can't encode characters in position 84-90: ordinal not in range(128)
mgogoulos commented 1 month ago

I cannot reproduce it, just uploaded a file with russian title in file name and it finished well. This has been an issue on an older version, so I believe your installation is using an old version. Try to update and re-upload