'seek of closed file' Error when using django-imagekit with Django 2.1.1

[Routhinator]

Ok, after a thorough read of #391 I can say it's not related, though it is similar.

I am using Django default local storage. No storage plugins of any kind.

The following model code:

    avatar = ProcessedImageField(
        upload_to="avatars",
        processors=[ResizeToFill(200, 200)],
        format='JPEG',
        options={'quality': 60}
    )

    avatar_100 = ImageSpecField(
        source='avatar',
        processors=[ResizeToFill(100, 100)],
        format='JPEG',
        options={'quality': 60}
    )

    avatar_50 = ImageSpecField(
        source='avatar',
        processors=[ResizeToFill(50, 50)],
        format='JPEG',
        options={'quality': 60}
    )

Causes the below error when attempting to retrive the model:

ValueError at /api/members/

seek of closed file

Versions (requirements.txt):

asn1crypto==0.24.0
astroid==2.0.4
block-disposable-email==1.0.1
certifi==2018.8.24
cffi==1.11.5
chardet==3.0.4
coreapi==2.3.3
coreschema==0.0.4
cryptography==2.3.1
Django==2.1.1
django-activity-stream==0.6.5
django-appconf==1.0.2
django-imagekit==4.0.2
django-precise-bbcode==1.2.10
django-rest-framework==0.1.0
django-rest-knox==3.1.5
djangorestframework==3.8.2
docutils==0.14
gunicorn==19.9.0
idna==2.7
isort==4.3.4
itypes==1.1.0
Jinja2==2.10
lazy-object-proxy==1.3.1
MarkupSafe==1.0
mccabe==0.6.1
mysqlclient==1.3.13
pilkit==2.0
Pillow==5.2.0
psycopg2-binary==2.7.5
pycparser==2.18
pylint==2.1.1
pylint-django==2.0.2
pylint-plugin-utils==0.4
pyOpenSSL==18.0.0
pytz==2018.5
PyYAML==3.13
redis==2.10.6
requests==2.19.1
six==1.11.0
typed-ast==1.1.0
uritemplate==3.0.0
urllib3==1.23
wrapt==1.10.11

Traceback:

Environment:

Request Method: GET
Request URL: http://amateurwriting.local:8000/api/members/

Django Version: 2.1.1
Python Version: 3.6.5
Installed Applications:
['rest_framework',
 'rest_framework.authtoken',
 'knox',
 'precise_bbcode',
 'imagekit',
 'theden_django.apps.theden',
 'theden_django.apps.core',
 'theden_django.apps.members',
 'theden_django.apps.comments',
 'theden_django.apps.karma',
 'theden_django.apps.discussions',
 'theden_django.apps.writings',
 'theden_django.apps.quotes',
 'theden_django.apps.online_users',
 'theden_django.apps.shoutbox',
 'theden_django.apps.legacy',
 'theden_django.apps.disposable_email_checker',
 'django.contrib.admin',
 'django.contrib.admindocs',
 'django.contrib.auth',
 'django.contrib.contenttypes',
 'django.contrib.sessions',
 'django.contrib.messages',
 'django.contrib.staticfiles']
Installed Middleware:
['django.middleware.security.SecurityMiddleware',
 'django.contrib.sessions.middleware.SessionMiddleware',
 'django.middleware.locale.LocaleMiddleware',
 'django.middleware.common.CommonMiddleware',
 'django.middleware.csrf.CsrfViewMiddleware',
 'django.contrib.auth.middleware.AuthenticationMiddleware',
 'django.contrib.messages.middleware.MessageMiddleware',
 'django.middleware.clickjacking.XFrameOptionsMiddleware',
 'theden_django.apps.online_users.middleware.OnlineNowMiddleware']

Traceback:

File "/usr/local/lib/python3.6/dist-packages/django/core/handlers/exception.py" in inner
  34.             response = get_response(request)

File "/usr/local/lib/python3.6/dist-packages/django/core/handlers/base.py" in _get_response
  156.                 response = self.process_exception_by_middleware(e, request)

File "/usr/local/lib/python3.6/dist-packages/django/core/handlers/base.py" in _get_response
  154.                 response = response.render()

File "/usr/local/lib/python3.6/dist-packages/django/template/response.py" in render
  106.             self.content = self.rendered_content

File "/usr/local/lib/python3.6/dist-packages/rest_framework/response.py" in rendered_content
  72.         ret = renderer.render(self.data, accepted_media_type, context)

File "/usr/local/lib/python3.6/dist-packages/rest_framework/renderers.py" in render
  724.         context = self.get_context(data, accepted_media_type, renderer_context)

File "/usr/local/lib/python3.6/dist-packages/rest_framework/renderers.py" in get_context
  681.             'content': self.get_content(renderer, data, accepted_media_type, renderer_context),

File "/usr/local/lib/python3.6/dist-packages/rest_framework/renderers.py" in get_content
  422.         content = renderer.render(data, accepted_media_type, renderer_context)

File "/usr/local/lib/python3.6/dist-packages/rest_framework/renderers.py" in render
  105.             allow_nan=not self.strict, separators=separators

File "/usr/local/lib/python3.6/dist-packages/rest_framework/utils/json.py" in dumps
  28.     return json.dumps(*args, **kwargs)

File "/usr/lib/python3.6/json/__init__.py" in dumps
  238.         **kw).encode(obj)

File "/usr/lib/python3.6/json/encoder.py" in encode
  201.             chunks = list(chunks)

File "/usr/lib/python3.6/json/encoder.py" in _iterencode
  430.             yield from _iterencode_dict(o, _current_indent_level)

File "/usr/lib/python3.6/json/encoder.py" in _iterencode_dict
  404.                 yield from chunks

File "/usr/lib/python3.6/json/encoder.py" in _iterencode_list
  325.                 yield from chunks

File "/usr/lib/python3.6/json/encoder.py" in _iterencode_dict
  404.                 yield from chunks

File "/usr/lib/python3.6/json/encoder.py" in _iterencode
  437.             o = _default(o)

File "/usr/local/lib/python3.6/dist-packages/rest_framework/utils/encoders.py" in default
  67.             return tuple(item for item in obj)

File "/usr/local/lib/python3.6/dist-packages/rest_framework/utils/encoders.py" in <genexpr>
  67.             return tuple(item for item in obj)

File "/usr/local/lib/python3.6/dist-packages/django/core/files/base.py" in __iter__
  78.         for chunk in self.chunks():

File "/usr/local/lib/python3.6/dist-packages/django/core/files/base.py" in chunks
  55.             self.seek(0)

Exception Type: ValueError at /api/members/
Exception Value: seek of closed file

[vstoykov]

From the traceback:

...
File "/usr/local/lib/python3.6/dist-packages/rest_framework/utils/encoders.py" in default
  67.             return tuple(item for item in obj)

File "/usr/local/lib/python3.6/dist-packages/rest_framework/utils/encoders.py" in <genexpr>
  67.             return tuple(item for item in obj)

File "/usr/local/lib/python3.6/dist-packages/django/core/files/base.py" in __iter__
  78.         for chunk in self.chunks():
...

Rest framework is trying to serialize the file by iterate trough it's content (bytes) which I'm not sure is what you want. Because you are using the djangorestframework can you show us also your serialyzer?

[Routhinator]

This is the serializer:

class MemberSerializer(ModelSerializer):
    """
    Serializer for public Members endpoint
    """
    groups = GroupSerializer(many=True, read_only=True)

    class Meta:
        model = Member
        fields = (
            'id',
            'url',
            'username',
            'bio',
            'language',
            'homepage',
            'location',
            'avatar_url',
            'avatar_100_url',
            'avatar_50_url',
            'groups',
            'background_url'
        )

It's a very vanilla DRF serializer. I ended up swapping imagekit out for django-versatileimagefield and everything works great with the same serializer in place, however I still have record of all the code that caused this in my git history so if any more information is required, I'll be happy to provide it.

[vstoykov]

Thank you for the information.

As I see you probably has few extra methods/properties in the model avatar_url, avatar_100_url, avatar_50_url, which will probably are just basic getter for the url property of the respecting field right?

This means that trying to create two thumbnails in the same request from the same source can fail.

Thank you again for reporting the issue.

[Routhinator]

Yes they return the url or a default noavatar image. They are marked with the @property decorator.

[SKYnv]

same with s3 as media storage. And all works well if i remove content.seek(0) app[web.1]: File "/app/.heroku/python/lib/python3.7/site-packages/imagekit/cachefiles/__init__.py", line 103, in _generate app[web.1]: content.seek(0) app[web.1]: ValueError: I/O operation on closed file.

[sysradium]

Any news on this? Getting same error on s3

[vstoykov]

Currently I have no time to look on this. If someone can debug it and came up with a patch I will be very greatfull.