Getting a 301 error when running distill on server

[esskaey]

Hi,

I am getting a strange 301 error which is not happening during local development, but only when running on an app server. The same distill action on a local server runs without any error.

Unfortunately, I cannot do a remote debug on the server. Some information on the error

I can add more debug information, but I don't know which would be of use here.

BR

SK

[meeb]

This is an issue with your site or URLs or hosting and not django-distill really. This error is because one of the URLs you've wrapped with django-distill is returning a 301 redirect and not a 200/OK with an HTML body. If this is only occurring remotely this is likely to be something like an automatic redirect for on a directory e.g.

https://www.mysite.com/page to https://www.mysite.com/page/

or

https://www.mysite.com/page/ to https://www.mysite.com/page

depending on how your deployment is handling training slashes. Either way, the fix for this is either to set the URL in Django to end with a slash, or remove it if set. Or, change how trailing slashes are handled in your application server.

Basically, you just need to make the URL that django-distill requests return a 200/OK.

[esskaey]

I already checked this part. Could it also be because of a redirect to login screen ? the response as marked in the screenshot has a different base address though "https://testserver/site/index"

[meeb]

Yes, if you are trying to statically render a URL that requires authentication with Django this would cause your issue. Internally django-distill makes an dummy request which does not provide or know about authentication so this would cause Django to return a 301 redirect to the login form. However, this should also be the case with your local deployment and not just the live hosted version.

Trying to render a static copy of a page which requires authentication doesn't make much logical sense in most deployment workflows so this hasn't come up as an issue before (given that authenticated URLs are generally for something dynamic or private which isn't something you'd want to render statically). Try removing the authentication requirement for the URL and see if this fixes your issue.

If this does resolve it, I suppose the easiest next question is can you work around this or does your project absolutely require statically rendering authenticated URLs?

Edit: and as for the https://testserver/ prefix, this is used by the Django test framework which django-distill uses internally, so this shouldn't cause an issue or be the cause of this issue.

[esskaey]

Thank you. I will check this part. I don't think the project requires rendering authenticated URLs. The rendering will be part of the celery task and it is best to have it without authentication

[esskaey]

I tried everything. It just doesnt seem to work :/ ... Have to remove items 1 by 1 to see where this is failing!

[meeb]

Well, the trace of the error tells you it's just the site/index.html URL so just check that view. Shouldn't be too hard to track down the cause.

[esskaey]

Hey,

I found the problem. There are two parts

1. SESSION_COOKIE_SECURE (or) SECURE_SSL_REDIRECT: I am not sure if it is both or just one of them, but if this setting is set TRUE in** the "settings.py" then the generation doesn't work! They have to be set false. I don't know the reason why, just figured it out now

2. The request factory used for the render is missing resolver arguments. I am using celery for tracking, and it needs a view_name for tracking and bug reporting. Because the request factory doesn't have this info, the generation fails. Link: https://github.com/getsentry/sentry-python/blob/master/sentry_sdk/integrations/django/views.py#:~:text=.update_active_thread_id()-,with%20hub.start_span(,return%20callback(request%2C%20*args%2C%20**kwargs),-return%20sentry_wrapped_callback

   I think point 2. could be added in the code base ?

Looking forward to hearing your view points

[meeb]

As per https://docs.djangoproject.com/en/4.1/ref/settings/#secure-ssl-redirect this makes perfect sense. The internal request is not over HTTPS so the SecurityMiddleware will return a 301 to the HTTPS URL. The easiest way I can think of to fix this without modifying the rest of your site is probably to just disable it for the static page generation if you're calling it programmatically. For example:

from django.test import override_settings
settings_manager = override_settings(SECURE_SSL_REDIRECT=False)
settings_manager.enable()
# do static generation here
settings_manager.disable()

A bit hacky, but it would probably work. There is a secure=True parameter you can pass to the test requests get request that would also fix this, however it's not enabled by default because it would break everyone who isn't using HTTPS even for internal requests and (at least several years ago the last time I looked) it was difficult to attempt to detect if the dummy internal request was "secure" or not (or flagged as secure, even when it really isn't because it's a dummy internal test request).

As for the second point, yes this sounds like something I could add, I've never used django-distill with Sentry so it's untested. Where exactly does Sentry expect the view_name to be injected?

[esskaey]

settings_manager.enable()

1. Do I override the settings in say a custom renderer class ? I currently do the generation using a management command call :/

2. I have marked the access point from sentry inside my last comment. I think you need to add something like request.resolver_match = resolve(uri)

[meeb]

Yes you should be able to override the settings in a management command call, I think. I've not tested it but I can't see any reason why that wouldn't work.

Your link didn't highlight anything for me, just linked to a file, perhaps it was mangled by markdown. I'll check if I can stuff in a spoofed resolver match.

[esskaey]

Thank you so much for the support!

[meeb]

No problem, I don't think I've seen anyone ever use django-distill programmatically with workers and request logging so I'm not shocked you've stumbled into unsupported territory. If it's a simple fix I don't mind adding it as a feature.

[esskaey]

Yeah ... That is exactly my use case, where I am running celery workers to run the django-distill tasks and deploying them to different places. Would greatly appreciate if you could add them!

I am also looking forward to trying out rendering single files :)