Error: "subscriptions are not allowed"

[levinotik]

Hey there, trying to get this set up in my project. I've gone through the README and set up the required pieces (I think). I'm now trying to use Graphiql to test out a subscription that I set up. When I do, I get this error response:

Subscriptions are not allowed. You will need to either use the subscribe function or pass allow_subscriptions=True

and can see this in stack trace as well:

  File "/Users/levinotik/.local/share/virtualenvs/great-control-ic_1WMqr/lib/python3.7/site-packages/graphql/execution/executor.py", line 176, in execute_operation
    "Subscriptions are not allowed. "

I must be missing something basic. Any ideas on what I'm doing wrong here? Thanks.

[levinotik]

I think I actually got past that issue by creating a custom GraphQLCoreBackend

class GraphQLCustomCoreBackend(GraphQLCoreBackend):
    def __init__(self, executor=None):
        # type: (Optional[Any]) -> None
        super().__init__(executor)
        self.execute_params['allow_subscriptions'] = True

and hooking it up in my urls.py as in path('graphql/', csrf_exempt(CustomGraphQLView.as_view(graphiql=True, backend=GraphQLCustomCoreBackend())), name='graphql').

The issue now is that I'm receiving this error:

"message": "Subscription must return Async Iterable or Observable. Received: <Promise at 0x1097354d0 rejected with AttributeError(\"'NoneType' object has no attribute 'filter'\")>"

My resolve function is defined as:

    def resolve_stage_changed(root, info, id):
        return root.filter(
            lambda event:
            event.operation == UPDATED and
            isinstance(event.instance, Turn) and
            event.instance.pk == int(id)
        ).map(lambda event: event.instance)

Any guidance on this?

[jaydenwindle]

Hey @levinotik! Yes, afaik the default GraphiQL view in Graphene doesn't support subscriptions out of the box. That's really neat that you can add subscription support to the default GraphiQL view by passing in a different backend, I didn't know that!

I usually use GraphQL Playground to test anything subscription-related. Can you test this example in GraphQL Playground to see whether it's a GraphiQL view issue or a subscriptions implementation issue that's happening here?

I'll see if I can reproduce this locally as well. Can you post some more details about your setup (requirements.txt, how you're triggering the subscription, etc) to help me debug?

[levinotik]

Thanks for the quick reply. I'm downloading GraphQL Playground right now to take a closer look. I'm using Django which may be the issue here. In your docs, you have this application bit

application = ProtocolTypeRouter({
    "websocket": URLRouter([
        path('graphql/', GraphqlSubscriptionConsumer)
    ]),
})

Not sure that plays nice with graphql/ endpoint I have set up in Django's urlpatterns. Will revert with more info.

[levinotik]

Yeah so I downloaded GraphQL playground and added a simple hello subscription into my project. I then tried to subscribe in the playground and never seem to receive an event. The playground seems aware of the schema just fine (it knows there's a hello subscription), but my guess is that the Django server doesn't know anything about the websocket routing.

[jaydenwindle]

@levinotik Since Django channels routes requests based on protocol (websocket vs http), having the same endpoint for both the GraphQL view and consumer should be fine.

I wonder if your custom GraphiQL view is trying to call the subscription via an HTTP request instead of connecting to the websocket endpoint here? That would explain why root is None, since GraphqlSubscriptionConsumer sets up the observable and passes it into each subscription resolver as the root parameter.

[levinotik]

Oh ok so that's not the issue, thanks. Actually, it looks like the requests are coming through. I'm seeing the ping to hello over and over again in the logs.

INFO 2019-12-09 20:31:58,171 views GraphQL request
INFO 2019-12-09 20:31:58,180 _internal 127.0.0.1 - - [09/Dec/2019 20:31:58] "POST /graphql/ HTTP/1.1" 200 -
INFO 2019-12-09 20:32:00,883 views GraphQL request
INFO 2019-12-09 20:32:00,889 _internal 127.0.0.1 - - [09/Dec/2019 20:32:00] "POST /graphql/ HTTP/1.1" 200 -
INFO 2019-12-09 20:32:03,627 views GraphQL request
INFO 2019-12-09 20:32:03,634 _internal 127.0.0.1 - - [09/Dec/2019 20:32:03] "POST /graphql/ HTTP/1.1" 200 -

But never actually see a response in the playground. Just shows a spinner forever.

[jaydenwindle]

Thanks, those logs are really helpful. Are you running the server with ./manage.py runsever? Have you set up Django Channels in your project? And are you using the same endpoint for both your GraphQL view and the consumer?

I don't see any websocket connection attempts in these logs, which makes me think that either the server isn't listening for them (an issue with Django channels), or that it isn't receiving them (because GraphQL Playground doesn't know how to access the websocket endpoint).

If you use different endpoints for your view and consumer, I believe there is some extra configuration you have to do to make GraphQL Playground play nicely with that setup.

[levinotik]

Thanks. So I've added channels to INSTALLED_APPS and added ASGI_APPLICATION setting to point to myproject.routing.application which contains the snippet you provided in your docs with the ProtocolTypeRouter, etc. In that sense it differs from the Django Channels instructions because it doesn't point to the empty ProtocolTypeRouter in their docs (which I assume is fine). Is this not correct?

Also, can you clarify what you mean by different endpoints for view and consumer? I'm not Django expert so I'm not clear what you're asking there. Thanks!

[jaydenwindle]

That sounds right to me :)

I think GraphQL Playground sends both HTTP and Websocket requests to the same URL by default. If GraphQLView is set up at path('graphql/' GraphQLView.as_view(graphiql=True)) and GraphqlSubscriptionConsumer is set up at some other path (e.g. path('graphql-ws/', GraphqlSubscriptionConsumer), then GraphQL Playground might not be able to send subscription websocket requests to your server without extra configuration.

Are you running the server with ./manage.py runserver? Or are you using gunicorn or another application server?

[levinotik]

I had been using npx sls wsgi serve -s local to run the server and just now tried manage.py runserver as well.

[levinotik]

I have them both set up at the same path (I think)...

I have

urlpatterns = [
    path('graphql/', csrf_exempt(CustomGraphQLView.as_view(graphiql=True, backend=GraphQLCustomCoreBackend())), name='graphql'),...

as well as this in routing.py

application = ProtocolTypeRouter({
    "websocket": URLRouter([
        path('graphql/', GraphqlSubscriptionConsumer)
    ]),
})

[jaydenwindle]

Ok, got it. So not a path issue :)

Did it work with ./manage.py runserver? Can you send some logs of subscription attempts while running it with the runserver command?

Django channels modifies the runserver command to use ASGI instead of WSGI, which lets the server respond to websocket requests. This library won't work with a WSGI server, since it needs to be able to process websocket requests asynchronously.

[levinotik]

Sure, here it is with runserver

⇒  ./manage.sh runserver
Serverless: Spawning pipenv run ./manage.py runserver...
Watching for file changes with StatReloader
INFO 2019-12-09 20:55:56,138 autoreload Watching for file changes with StatReloader
Performing system checks...

System check identified no issues (0 silenced).
December 09, 2019 - 20:55:56
Django version 2.2.8, using settings 'great_control.settings'
Starting ASGI/Channels version 2.3.1 development server at http://127.0.0.1:8000/
Quit the server with CONTROL-C.
INFO 2019-12-09 20:55:56,671 server HTTP/2 support not enabled (install the http2 and tls Twisted extras)
INFO 2019-12-09 20:55:56,672 server Configuring endpoint tcp:port=8000:interface=127.0.0.1
INFO 2019-12-09 20:55:56,672 server Listening on TCP address 127.0.0.1:8000
DEBUG 2019-12-09 20:55:58,383 http_protocol HTTP b'POST' request for ['127.0.0.1', 60751]
INFO 2019-12-09 20:55:59,209 views GraphQL request
DEBUG 2019-12-09 20:55:59,215 http_protocol HTTP 200 response started for ['127.0.0.1', 60751]
DEBUG 2019-12-09 20:55:59,217 http_protocol HTTP close for ['127.0.0.1', 60751]
DEBUG 2019-12-09 20:55:59,217 http_protocol HTTP response complete for ['127.0.0.1', 60751]
HTTP POST /graphql/ 200 [0.83, 127.0.0.1:60751]
INFO 2019-12-09 20:55:59,218 runserver HTTP POST /graphql/ 200 [0.83, 127.0.0.1:60751]
DEBUG 2019-12-09 20:56:01,235 http_protocol HTTP b'POST' request for ['127.0.0.1', 60751]
INFO 2019-12-09 20:56:01,905 views GraphQL request
DEBUG 2019-12-09 20:56:01,911 http_protocol HTTP 200 response started for ['127.0.0.1', 60751]
DEBUG 2019-12-09 20:56:01,912 http_protocol HTTP close for ['127.0.0.1', 60751]
DEBUG 2019-12-09 20:56:01,913 http_protocol HTTP response complete for ['127.0.0.1', 60751]
HTTP POST /graphql/ 200 [0.68, 127.0.0.1:60751]
INFO 2019-12-09 20:56:01,913 runserver HTTP POST /graphql/ 200 [0.68, 127.0.0.1:60751]
DEBUG 2019-12-09 20:56:03,924 http_protocol HTTP b'POST' request for ['127.0.0.1', 60751]
INFO 2019-12-09 20:56:04,589 views GraphQL request
DEBUG 2019-12-09 20:56:04,595 http_protocol HTTP 200 response started for ['127.0.0.1', 60751]
DEBUG 2019-12-09 20:56:04,596 http_protocol HTTP close for ['127.0.0.1', 60751]
DEBUG 2019-12-09 20:56:04,596 http_protocol HTTP response complete for ['127.0.0.1', 60751]
HTTP POST /graphql/ 200 [0.67, 127.0.0.1:60751]
INFO 2019-12-09 20:56:04,596 runserver HTTP POST /graphql/ 200 [0.67, 127.0.0.1:60751]
DEBUG 2019-12-09 20:56:06,606 http_protocol HTTP b'POST' request for ['127.0.0.1', 60751]
INFO 2019-12-09 20:56:07,308 views GraphQL request
DEBUG 2019-12-09 20:56:07,313 http_protocol HTTP 200 response started for ['127.0.0.1', 60751]
DEBUG 2019-12-09 20:56:07,314 http_protocol HTTP close for ['127.0.0.1', 60751]
DEBUG 2019-12-09 20:56:07,314 http_protocol HTTP response complete for ['127.0.0.1', 60751]
HTTP POST /graphql/ 200 [0.71, 127.0.0.1:60751]
INFO 2019-12-09 20:56:07,314 runserver HTTP POST /graphql/ 200 [0.71, 127.0.0.1:60751]
DEBUG 2019-12-09 20:56:09,326 http_protocol HTTP b'POST' request for ['127.0.0.1', 60751]
INFO 2019-12-09 20:56:10,018 views GraphQL request
DEBUG 2019-12-09 20:56:10,024 http_protocol HTTP 200 response started for ['127.0.0.1', 60751]
DEBUG 2019-12-09 20:56:10,026 http_protocol HTTP close for ['127.0.0.1', 60751]
DEBUG 2019-12-09 20:56:10,026 http_protocol HTTP response complete for ['127.0.0.1', 60751]

[jaydenwindle]

Ok, awesome - super helpful :)

Still not seeing any websocket requests in there, which makes me think it's a configuration issue. What's the exact value of your ASGI_APPLICATION setting?

[levinotik]

ASGI_APPLICATION = "great_control.routing.application" and the application is in the root project dir at great_control/routing.py which contains:

from channels.routing import ProtocolTypeRouter, URLRouter
from django.urls import path

from graphene_subscriptions.consumers import GraphqlSubscriptionConsumer

application = ProtocolTypeRouter({
    "websocket": URLRouter([
        path('graphql/', GraphqlSubscriptionConsumer)
    ]),
})

[jaydenwindle]

Got it, that looks right to me.

Let me try to reproduce this locally. Is it possible to share the source with me in a private repo so I can diagnose? No worries if not, I understand it may be sensitive.

[levinotik]

Wish I could :( Unfortunately I cannot.

[jaydenwindle]

No worries at all :)

Let me take a stab a reproducing this and get back to you.

[levinotik]

Thanks so much for looking into this, really appreciate it.

[jaydenwindle]

Ok, I think I may have found the issue. This is definitely on me for not documenting something this important in the readme 🤦‍♂️.

You'll need to set up Channel Layers in order for subscriptions to work properly. If you don't want to set up a Redis in your dev environment, you can set up Channel Layers using an in-memory backend by adding the following to your settings.py file:

CHANNEL_LAYERS = {
    "default": {
        "BACKEND": "channels.layers.InMemoryChannelLayer"
    }
}

I'm updating the REAME now. Can you give this a try and see if it works for you? I still think you might have a config issue (since there are no websocket connections in the logs), but let's try this first.

[jaydenwindle]

Also, I just noticed that you're running ./manage.sh runserver instead of ./manage.py runserver. Is ./manage.sh a custom script that is a part of your project? If so, I wonder if there's something in there that's interfering with how the runserver command normally operates.

[levinotik]

I doubt that's the case, but just to be sure I created a new project and just used manage.py. Can't seem to get even minimal example working. Repo is here: https://github.com/levinotik/graphene-django-subscriptions

[levinotik]

And yes, that's a custom script which just does

command="pipenv run ./manage.py $@"
npx sls shell -s local -S "$command"

[levinotik]

Oh whoops, didn't see previous message about channel layers. Trying now!

[levinotik]

Pushed commit adding channel layers to that repo. Still no dice. https://github.com/levinotik/graphene-django-subscriptions/commits/master

[levinotik]

To be clear, in the GraphQL playground I've just entered

subscription hello {
  hello
}

and clicked "Play". That's correct, right?

[jaydenwindle]

I also updated the hello world subscription in the README - you'll need to make sure to pass an argument to the lambda function which you pass to map in the resolver, like so:

import graphene
from rx import Observable

class HelloSubscription(graphene.ObjectType):
    hello = graphene.String()

    def resolve_hello(root, info):
        return Observable.interval(3000) \
                         .map(lambda i: "hello world!")

Yep, that's correct. The example in your repo seems to be working fine for me - here it is in action:

[levinotik]

Well, I'll be damned. Works now! Thanks so much for your time on this, really appreciate it!

[jaydenwindle]

Ayyyy, amazing! So glad you got it working :D

Feel free to open another issue if you run into any more problems :)

[levinotik]

Thanks. Now to get this working in the actual project lol...

[levinotik]

Added the channel layer to the project settings (as I did what that sample repo I shared) and get this when I try subscription in GraphQL Playground. 😔

[jaydenwindle]

Did this happen as soon as you clicked the play button? Or was it after some time? Can you share the logs from your app? Usually this means something crashed, and you can see the stack trace in the logs.

[levinotik]

I'm not sure, taking a closer look. I first need to fix some issues that will let me run with the regular runserver and not WSGI which you had said won't work because WS requests need to be processed asynchronously.

[levinotik]

Ok, just passed runserver to our custom manage.sh script and it seems to work! All good now.

[jaydenwindle]

Awesome! Glad it's working for you :D

[levinotik]

Ok so I've got that basic subscription working (the hello example), but can't seem to get Django model updates working.

Using Graphiql, I'm executing a mutation and subscribing to that model ID using GQL Playground. I'm not receiving any events and don't see my print statement in the resolve function in my subscription class so it doesn't seem to be getting called at all. I'm using the graphene-django package so my GQL type inherits from that, i.e. MyType(DjangoObjectType):... Is that a problem or should be fine?

My DjangoObjectType is pretty straightforward with some Meta and a bunch of graphene.Fields

My subscription looks like this:

class TurnUpdatedSubscription(graphene.ObjectType):
    turn_updated = graphene.Field(TurnType, id=graphene.Int())

    def resolve_turn_updated(root, info, id):
        print('###resolve turn updated###')
        return root.filter(
            lambda event:
            event.operation == UPDATED and
            isinstance(event.instance, Turn) and
            event.instance.pk == id
        ).map(lambda event: event.instance)

And in signals.py I have:

post_save.connect(post_save_subscription, sender=Turn, dispatch_uid="turn_post_save")

Anything look off here at first glance?

[jaydenwindle]

This looks good so far. Your resolve function should get called once for each time a client subscribes, not every time an update is fired. Does your resolve function not get called at all?

Have you imported signals.py in your your_app/apps.py file? That's necessary to make sure your signals fire correctly.

# your_app/apps.py
from django.apps import AppConfig

class YourAppConfig(AppConfig):
    name = 'your_app'

    def ready(self):
        import your_app.signals

[levinotik]

Thanks. Yes, I have imported into apps.py.

[levinotik]

By the way, I went to take a look at your tests to make sure I was doing a model update subscription correctly and it doesn't look like your test actually checks the update case? Looks like it does a create. I would've expected to see updating some model field and then doing a save() to test this case. That's the case I'm trying to get working right now and it involves an update to a Django model, not a create.

See: https://github.com/jaydenwindle/graphene-subscriptions/blob/master/tests/test_model_subscriptions.py#L94

[jaydenwindle]

The save() call is on line 109. The signal will still fire if no field has been updated.

[levinotik]

Right, but the operation is a create, i.e. s = await sync_to_async(SomeModel.objects.create)(name="test name")

Also, I am now seeing my resolve function called so that's good. Still trying to get the update event back by executing a subscription. Digging further.

[jaydenwindle]

Yep - I had to create the model first in the test (line 94) in order to update it on line 109 :)

Awesome, glad the resolver is firing! One thing that I've found helpful while debugging is to use this resolver to print each observable update value:

class TurnUpdatedSubscription(graphene.ObjectType):
    turn_updated = graphene.Field(TurnType, id=graphene.Int())

    def resolve_turn_updated(root, info, id):
        print('###resolve turn updated###')
        return root.map(lambda event: print(event))

[levinotik]

My point is, shouldn't the test be using something like

someModel = SomeModel.objects.get(id=1)
someModel.value = whatever  # update field
someModel.save() 

?

[levinotik]

Thanks for the debug example. Don't seem to be seeing any prints using that.

[jaydenwindle]

That would probably make the test clearer, for sure :)

Technically the post_save signal (and therefore the UPDATED event) will still be fired even if a model is saved with no field changes, so the test does work as is.

[jaydenwindle]

If you try connecting another signal in your signals.py file, does it fire? E.g.

def debug_signal(sender, **kwargs):
    print(sender, kwargs)

post_save.connect(debug_signal, sender=Turn, dispatch_uid="post_save_debug")

[levinotik]

That's a good test, thanks. Doesn't look like that's getting called.

[levinotik]

So it would seem something is off with Django signals themselves, unrelated to graphene-subscriptions really...

[levinotik]

Hmm...I have these other functions defined annotated with @receiver(post_save, sender=Turn) and they seem to fire just fine...

[levinotik]

Ok got it working. For some reason when I have the post_save.connect(debug_signal, sender=Turn, dispatch_uid="post_save_debug") bit defined in a separate file, it won't fire. I added it to the bottom of an existing signals file that I have in turns/signals which is also listed in turns/signals/__init__.py. Perhaps it would work as a standalone file if I were to list it in __init__.py? I couldn't quite figure out the syntax for importing there...