mljar / mercury

Convert Jupyter Notebooks to Web Apps
https://RunMercury.com
GNU Affero General Public License v3.0
3.94k stars 251 forks source link

Nginx Proxy Manager Help and Django CSRF #299

Open c157fl opened 1 year ago

c157fl commented 1 year ago

Hi there,

I'm trying to get Mercury working behind Nginx Proxy Manager as I already have multiple services proxied there. I'm also running openvscodeserver that's from where Mercury is started.

I can access Mercury and click to open notebooks. On the left side it does show as connected but the worker information just continues as waiting for worker. From the browser console I get WS connect ...unknown 0 and worker-state Queued

From running in verbose I get

Starting new https connection (1): ****:443 *****:443 "GET /api/v1/worker/xxxxxxxxxxxxxxxxxx/xx/x/nb HTTP/1.1 403 None Exception when notebook load, quit Can't load notebook


I also try to access django admin and get CSRF verification failed even though new to django I improvised a change on mercury/server/settings.py adding CSRF_TRUSTED_ORIGINS since creatimg such a file in the main folder was also not having effect. Neither option worked.

How can this be set?

Any help will be greatly appreciated, Thank you

pplonski commented 1 year ago

Hi @c157fl,

Thank you for reporting the issue. Looks like you can load frontend and your backend is working (you are almost there). There is a problem with a worker. Did you set Mercury Site as private?

Please create a separate issue for Admin Panel access, so we can track it better. Have you created a superuser to access Admin Panel?

c157fl commented 1 year ago

I had not changed anything to private nor public. Added now a first cell as:

  {
   "cell_type": "raw",
   "metadata": {},
   "source": [
    "---\n",
    "title: Hello 🌍🪐\n",
    "description: Hello app\n",
    "share: public\n",
    "---"
   ]
  },

But without success. I had also created a superuser before but was never able to access the panel. I'll create a separate issue for the Admin Panel and CSRF later today.

pplonski commented 1 year ago

Ups, looks like you are using the old version (V1). We created a new version (V2) of Mercury and moved away from YAML to Python API. You can find documentation here: https://RunMercury.com

Sorry, I didnt have time to fix all pages/information about the update. Apologize for confusion.

May I ask what is your use-case?

c157fl commented 1 year ago

I believe that might be a verbose mix-up as I seem to be running the latest version. This is a new venv on 3.10.

I sought that "share" option online as I had not seen it anywhere.

image

Doing "mercury run demo" yields the same result when opening any of the demo notebooks.

For now I want to share a single notebook to showcase what can be done with python and thus have some demo code which creates a bunch of graphs based on some data and parameters defined.

pplonski commented 1 year ago

Could you please provide me details on how to reproduce your environment to reproduce this issue? I'd love to fix it!

My normal workflow is:

From your description I understand that you are running remote VS Code instance. I've never done it. The first result from Google search for openvscodesrver gives link to https://github.com/gitpod-io/openvscode-server. Do I need to register at gitpod to reproduce the issue?

c157fl commented 1 year ago

I have it remotely on a server but never used gitpod. It's mostly docker based:

OpenVSCode from hub.docker.com/r/linuxserver/openvscode-server

---
version: "2.1"
services:
  openvscode-server:
    image: lscr.io/linuxserver/openvscode-server:latest
    container_name: openvscode-server
    environment:
      - PUID=1000
      - PGID=1000
      - SUDO_PASSWORD=password
    volumes:
      - ~/docker/ovsc/config:/config
      - ~/docker/ovsc/code:/code
    ports:
      - 3000:3000
    restart: unless-stopped

NginxProxyManager from https://nginxproxymanager.com/setup/ Nginx then has two hosts relevant

Domain Names vscode.***********
Scheme: http
Forward Hostname / IP: openvscode-server
Forward Port: 3000
Websocket support: On
Domain Names mercury.***********
Scheme: http
Forward Hostname / IP: openvscode-server
Forward Port: 8000
Websocket support: On

There's then Cloudflare linking to the domains with SSL..

In the vscode, python was installed (password for root is password), a venv created and installed mercury, pandas, numpy, matplotlib and jupyterlab in that same venv. Still attached to that venv, mercury is started.


If you try mercury run 0.0.0.0.:8000 -verbose locally, do you get /api/v2/?

pplonski commented 1 year ago

Thank you @c157fl for details. I've tried to give a chance to GitPod service.

I think I've reproduced your issue. Below is video from running mercury on remote VS code: Peek 2023-05-30 08-53

Is it similar problem as your?

I was able to fix it by making port public: Peek 2023-05-30 08-54

c157fl commented 1 year ago

It is similar in terms of behaviour. But I already see the port open from my side. I think the ports tab is an exclusive when doing cloud on gitpod's infrastructure. This is a screenshot from nmap running against the code server when mercury is open, where I see port 8000.

image

Also I did notice in your case the error thrown was 401 0, while in my case is 403 None.

log

``` DJ INFO 2023-05-30 10:34:54,598 runserver HTTP GET /app/test 200 [0.01, 172.23.0.4:33798] DJ INFO 2023-05-30 10:34:54,859 runserver HTTP GET /static/jupyter-syntax.css 200 [0.00, 172.23.0.4:33808] DJ INFO 2023-05-30 10:34:55,044 runserver HTTP GET /static/jupyter-additional.css 200 [0.00, 172.23.0.4:33812] DJ INFO 2023-05-30 10:34:55,403 runserver HTTP GET /static/css/2.c6cf5ff9.chunk.css 200 [0.01, 172.23.0.4:33818] DJ INFO 2023-05-30 10:34:55,406 runserver HTTP GET /static/css/main.80c2d41b.chunk.css 200 [0.01, 172.23.0.4:33838] DJ INFO 2023-05-30 10:34:55,408 runserver HTTP GET /static/js/2.206848a5.chunk.js 200 [0.01, 172.23.0.4:33826] DJ INFO 2023-05-30 10:34:55,409 runserver HTTP GET /static/js/main.f1889776.chunk.js 200 [0.01, 172.23.0.4:33848] DJ INFO 2023-05-30 10:34:55,412 runserver HTTP GET /static/jupyter-theme-light.css 200 [0.01, 172.23.0.4:33856] DJ INFO 2023-05-30 10:34:59,418 runserver HTTP GET /static/js/2.206848a5.chunk.js.map 200 [0.00, 172.23.0.4:33868] DJ INFO 2023-05-30 10:34:59,826 runserver HTTP GET /static/js/main.f1889776.chunk.js.map 200 [0.00, 172.23.0.4:33880] DJ INFO 2023-05-30 10:35:00,126 runserver HTTP GET /static/mercury_logo.svg 200 [0.00, 172.23.0.4:33882] DJ INFO 2023-05-30 10:35:00,128 runserver HTTP GET /static/css/main.80c2d41b.chunk.css.map 200 [0.00, 172.23.0.4:33886] DJ INFO 2023-05-30 10:35:00,159 runserver HTTP GET /static/css/2.c6cf5ff9.chunk.css.map 200 [0.00, 172.23.0.4:33892] DJ INFO 2023-05-30 10:35:00,167 runserver HTTP GET /static/media/fontawesome-webfont.20fd1704.woff2 200 [0.01, 172.23.0.4:33906] DJ INFO 2023-05-30 10:35:00,177 runserver HTTP GET /api/v1/get-site/single-site/ 200 [0.01, 172.23.0.4:33910] DJ INFO 2023-05-30 10:35:00,303 runserver HTTP GET /api/v1/get-site/single-site/ 200 [0.01, 172.23.0.4:33918] DJ INFO 2023-05-30 10:35:03,969 runserver HTTP GET /api/v1/1/getnb/test/ 200 [0.01, 172.23.0.4:43048] DJ INFO 2023-05-30 10:35:04,104 runserver HTTP GET /media/test-ce5dd.html 200 [0.01, 172.23.0.4:43056] DJ INFO 2023-05-30 10:35:04,170 runserver HTTP GET /static/favicon.ico 200 [0.00, 172.23.0.4:43066] DJ INFO 2023-05-30 10:35:04,223 runserver WebSocket HANDSHAKING /ws/client/1/631f4f5d-b2e2-4c9c-958e-dc3b1bca9ff5/ [172.23.0.4:43070] DJ INFO 2023-05-30 10:35:04,230 runserver WebSocket CONNECT /ws/client/1/631f4f5d-b2e2-4c9c-958e-dc3b1bca9ff5/ [172.23.0.4:43070] DJ INFO 2023-05-30 10:35:04,282 runserver HTTP GET /static/manifest.json 200 [0.00, 172.23.0.4:43080] [2023-05-30 10:35:05,301: INFO/MainProcess] Task apps.ws.tasks.task_start_websocket_worker[f9388340-344c-431f-bcd6-4dc2a5dbfe90] received [2023-05-30 10:35:05,301: DEBUG/MainProcess] TaskPool: Apply (args:('apps.ws.tasks.task_start_websocket_worker', 'f9388340-344c-431f-bcd6-4dc2a5dbfe90', {'lang': 'py', 'task': 'apps.ws.tasks.task_start_websocket_worker', 'id': 'f9388340-344c-431f-bcd6-4dc2a5dbfe90', 'shadow': None, 'eta': None, 'expires': None, 'group': None, 'group_index': None, 'retries': 0, 'timelimit': [None, None], 'root_id': 'f9388340-344c-431f-bcd6-4dc2a5dbfe90', 'parent_id': None, 'argsrepr': "({'notebook_id': 1, 'session_id': '631f4f5d-b2e2-4c9c-958e-dc3b1bca9ff5', 'worker_id': 242, 'server_url': 'wss://mercury.cm157.pt.eu.org'},)", 'kwargsrepr': '{}', 'origin': 'gen1566@16b64a939132', 'ignore_result': False, 'properties': {'correlation_id': 'f9388340-344c-431f-bcd6-4dc2a5dbfe90', 'reply_to': '5e5be81b-4cc1-3527-84bd-0da7e56a3092', 'delivery_mode': 2, 'delivery_info': {'exchange': '', 'routing_key': 'ws'}, 'priority': 0, 'body_encoding': 'base64', 'delivery_tag': '01a5deaa-a44f-4a4a-a5cd-982f9d31fea3'}, 'reply_to': '5e5be81b-4cc1-3527-84bd-0da7e56a3092', 'correlation_id':... kwargs:{}) [2023-05-30 10:35:05,306: DEBUG/MainProcess] NbWorkers per machine: 20 [2023-05-30 10:35:05,309: DEBUG/MainProcess] Workers count: 0 machine_id=16b64a939132 [2023-05-30 10:35:05,309: DEBUG/MainProcess] Start /VSCodeServer/.venv/bin/python /VSCodeServer/.venv/lib/python3.10/site-packages/mercury/apps/nbworker 1 631f4f5d-b2e2-4c9c-958e-dc3b1bca9ff5 242 wss://xxxxxxxxxxxxxxxxxxxxxxxx [2023-05-30 10:35:05,344: INFO/MainProcess] Task apps.ws.tasks.task_start_websocket_worker[f9388340-344c-431f-bcd6-4dc2a5dbfe90] succeeded in 0.03826467401813716s: None NB 2023-05-30 10:35:06,632 matplotlib data path: /VSCodeServer/.venv/lib/python3.10/site-packages/matplotlib/mpl-data NB 2023-05-30 10:35:06,636 CONFIGDIR=/config/.config/matplotlib NB 2023-05-30 10:35:06,637 interactive is False NB 2023-05-30 10:35:06,638 platform is linux NB 2023-05-30 10:35:06,653 CACHEDIR=/config/.cache/matplotlib NB 2023-05-30 10:35:06,952 Loaded backend module://matplotlib_inline.backend_inline version unknown. NB 2023-05-30 10:35:06,954 Loaded backend module://matplotlib_inline.backend_inline version unknown. NB 2023-05-30 10:35:07,084 Start NBWorker with arguments ['/VSCodeServer/.venv/lib/python3.10/site-packages/mercury/apps/nbworker', '1', '631f4f5d-b2e2-4c9c-958e-dc3b1bca9ff5', '242', 'wss://xxxxxxxxxxxxxxxxxxxxxxxxx'] NB 2023-05-30 10:35:07,084 Load notebook id=1 NB 2023-05-30 10:35:07,086 Starting new HTTPS connection (1): xxxxxxxxxxxxxxxxxxxxxx:443 NB 2023-05-30 10:35:07,166 https://xxxxxxxxxxxxxxxxxxxxxx:443 "GET /api/v1/worker/631f4f5d-b2e2-4c9c-958e-dc3b1bca9ff5/242/1/nb HTTP/1.1" 403 None NB 2023-05-30 10:35:07,167 Exception when notebook load, quit Traceback (most recent call last): File "/VSCodeServer/.venv/lib/python3.10/site-packages/mercury/apps/nbworker/rest.py", line 33, in load_notebook raise Exception("Cant load notebook") Exception: Cant load notebook ```

Do you have any other websocker related info after:

WebSocket CONNECT /ws/client/1/

?

Other docker implementation details

Both containers then also have this, with different X ``` networks: nginx_proxy_manager_network: ipv4_address: 172.23.0.X networks: nginx_proxy_manager_network: external: true driver: bridge ipam: driver: default config: - subnet: 172.23.0.0/24 ```

pplonski commented 1 year ago

May I ask you for help with debugging the issue. It is hard for me to reproduce your environment.

Please change two files.

  1. Please add print(response.status_code) above this line https://github.com/mljar/mercury/blob/22d8bf882a49e5871aca6fd67a2edacd31d617ff/mercury/apps/nbworker/rest.py#L32
  2. Please update the function https://github.com/mljar/mercury/blob/22d8bf882a49e5871aca6fd67a2edacd31d617ff/mercury/apps/workers/views.py#L20-L30

to:

class WorkerGetNb(APIView):
    def get(self, request, session_id, worker_id, notebook_id, format=None):
        try:
            print(session_id, worker_id, notebook_id)
            for w in Worker.objects.all():
                print(w)
                print(w.session_id, w.id, w.notebook.id)
            Worker.objects.get(
                pk=worker_id, session_id=session_id, notebook__id=notebook_id
            )
            print("check")
            nb = Notebook.objects.get(pk=notebook_id)
            print(nb)
            return Response(NotebookSerializer(nb).data)
        except Exception as e:
            print(str(e))
        return Response(status=status.HTTP_404_NOT_FOUND)

You can update files directly in the package code: /VSCodeServer/.venv/lib/python3.10/site-packages/mercury/apps/nbworker/rest.py

Here is typical output when connecting to the server and worker:

DJ INFO 2023-05-30 11:18:29,748 runserver WebSocket HANDSHAKING /ws/client/8/cf340520-5bbc-4881-935e-eedf3345b3e4/ [127.0.0.1:44360]
DJ INFO 2023-05-30 11:18:29,762 runserver WebSocket CONNECT /ws/client/8/cf340520-5bbc-4881-935e-eedf3345b3e4/ [127.0.0.1:44360]
DJ INFO 2023-05-30 11:18:29,783 runserver HTTP GET /media/altair-b317e.html 200 [0.01, 127.0.0.1:44336]
NB 2023-05-30 11:18:30,453 WS on_message {"purpose": "worker-ping"}
NB 2023-05-30 11:18:30,454 Total run time 8.51786208152771
NB 2023-05-30 11:18:30,454 Elapsed from last execution 8.504785060882568
NB 2023-05-30 11:18:30,454 Worker id=27 set state Running uuid komp2
NB 2023-05-30 11:18:30,458 Starting new HTTP connection (1): 127.0.0.1:8000
DJ INFO 2023-05-30 11:18:30,481 runserver HTTP POST /api/v1/worker/fee44d9a-2099-4833-880a-583a8a4dc24e/27/23/set-worker-state 200 [0.02, 127.0.0.1:44362]
NB 2023-05-30 11:18:30,482 http://127.0.0.1:8000 "POST /api/v1/worker/fee44d9a-2099-4833-880a-583a8a4dc24e/27/23/set-worker-state HTTP/1.1" 200 113
NB 2023-05-30 11:18:30,483 Send state Running
NB 2023-05-30 11:18:30,483 Worker id=27 exists
NB 2023-05-30 11:18:30,487 Starting new HTTP connection (1): 127.0.0.1:8000
DJ INFO 2023-05-30 11:18:30,506 runserver HTTP GET /api/v1/worker/fee44d9a-2099-4833-880a-583a8a4dc24e/27/23/worker 200 [0.02, 127.0.0.1:44364]
NB 2023-05-30 11:18:30,507 http://127.0.0.1:8000 "GET /api/v1/worker/fee44d9a-2099-4833-880a-583a8a4dc24e/27/23/worker HTTP/1.1" 200 113
NB 2023-05-30 11:18:30,507 Send state Running
[2023-05-30 11:18:30,636: INFO/MainProcess] Task apps.ws.tasks.task_start_websocket_worker[9b0564bc-ddb1-4e79-b771-2e672f971600] received
[2023-05-30 11:18:30,637: DEBUG/MainProcess] TaskPool: Apply <function fast_trace_task at 0x7f206be68f70> (args:('apps.ws.tasks.task_start_websocket_worker', '9b0564bc-ddb1-4e79-b771-2e672f971600', {'lang': 'py', 'task': 'apps.ws.tasks.task_start_websocket_worker', 'id': '9b0564bc-ddb1-4e79-b771-2e672f971600', 'shadow': None, 'eta': None, 'expires': None, 'group': None, 'group_index': None, 'retries': 0, 'timelimit': [None, None], 'root_id': '9b0564bc-ddb1-4e79-b771-2e672f971600', 'parent_id': None, 'argsrepr': "({'notebook_id': 8, 'session_id': 'cf340520-5bbc-4881-935e-eedf3345b3e4', 'worker_id': 28, 'server_url': 'ws://127.0.0.1:8000'},)", 'kwargsrepr': '{}', 'origin': 'gen2658688@komp2', 'ignore_result': False, 'properties': {'correlation_id': '9b0564bc-ddb1-4e79-b771-2e672f971600', 'reply_to': 'fc64dfcf-4983-3126-8688-b7227d0c10e4', 'delivery_mode': 2, 'delivery_info': {'exchange': '', 'routing_key': 'ws'}, 'priority': 0, 'body_encoding': 'base64', 'delivery_tag': 'e303270d-4dee-47d8-81a3-9e75ce750545'}, 'reply_to': 'fc64dfcf-4983-3126-8688-b7227d0c10e4', 'correlation_id': '9b0564bc-ddb1-4e79-b771-2e672f971600',... kwargs:{})
[2023-05-30 11:18:30,645: DEBUG/MainProcess] NbWorkers per machine: 20
[2023-05-30 11:18:30,649: DEBUG/MainProcess] Workers count: 2 machine_id=komp2
[2023-05-30 11:18:30,650: DEBUG/MainProcess] Start /home/piotr/sandbox/mercury/mercury/menv/bin/python /home/piotr/sandbox/mercury/mercury/apps/nbworker 8 cf340520-5bbc-4881-935e-eedf3345b3e4 28 ws://127.0.0.1:8000
[2023-05-30 11:18:30,696: INFO/MainProcess] Task apps.ws.tasks.task_start_websocket_worker[9b0564bc-ddb1-4e79-b771-2e672f971600] succeeded in 0.0519577800296247s: None
NB 2023-05-30 11:18:31,666 matplotlib data path: /home/piotr/sandbox/mercury/mercury/menv/lib/python3.8/site-packages/matplotlib/mpl-data
NB 2023-05-30 11:18:31,669 CONFIGDIR=/home/piotr/.config/matplotlib
NB 2023-05-30 11:18:31,670 interactive is False
NB 2023-05-30 11:18:31,670 platform is linux
NB 2023-05-30 11:18:31,681 CACHEDIR=/home/piotr/.cache/matplotlib
NB 2023-05-30 11:18:31,903 Loaded backend module://matplotlib_inline.backend_inline version unknown.
NB 2023-05-30 11:18:31,904 Loaded backend module://matplotlib_inline.backend_inline version unknown.
NB 2023-05-30 11:18:31,999 Start NBWorker with arguments ['/home/piotr/sandbox/mercury/mercury/apps/nbworker', '8', 'cf340520-5bbc-4881-935e-eedf3345b3e4', '28', 'ws://127.0.0.1:8000']
NB 2023-05-30 11:18:31,999 Load notebook id=8
NB 2023-05-30 11:18:32,001 Starting new HTTP connection (1): 127.0.0.1:8000
DJ INFO 2023-05-30 11:18:32,018 runserver HTTP GET /api/v1/worker/cf340520-5bbc-4881-935e-eedf3345b3e4/28/8/nb 200 [0.02, 127.0.0.1:44366]
NB 2023-05-30 11:18:32,019 http://127.0.0.1:8000 "GET /api/v1/worker/cf340520-5bbc-4881-935e-eedf3345b3e4/28/8/nb HTTP/1.1" 200 1410
NB 2023-05-30 11:18:32,019 WS connect to ws://127.0.0.1:8000/ws/worker/8/cf340520-5bbc-4881-935e-eedf3345b3e4/28/
DJ INFO 2023-05-30 11:18:32,023 runserver WebSocket HANDSHAKING /ws/worker/8/cf340520-5bbc-4881-935e-eedf3345b3e4/28/ [127.0.0.1:44368]
DJ INFO 2023-05-30 11:18:32,030 runserver WebSocket CONNECT /ws/worker/8/cf340520-5bbc-4881-935e-eedf3345b3e4/28/ [127.0.0.1:44368]
NB 2023-05-30 11:18:32,031 Websocket connected
NB 2023-05-30 11:18:32,032 Open ws connection
NB 2023-05-30 11:18:32,032 Worker id=28 exists
NB 2023-05-30 11:18:32,032 Porcess msg {"purpose": "init-notebook"}
NB 2023-05-30 11:18:32,034 Starting new HTTP connection (1): 127.0.0.1:8000
c157fl commented 1 year ago

I'll be happy to help.

Here's the log after the two changes with two added break lines.

NB 2023-05-30 11:46:08,012 Start NBWorker with arguments ['/VSCodeServer/.venv/lib/python3.10/site-packages/mercury/apps/nbworker', '1', '15229bfc-9cb8-431d-b097-10d60e487cff', '279', 'wss://xxxxxxxxxxxxxxxxxxx']
NB 2023-05-30 11:46:08,012 Load notebook id=1
NB 2023-05-30 11:46:08,014 Starting new HTTPS connection (1): xxxxxxxxxxxxxxxxxxx:443

NB 2023-05-30 11:46:08,094 https://xxxxxxxxxxxxxxxxxxx:443 "GET /api/v1/worker/15229bfc-9cb8-431d-b097-10d60e487cff/279/1/nb HTTP/1.1" 403 None
403

NB 2023-05-30 11:46:08,094 Exception when notebook load, quit
Traceback (most recent call last):
  File "/VSCodeServer/.venv/lib/python3.10/site-packages/mercury/apps/nbworker/rest.py", line 34, in load_notebook
    raise Exception("Cant load notebook")
Exception: Cant load notebook
pplonski commented 1 year ago

Thank you! I still have no idea what is the reason.

What if you add this in rest.py:

print(response.status_code)
response.raise_for_status()

There are two things that are strange to me:

pplonski commented 1 year ago

Maybe try to force no authentication:

class WorkerGetNb(APIView):

    # no authentication
    authentication_classes = []

    def get(self, request, session_id, worker_id, notebook_id, format=None):
        try:
            Worker.objects.get(
                pk=worker_id, session_id=session_id, notebook__id=notebook_id
            )
            nb = Notebook.objects.get(pk=notebook_id)
            return Response(NotebookSerializer(nb).data)
        except Exception:
            pass
        return Response(status=status.HTTP_404_NOT_FOUND)
c157fl commented 1 year ago

Thank you! I still have no idea what is the reason.

What if you add this in rest.py:

print(response.status_code)
response.raise_for_status()

Added and directly get the following error:

NB 2023-05-30 13:22:14,446 https://xxxxxxxxxxxxxxxxxxxx:443 "GET /api/v1/worker/ea4c7e3d-23ff-44b9-9819-4809ed0f015e/299/1/nb HTTP/1.1" 403 None
403
NB 2023-05-30 13:22:14,447 Exception when notebook load, quit
Traceback (most recent call last):
  File "/VSCodeServer/.venv/lib/python3.10/site-packages/mercury/apps/nbworker/rest.py", line 33, in load_notebook
    response.raise_for_status()
  File "/VSCodeServer/.venv/lib/python3.10/site-packages/requests/models.py", line 1021, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://xxxxxxxxxxxxxxxxxxxx/api/v1/worker/ea4c7e3d-23ff-44b9-9819-4809ed0f015e/299/1/nb

There are two things that are strange to me:

  • you have no outputs from server (request doesn't reach the server),
  • do you think it might be possible that your proxy is adding some CSRF tokens to the requests?

Maybe... That could explain why django then complains about CSRF (#300). Any way to print those?

With the no authentication, and without the raise_for_status:

443 "GET /api/v1/worker/5e361c77-69b1-42f2-844b-bab844eed02d/312/1/nb HTTP/1.1" 403 None
403
NB 2023-05-30 13:34:59,063 Exception when notebook load, quit
Traceback (most recent call last):
  File "/VSCodeServer/.venv/lib/python3.10/site-packages/mercury/apps/nbworker/rest.py", line 35, in load_notebook
    raise Exception("Cant load notebook")
Exception: Cant load notebook
pplonski commented 1 year ago

Yes, it might be connected with #300. I need to search for solution.

In the meantime, could you please check on fresh web browser and incognito (private) tab if the same error occurs?

c157fl commented 1 year ago

On different browsers with protections disabled it still occurs.

Would there be any logs or similar that I could get that would help?

pplonski commented 1 year ago

Thank you @c157fl for your help.

I would check the request header that are send to the server. But it looks like the request is bounced from server, so maybe try to print response headers in the rest.py?

print(response.headers)
print(response.status_code)
print(response.content)
c157fl commented 1 year ago

So it seems cloudflare is blocking it. I'll try taking a look at it from my side later today or tomorrow and see if I can tweak something. I've added line-breaks to the code elements.

print(response.headers)

{'Date': 'Wed, 31 May 2023 16:22:16 GMT', 'Content-Type': 'text/html; charset=UTF-8', 'Transfer-Encoding': 'chunked', 
'Connection': 'keep-alive', 'Cache-Control': 'max-age=15', 'Expires': 'Wed, 31 May 2023 16:22:31 GMT', 'X-Frame-Options': 
'SAMEORIGIN', 'Report-To': '{"endpoints":[{"url":"https:\\/\\/a.nel.cloudflare.com\\/report\\/v3?
s=%2BIZfRNg2vm8JYS0qQp2r0wbqV0P2FqsVG2wGPcm2w3u0b9WxJ%2FcuakCLBu%2BDenWJ872DIZE%2FbUCuUGeVNW98HU
I4vYDdI8HuwM3SOsM6Ol2UsyXLbiedPo4FUDJuvRzwKKfJELasPBS61g%3D%3D"}],"group":"cf-nel","max_age":604800}', 'NEL': 
'{"success_fraction":0,"report_to":"cf-nel","max_age":604800}', 'Vary': 'Accept-Encoding', 'Server': 'cloudflare', 'CF-RAY': 
'7d008dff2be41c42-FRA', 'Content-Encoding': 'gzip', 'alt-svc': 'h3=":443"; ma=86400'}

print(response.status_code)

403

print(response.content)


b'<!DOCTYPE html>\n<!--[if lt IE 7]> <html class="no-js ie6 oldie" lang="en-US"> <![endif]-->\n<!--[if IE 7]>    <html 
class="no-js ie7 oldie" lang="en-US"> <![endif]-->\n<!--[if IE 8]>    <html class="no-js ie8 oldie" lang="en-US"> <![endif]--
>\n<!--[if gt IE 8]><!--> <html class="no-js" lang="en-US"> <!--<![endif]-->\n<head>\n<title>Attention Required! | 
Cloudflare</title>\n<meta charset="UTF-8" />\n<meta http-equiv="Content-Type" content="text/html; charset=UTF-8" 
/>\n<meta http-equiv="X-UA-Compatible" content="IE=Edge" />\n<meta name="robots" content="noindex, nofollow" 
/>\n<meta name="viewport" content="width=device-width,initial-scale=1" />\n<link rel="stylesheet" id="cf_styles-css" 
href="/cdn-cgi/styles/cf.errors.css" />\n<!--[if lt IE 9]><link rel="stylesheet" id=\'cf_styles-ie-css\' href="/cdn-
cgi/styles/cf.errors.ie.css" /><![endif]-->\n<style>body{margin:0;padding:0}</style>\n\n\n<!--[if gte IE 10]><!-->\n<script>\n  
if (!navigator.cookieEnabled) {\n    window.addEventListener(\'DOMContentLoaded\', function () {\n      var cookieEl = 
document.getElementById(\'cookie-alert\');\n      cookieEl.style.display = \'block\';\n    })\n  }\n</script>\n<!--<![endif]--
>\n\n\n</head>\n<body>\n  <div id="cf-wrapper">\n    <div class="cf-alert cf-alert-error cf-cookie-error" id="cookie-alert" 
data-translate="enable_cookies">Please enable cookies.</div>\n    <div id="cf-error-details" class="cf-error-details-
wrapper">\n      <div class="cf-wrapper cf-header cf-error-overview">\n        <h1 data-translate="block_headline">Sorry, you 
have been blocked</h1>\n        <h2 class="cf-subheadline"><span data-translate="unable_to_access">You are unable to 
access</span> DOMAIN WITHOUT SUBDOMAIN </h2>\n      </div><!-- /.header -->\n\n      <div class="cf-section cf-
highlight">\n        <div class="cf-wrapper">\n          <div class="cf-screenshot-container cf-screenshot-full">\n            \n           
  <span class="cf-no-screenshot error"></span>\n            \n          </div>\n        </div>\n      </div><!-- /.captcha-container --
>\n\n      <div class="cf-section cf-wrapper">\n        <div class="cf-columns two">\n          <div class="cf-column">\n            
<h2 data-translate="blocked_why_headline">Why have I been blocked?</h2>\n\n            <p data-
translate="blocked_why_detail">This website is using a security service to protect itself from online attacks. The action you just 
performed triggered the security solution. There are several actions that could trigger this block including submitting a certain 
word or phrase, a SQL command or malformed data.</p>\n          </div>\n\n          <div class="cf-column">\n            <h2 
data-translate="blocked_resolve_headline">What can I do to resolve this?</h2>\n\n            <p data-
translate="blocked_resolve_detail">You can email the site owner to let them know you were blocked. Please include what you 
were doing when this page came up and the Cloudflare Ray ID found at the bottom of this page.</p>\n          </div>\n        
</div>\n      </div><!-- /.section -->\n\n      <div class="cf-error-footer cf-wrapper w-240 lg:w-full py-10 sm:py-4 sm:px-8 mx-
auto text-center sm:text-left border-solid border-0 border-t border-gray-300">\n  <p class="text-13">\n    <span class="cf-
footer-item sm:block sm:mb-1">Cloudflare Ray ID: <strong class="font-semibold">7d008dff2be41c42</strong></span>\n    
<span class="cf-footer-separator sm:hidden">&bull;</span>\n    <span id="cf-footer-item-ip" class="cf-footer-item hidden 
sm:block sm:mb-1">\n      Your IP:\n      <button type="button" id="cf-footer-ip-reveal" class="cf-footer-ip-reveal-btn">Click 
to reveal</button>\n      <span class="hidden" id="cf-footer-ip">130.61.175.11</span>\n      <span class="cf-footer-separator 
sm:hidden">&bull;</span>\n    </span>\n    <span class="cf-footer-item sm:block sm:mb-1"><span>Performance &amp; 
security by</span> <a rel="noopener noreferrer" href="https://www.cloudflare.com/5xx-error-landing" id="brand_link" 
target="_blank">Cloudflare</a></span>\n    \n  </p>\n  <script>(function(){function d(){var b=a.getElementById("cf-footer-
item-ip"),c=a.getElementById("cf-footer-ip-reveal");b&&"classList"in b&&
(b.classList.remove("hidden"),c.addEventListener("click",function(){c.classList.add("hidden");a.getElementById("cf-footer-
ip").classList.remove("hidden")}))}var a=document;document.addEventListener&&a.addEventListener("DOMContentLoaded",d)})
();</script>\n</div><!-- /.error-footer -->\n\n\n    </div><!-- /#cf-error-details -->\n  </div><!-- /#cf-wrapper -->\n\n  
<script>\n  window._cf_translation = {};\n  \n  \n</script>\n\n</body>\n</html>\n'
pplonski commented 1 year ago

@c157fl great catch! I hope you will be able to enable it in Cloudflare.