glorietaru commented 3 years ago

Running justpy below the root url

I run a jupyter notebook at eg https://<my_site>/jupyter.
This is enabled by setting c.NotebookApp.base_url = '/jupyter/' in my jupyter_notebook_config.
Is there an equivalent justpy setting that will allow me to run a development version of justpy at eg https://<my_site>/dvl?
- (I'm running behind nginx with location /dvl making a proxy_pass to the justpy instance.)
- justpy reports: JustPy says: Page not found.
- I've tried running uvicorn --host 0.0.0.0 --port $JUSTPY_PORT --root-path "/dvl" server:app
  - but this does the opposite of what I want: "GET /dvl/dvl HTTP/1.1" 200 OK.

elimintz commented 3 years ago

I think the following example does what you want:

import justpy as jp

base_url = '/test_site'

@jp.SetRoute(base_url + '/')
def page1(request):
    wp = jp.WebPage()
    jp.Div(text='First page', classes='m-4 text-xl', a=wp)
    return wp

@jp.SetRoute(base_url + '/info')
def page2(request):
    wp = jp.WebPage()
    jp.Div(text='info page', classes='m-4 text-xl', a=wp)
    return wp

def error_page(request):
    wp = jp.WebPage()
    jp.Div(text=f'Unknown url error page for test version with base url: {base_url}', classes='m-4 text-xl', a=wp)
    return wp

jp.justpy(error_page)

Please let me know if this is not what you had in mind.

glorietaru commented 3 years ago

I'm not sure I follow this, to be honest.
My aim is to have production and development services on the same machine but at separate urls.

I may be going about this the wrong way. (But as I say, something similar works with jupyter.)

nginx

upstream justpy {
  server localhost:8000 fail_timeout=0;
}

upstream dvljustpy {
  server localhost:7999 fail_timeout=0;
}

       location / {
            proxy_pass http://justpy;  # justpy service.
            proxy_set_header X-Real-IP $remote_addr;
            proxy_set_header Host $host;
            proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
            proxy_set_header X-Forwarded-Proto https;
            # WebSocket support
            proxy_http_version 1.1;
            proxy_set_header Upgrade $http_upgrade;
            proxy_set_header Connection $connection_upgrade;
        }

The above all works fine with justpy running on :8000

But this doesn't:

    location /dvl {
        proxy_pass http://dvljustpy;  # justpy service.
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header Host $host;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto https;
        # WebSocket support
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection $connection_upgrade;
    }

justpy server code

base_url = "/dvl" if os.environ.get("JUSTPY_STAGE") == "dvl" else ""

....

@jp.SetRoute(base_url + "/")
def root(request):
wp = jp.WebPage(delete_flag=False)
""" useful code here """
return wp

....

jp.justpy()

Result:

JustPy says: Page not found
If I have instead:

jp.justpy(error_page)

The result is: Unknown url error page for test version with base url: /dvl.

elimintz commented 3 years ago

What happens when you change the last line to:

jp.justpy(error_page, port=7999)

And then got to /dv1/ ?

glorietaru commented 3 years ago

Unknown url error page for test version with base url: /dvl

Is there some way for me to investigate the JustPy says: Page not found response to see how it is reached?

elimintz commented 3 years ago

It is reached when no other route matches. What is the full url you are using and getting the error?

glorietaru commented 3 years ago

https://glorieta.kingsgroup.org/dvl

elimintz commented 3 years ago

Please try https://glorieta.kingsgroup.org/dvl/
(an extra slash at the end)

glorietaru commented 3 years ago

Embarrassingly, that works!

Sorry for the confusion.

On Thu, 18 Feb 2021 at 17:14, elimintz notifications@github.com wrote:

Please try https://glorieta.kingsgroup.org/dvl/ (an extra slash at the end)

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/elimintz/justpy/issues/220#issuecomment-781457192, or unsubscribe https://github.com/notifications/unsubscribe-auth/AQ5KW7L646CKOB3BVC2QB4LS7U4GVANCNFSM4XX6ONGA .

-- David Robinson Data Manager King's College, The British School of Alicante Glorieta del Reino Unido 5, 03008, Alicante (Spain) Tel: (+34) 965 106 351 david.robinson@kings.education www.kingscollegeschools.org At the forefront of British education internationally Spain (Madrid, Alicante, Elche, Murcia) • Latvia (Riga) • Panama (Panama City)

This message and any attachments are confidential and are protected by current legislation. If you are not the intended recipient please let us know and immediately destroy the original message. In compliance with the Regulation (EU) 2016/679 General Data Protection and the Organic Law of Protection of Personal Data and Guarantee of Digital Rights, 3/2018, 5 December, we inform you that your personal data are subject to treatment by King’s Group, with CIF B86161262, addressed in Avda. Pio XII, 92, 28036 Madrid, or Oldwood Road, Tenbury Wells, Worcestershire, WR15 8PH, UK, and they will be treated to meet your requests for information based on your email sent. The data will not be transferred to third parties except legal obligation. You have the right to withdraw consent at any time, to request access to and rectification or erasure of personal data or restriction of processing concerning the data subject or to object to processing as well as the right to data portability, and right to object as well as other rights, as explained in the additional information. ou can consult the additional and detailed information on data protection on our website https://www.kingscollegeschools.org/data-protection-policy/ or by going directly to our company. We have a Data Protection Officer who can be reached by e-mail to dpo@kingsgroup dpo@kingsgroup.org.

Este mensaje y cualquier documento adjunto son confidenciales y están protegidos por la legislación vigente. Si usted no es el destinatario deseado, avísenos y destruya inmediatamente el mensaje original. En cumplimiento del Reglamento (UE) 2016/679 de Protección General de Datos y la Ley Orgánica de Protección de Datos Personales y Garantías de Derechos Digitales, 3/2018 de 5 de diciembre, le informamos que sus datos personales están sujetos a tratamiento por parte de King's Group, con CIF B86161262, con domicilio en Avda. Pío XII, 92, 28036 Madrid, o en Oldwood Road, Tenbury Wells, Worcestershire, WR15 8PH, Reino Unido, y serán tratados para satisfacer sus solicitudes de información basadas en su correo electrónico enviado. Los datos no serán transferidos a terceros, salvo obligación legal. Usted tiene derecho a retirar el consentimiento en cualquier momento, a solicitar el acceso y la rectificación o la supresión de los datos personales, asi como la limitación del tratamiento en relación con el interesado o el objeto del procesamiento, así como el derecho a la portabilidad de los datos y de oposici&n , así como otros derechos, como se explica en la información adicional. Puede consultar la información adicional y detallada sobre protección de datos en nuestro sitio web https://www.kingscollegeschools.org/data-protection-policy/ o dirigiéndose directamente a nuestra compañía. Tenemos un Delegado de Protección de Datos a quien puede contactar por correo electrónico a dpo@kingsgroup dpo@kingsgroup.org.

rodja commented 3 years ago

When trying the above configuration the justpy main page loads successfully. But ressources like vue.js are loaded from /templates/local and not /test_site/templates/local.

elimintz commented 3 years ago

If you set the NO_INTERNET configuration variable to False, the libraries will be loaded from the internet. https://justpy.io/tutorial/configuration/#working-locally-without-an-internet-connection

Alternatively, you can change the template files, but that is a more complex. https://justpy.io/tutorial/static/#advanced-configuration

What I do is run the test system on the same url but use a different port (8001 for example). You can supply the port directly in the justpy command and it overrides what is found in the justpy.env file. This requires using a test_flag also and is not as elegant a solution as I would like.

A cleaner solution is required for setting up a testing site with a different url. I will try to figure this out.

rodja commented 3 years ago

If you set the NO_INTERNET configuration variable to False, the libraries will be loaded from the internet. https://justpy.io/tutorial/configuration/#working-locally-without-an-internet-connection

Nice hack. This seems to work for my scenario. Almost. Somehow the page constantly reloads now about every 2 seconds.

Alternatively, you can change the template files, but that is a more complex. https://justpy.io/tutorial/static/#advanced-configuration

What I do is run the test system on the same url but use a different port (8001 for example). You can supply the port directly in the justpy command and it overrides what is found in the justpy.env file. This requires using a test_flag also and is not as elegant a solution as I would like.

Sure, a different port would work. But we want to run multiple Justpy containers beside other microservices. All managed and routed by a single Traefik proxy.

A cleaner solution is required for setting up a testing site with a different url. I will try to figure this out.

Maybe you need to pass the referer path of the request to the template:

from urllib.parse import urlparse
...
context = {'request': request, 'page_id': load_page.page_id, 'justpy_dict': json.dumps(page_dict, default=str),
                  'use_websockets': json.dumps(WebPage.use_websockets), 'options': template_options, 'page_options': page_options,
                  'html': load_page.html, 'path_prefix': urlparse(request.headers['Referer']).path}

Then the templates could use this path_prefix like this:

<link href="{{ path_prefix }}templates/local/tailwind/base.css" rel="stylesheet">

rodja commented 3 years ago

It was a bit more complicated than stated above. See my pull request https://github.com/elimintz/justpy/pull/258 for a complete diff. I hope I've not missed something.

WolfgangFahl commented 1 year ago

Can this be closed?

rodja commented 1 year ago

Not jet. We found a nicer solution which I'll provide as a pull request in the next days.

rodja commented 1 year ago

I now created https://github.com/elimintz/justpy/pull/459 with the solution we have used in NiceGUI and already use in production environments. It uses the <base> tag to reduce the code clutter from our previous solution which can be viewed in https://github.com/elimintz/justpy/pull/258.

Also @frankie567 created https://github.com/elimintz/justpy/pull/451 which uses the url_for() call from Starlette to solve the problem.

To better compare and test the different approaches I've just pushed a reverse proxy docker setup: https://github.com/elimintz/justpy/commit/72f94a9f1e7fcd2734b3c947ba2e830387083698.

Usage:

cd docker/reverse_proxy_demo
docker-compose up -d
open browser at http://localhost:8000 to see the JustPy "hello world" served directly from uvicorn
open browser at http://localhost:8888/justpy to see the JustPy "hello world" served from behind Traefik reverse proxy at a non-root-path

rodja commented 1 year ago

@WolfgangFahl I don't think the issue is completed. You merged and then reverted https://github.com/justpy-org/justpy/pull/459. I'm surprised by both actions: the pull request was not ready because had not decided whether to use the <base> tag or not. And the revert happened due to failing tests. But the test have been passing on the pull request and the main branch is currently failing. Even without the changes made in https://github.com/justpy-org/justpy/pull/459.

WolfgangFahl commented 1 year ago

@rodja thanks for getting back to issues and pull request. Indeed the testing and release procedure has not settled to a satisfactory state at this point and quite a few of the recent actions were try and error style. The refactoring is currently risky and that's why i increased the release numbers to make sure people can revert to 0.2.x versions should the refactoring be too trouble some. This morning I'll work on #481 together with Tim and we'll at least check all http://jpdemo.bitplan.com/ examples and our own applications.

Please note that #478 now uses inheritance for Routing and Route should therefore be replaced with JpRoute. The base_url handling could IMHO therefore be using the JpRouter functionality. To do this properly we IMHO need to do a better initialization of jp.app

app = Starlette(middleware=middleware,debug=DEBUG)

in comparison to https://www.starlette.io/applications/ and https://www.starlette.io/routing/ where the routes are parameters of the startup.

What compatbility issues do you see with the current 0.4.x state of affairs and are the planned milestones still ok for you?

Will the new routing approach work for your usescases:

from starlette.routing import Route, Match
import typing

class JpRoute(Route):
    '''
    extends starlette Routing

    see 
       https://www.starlette.io/routing/

       https://github.com/encode/starlette/blob/master/starlette/routing.py
    '''
    # map for all routes that are defined
    routesByPath={}

    @classmethod
    def reset(cls):
        JpRoute.routesByPath={}

    @classmethod
    def getFuncForRequest(cls,request):
        '''
        get the function for the given request

        Args:
            request: the starlette request

        Returns:
            Callable: the function that is bound to the path of the given request
        '''
        scope=request.scope
        return JpRoute.getFuncForScope(scope)

    @classmethod
    def getFuncForScope(cls,scope):
        '''
        get the function (endpoint in starlette jargon) for the given scope

        Args:
            path: the path to check
        Returns:
            Callable: the function that is bound to the given path 
        '''
        for _path,route in JpRoute.routesByPath.items():
            match,_matchScope=route.matches(scope)
            if match is not Match.NONE:
                func_to_run=route.endpoint
                return func_to_run
        return None

    def __init__(self, path: str, endpoint: typing.Callable,**kwargs):
        '''
        constructor
        '''
        # call super constructor
        Route.__init__(self, path=path,endpoint=endpoint,**kwargs)
        # remember my routes 
        JpRoute.routesByPath[path]=self

    def __repr__(self):
        return f'{self.__class__.__name__}(name: {self.name}, path: {self.path}, format: {self.path_format}, func: {self.endpoint.__name__}, regex: {self.path_regex})'

class SetRoute:
    '''
    Justpy specific route annotation
    '''

    def __init__(self, route, **kwargs):
        '''
        constructor

        Args:
            route(Route): the starlette route to set
            **kwargs: Arbitrary keyword arguments.
        '''
        self.route = route
        self.kwargs = kwargs

    def __call__(self, fn, **_instance_kwargs):
        '''
        Args:
            fn(Callable): the function
            **_instance_kwargs: Arbitrary keyword arguments (ignored).

        '''
        # create a new route
        JpRoute(path=self.route, endpoint=fn,  name=self.kwargs.get('name', None))
        return fn

rodja commented 1 year ago

I would rather like to wait until the code is more stable again. It's very time consuming to prepare and test a pull request. If things are changing a lot its not the best time to do more changes.

WolfgangFahl commented 1 year ago

With #502 fixed it should now be possible to use standard starlette mounting/routing system

frankie567 commented 1 year ago

@WolfgangFahl Unless I'm mistaken, I don't think the underlying problem of serving Justpy through a path prefix is solved. There are still URL that are generated statically, in particular the WebSocket URL:

https://github.com/justpy-org/justpy/blob/8e903b0e7225d00af010aec60bd58092b26c8e05/justpy/templates/js/justpy_core.js#L193-L202

Could you clarify this?

WolfgangFahl commented 10 months ago

see #685

justpy-org / justpy

Equivalent of base_url? #220

Running justpy below the root url

nginx

justpy server code

Result: