Namespacing component ID support for large or multi-page apps

[valentijnnieman]

I wanted to use a code example - loaded with tools.load_example- more than once and found I couldn't because of errors regarding the outputs of callbacks already being in use. This issue is to discuss how we should handle having an example be used more than once. We could add a random id to the outputs, or some sort of prefix like page name. Let me know what you think!

[alexcjohnson]

@valentijnnieman I moved this over to the dash repo because I feel like we may want a more generic solution for multi-file apps. I'm thinking about something like:

In app1.py, instead of importing app and calling app.callback, do something like:

from dash import Section
section1 = Section()

section1.layout = html.Div(...)

@section1.callback(...)
def ...

And then the main app.py file:

from app1 import section1

app = Dash()
app.section(section1, scope='section1')

And then it would be up to you to include section1.layout (which would have its ids modified by the scope as well) wherever you want.

That way you wouldn't need the index.py dance, app.py could be the single top-level entry point, and it pulls in sections and scopes their ids, so each sub-app wouldn't need to know anything about the ids in the others.

[ned2]

Reviving this issue (and taking the liberty to update the title to reflect what I think this is really about).

I've been struggling with managing even medium size multi-page Dash apps due to the restriction of component IDs needing to be unique across the entire app. I've started up a thread around this issue on the community forums and I think the discussion has shown that this is a common issue that people are coming up with workarounds for. It would be great to see Dash add support to its API to solve this problem.

I really like @alexcjohnson's sketch of a proposal. One feature that would be nice to have support for, is Sections being able to be run as standalone apps, for debugging and better code-reusability. That way a single Python file containing a section could be run as a standalone Dash app, or be a page in a multi-page app.

[sjtrny]

One feature that would be nice to have support for, is Sections being able to be run as standalone apps, for debugging and better code-reusability. That way a single Python file containing a section could be run as a standalone Dash app, or be a page in a multi-page app.

I've just added a variant of this to https://github.com/sjtrny/dash-multipage. It seems like it is a feasible extension to the existing Dash framework without changing any internals. However there might be other reasons where changing the internals would make life easier (like namespacing).

From https://github.com/sjtrny/dash-multipage/blob/master/standalone.py

from page1 import Page1

external_stylesheets = ['https://codepen.io/chriddyp/pen/bWLwgP.css']

app = Page1.as_app(__name__, external_stylesheets=external_stylesheets)

if __name__ == '__main__':
    app.run_server(debug=True)

[alexcjohnson]

It may be helpful to think about this in terms of two distinct use cases: (1) multiple independent pages (2) composable sections that may or may not exist together on one page

For (1) a solution like @sjtrny's where each app is registered independently with Flask has a lot of advantages: simple structure, no concerns about id collisions, only load the callbacks needed on that page - making it more likely you can leave callback validation enabled. One thing you might not expect though: I think every page will still load all the component packages used by any page, because of the way we implicitly register them. So if you have big components like cytoscape or dash-bio on any page they'll load in all the pages.

So performance-wise, unless we figure out a better way to manage dependencies, it's still better to use the current recommendation where each page is actually a callback response updating a portion of the layout https://dash.plot.ly/urls - that way you also need to load all the component packages up front, but at least you don't have to reload them every time you change pages. Also this solution allows the pages to share some pieces (like navigation elements and dcc.Store components for common state) But you do need to worry about id collisions, and you can't validate callbacks.

(2) is the use case that inspired this issue - specifically in the context of dash-docs where we have multiple example "apps" on one page and id collisions are hard to avoid. But you can also imagine wanting to use the same section multiple times on one page with slightly different inputs - say you choose a list of graphs to display, with different data but all the same controls for each graph.

So I still think I like the general framework I proposed above for both of these use cases. I think making the containing app responsible for placing each section in a unique scope (note: sections should also be able to sub-scope other sections) is really the only way to manage id collisions, and it may enable us to reinstate callback validation by handling it at the section level - so when you're not displaying a section you get no errors if that section's callbacks refer to nonexistent elements.

Where I left off is thinking about how to connect data / callbacks into, out of, and between sections. Probably need to work through some specific use cases to come up with a good solution there.

[ned2]

Thanks for getting the discussion rolling again @alexcjohnson!

I'm also in favour of your approach sketched above. I see it working well for both scenario 1 and 2 you outlined. As I mentioned above, it would be nice to provide an affordance like @sjtrny included, whereby it's easy to run a section as a standalone app, not as the primary way of registering it scoped in a parent application, but to facilitate testing/code re-use outside of the parent app. Although I guess it's not too hard to create a Dash instance and register the section's layout with it inside a __name__ == '__main__' test.

What sort of thing do you think would be helpful in terms of concrete use-cases to drive this forward? I would be up for putting some examples together.

[alexcjohnson]

@ned2 examples would be very helpful - thanks! Most of the potential uses in dash-docs (which is where I personally would first use this feature) are decoupled - the Section wouldn't need any input from elsewhere in the app, nor would it provide any output to be used elsewhere. The API for something like that is easy to design. But I suspect that many if not most of the real-world applications of this will need coupling in one of those two ways, and I'd like to use these use cases to figure out what that should look like.

So for example, a Section for a detailed report in a Tab, after you've already provided input parameters and seen an overview of the output in a different tab? Or the other way, a Section for configuration settings that will be used elsewhere in the app. Or both together, like an "advanced config" tab that needs to use settings from the "basic config" part to tailor the available options, then the complete outputs are used by the rest of the app. Those are some of the scenarios that come to mind, I'd just like to make them concrete enough that we can design an effective API to support them.

[sjtrny]

making it more likely you can leave callback validation enabled

Hopefully in the future we won't even need to worry about it if someone has time to work on https://github.com/plotly/dash/issues/519

The terminology around "Sections" is a little confusing to me. I consider a section of a website "a group of pages" e.g. the "sports section" of a news website. Whereas some of the discussion seems to indicate that a section is a collection of elements on a single page. Which one is it and if its the later perhaps a different name would be better.

I think it might be helpful to compile a list of requirements which we think Multi-Page support should cover. My list is:

1. Avoid the index.py and app.py juggle
2. Page definition is agnostic to whether it is in a multi-page app or single page app
3. A dash app can use a single or multiple pages
4. Pages can be nested or grouped (my definition of sections)
5. Simple routing system
6. No (or little) performance reduction

I'm sure I've forgotten some but I think those are a good starting point, which would be widely used by the community.

[wtfzambo]

I like the idea of "sections" proposed by @alexcjohnson . Not much because of the id namespace problem (which I avoided prefixing every id with its page name), but because of a need I have right now in the app I'm developing: I want to load the callbacks of a page only when that page is actually visited and not before, and right now the only way for me to do it would be nesting all callbacks inside a function and call it on page visit, which is quite un-elegant.

More details here about my use case: dash forum post

[sjtrny]

Just want to note that there are many people coming up with their own solution to this problem, e.g.

https://community.plot.ly/t/show-and-tell-dash-single-page-application-spa-framework/29151 and https://community.plot.ly/t/show-and-tell-full-authentication-flow-example-dash-auth-flow/27415

including adding support for authentication. Clearly there is a large amount of interest in this area.

[chriddyp]

Another approach and discussion in the community forum about encapsulating components: https://dash-building-blocks.readthedocs.io/en/latest/overview.html & https://community.plotly.com/t/object-oriented-dash-components/10809/4

[sdementen]

I had a similar need but not for multi-page apps but for reusable components (layout+callbacks). You can see the API use on https://github.com/sdementen/dash-extensions/tree/composed-components#composed-components. There is a fair bit of id mangling to 1) avoid id clashing and 2) support pattern matching at all level (for callbacks outside the ComposedComponent and for callbacks registered inside the ComposedComponent).

A related issue is #1465 to allow to make this type of package available as a potential Dash extension.

[calang]

@sdementen : I checked composed-components and it looks pretty cool; well documented code. I would like to use it, but wonder: how stable is the code at this point?, especially since it's still a branch of its base repo. How would you recommend to install it? -- I didn't find this in pypi just yet. Copy source and local install, perhaps? Thanks in advance.

[sdementen]

@calang I just read https://community.plotly.com/t/modular-reusable-python-components-with-attached-callbacks/53998 and I think it would be a safer bet than using my branch

[calang]

@sdementen thanks a lot for the quick reply. I hadn't seen that other thread. Time to chip in and make suggestions, it seems. Thanks.

On Thu, 1 Jul 2021 at 11:56, sdementen @.***> wrote:

@calang https://github.com/calang I just read https://community.plotly.com/t/modular-reusable-python-components-with-attached-callbacks/53998 and I think it would be a safer bet than using my branch

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/plotly/dash/issues/637#issuecomment-872441287, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABYEDPOQ7ZER3U4SXBNXM33TVST5XANCNFSM4G4W5DHQ .

-- carlos lang-sanou tel +506 8817 0968