Open mohanarpit opened 3 years ago
I'd add one more success criteria, users should be able to understand how Appsmith works in a few seconds without needing to read a document.
I'm breaking this down a little more to see if it can add value to how we think about global actions:
1) The reason local actions are a pain point is because it causes unnecessary duplication when trying to reuse APIs across pages. Hence the objective with making actions visible across pages is only to re-use the "function" and not the "data". 2) If we go the global actions route, these actions would then be idempotent and stateless. This applies to headers, body, authorization mechanisms, and response data (whichever ones apply for particular actions). 3) In effect, each time a global action is used, it creates a locally available action that has a "reference" to the global action to replicate its "function", but retains all page level data within this copy. 4) This copy would then be responsible for configurations that only make sense at page level like "onPageLoad" or referring to page level widgets. In this way, we don't have to set variables for each call, and the data is visibly stored at page level for reuse.
This reasoning fails if we are also trying to share state across pages. :)
Does solving this issue #1920 help us with this problem? It seems like making it easy to reuse APIs will solve the problem for APIs at least. We'll have to expand the scope of the issue to cover it though.
Does solving this issue #1920 help us with this problem? It seems like making it easy to reuse APIs will solve the problem for APIs at least. We'll have to expand the scope of the issue to cover it though.
No. That is a separate issue. In this particular issue, we are referring to actions in a generic sense. These could be APIs, DB queries or other JS functions. #1920 refers to API actions specifically.
We may to re-think the UX where a specific API action will have 2 CTAs: "Store as datasource" & "Convert to global action".
3. In effect, each time a global action is used, it creates a locally available action that has a "reference" to the global action to replicate its "function", but retains all page level data within this copy. 4. This copy would then be responsible for configurations that only make sense at page level like "onPageLoad" or referring to page level widgets. In this way, we don't have to set variables for each call, and the data is visibly stored at page level for reuse.
@nidhi-nair Can you please expand on both these points? I'm not sure I understood the UX of having a "local copy" of an action in the page.
@mohanarpit The premise for this proposal is that we distribute storage of functionality and state between global and local actions. The global actions would be the single source of truth for functionality. On each page, this functionality might need to be used differently depending on the state of the local actions.
Example of function
: query + datasource
Example of state
: dynamic bindings + headers
When a user converts an existing action to global (or simply creates a new global action), they are basically storing the skeleton function
for that action globally. Such actions cannot be edited locally. In their local version, users can only edit their parameters or state
.
The reason we maintain a separate copy locally is so that the data that is retrieved from these calls can be stored without having to execute actions for every use. For example, a trivial JSFunctionGlobal might be a transformation that converts an ordered list to a checked list. JSFunctionLocal1 exists within Page1, applies JSFunctionGlobal with its local dynamic binding {{List1}}. JSFunctionLocal2 might also be another application that exists in the same page for some other state.
The important thing to note in this solution is that we are disallowing sharing of state between pages via actions. State sharing, if needs to happen, will need to be introduced elsewhere (the store I think allow that?).
Another extension could also be default (non-dynamically bound) states for global actions.
Root assumption: This tool is for people who can read/write code.
It is easier for devs to read code/pseudo-code, than to build a mental model by trial and error or learn an unfamiliar layer of indirection.
Unless our "visual IDE" or "framework" gives a significant improvement over that in UX, it's better to stick to the default mental model of doing things.
This is more a continuation of
The global actions will have variables of the nature "x, y"
and
users should be able to understand how Appsmith works in a few seconds without needing to read a document.
The examples below highlight some of the implementation details as a means to explain the philosophy.
arg.X
here, not the best choice IMHO)PS: The tables are meant to be a replacement for UI 😅
action name | getPost |
---|---|
URL | https://example.com/api/users/{{arg.userID}}/posts/{{arg.postID}} |
METHOD | GET |
actions.getPost = (userID, postID) => {
return fetch(`https://example.com/api/users/${userID}/posts/${postID}`);
}
action name | addComment |
---|---|
URL | https://example.com/api/users/{{arg.userID}}/posts/{{arg.postID}/comment |
METHOD | POST |
HEADERS | Authorization: Bearer {{appsmith.store.key.authToken}} |
BODY | { comment: {{arg.text}} } |
// Note: The pseudo-code ignores code to interpolate variables referenced from global scope.
actions.addComment = (userID, postID, text) => {
const headers = {
Authorization: "Bearer {{appsmith.store.key.authToken}}"
};
const body = JSON.stringify({comment: `${text}`})
return fetch(
`https://example.com/api/users/${userID}/posts/${postID}/comment`, {
method: "POST",
headers: headers,
body: body,
});
}
{{
actions.getPost(
userTable.SelectedRow.id,
postTable.SelectedRow.id
).then((response)=>{})
}}
fetch
.Probably make the UI look like a function signature for immediate connect?
PS: The proposal here is living and breathing, it's not all or nothing.
Perhaps I'm not fully grasping the concept of Global actions, but I don't feel very strongly for it.
In my head, pages are like files (or modules, as called in some programming languages). Anything that can run (a function, a class etc.), has to live inside a module. Any module. It cannot live outside of modules, i.e., things can't live outside files, which is what global actions are trying to do. So, global actions feels more like PHP's superglobals that are magically available in all modules everywhere. Convenient at first, but can quickly get annoying in large applications.
Also, with the existence of global actions, what would be the appeal of page-local actions? Just namespacing?
So I'm thinking of a different approach (this has been suggested before, I forget who and when). The idea is natural if we continue the above analogy. Pages are modules, and modules can import other modules. Such a thing can work like {{page2.api1.run()}}
(syntax and semantics can be talked about).
This actually solves another long-standing shortcoming. Today, an action cannot use the values of input widgets from two different pages. It can only access values of input widgets from the action's owner page. If pages are modules that can be referred to when needed, widgets inside those pages can also be referred to as {{page2.input1.text}}
or something like that.
Obviously, the major hurdle here is that page names today don't have to be valid variable names. The above code fragments won't pass in such cases. We could introduce a page
callable as a replacement for that, so {{page("Historical Data").api1.run()}}
or {{page(2).api1.run()}}
could work. If we go this way, we have to make sure renaming a page updates calls like this.
P.S.: If we want to do global actions, wouldn't it be simpler to provide a runApiAction(...)
function that can be used in JS Pane, where they can call the function with whatever arguments they want. If we do the JS Pane, might as well leverage it for this I think.
Update: We will allow for both global and local actions but not have a strong inherent distinction between them. Users should be able to define any action as global/local These are nothing but functions that are either reading from a variable outside their scope or have arguments defined in them to accept parameters
Next items: @appsmithorg/design-team to research ux on How do we define arguments in the queries? How do we define default values and work with values on the current page? How do we organise queries? How does a user send data to the query with arguments defined? (this.params experience is poor today)
@Nikhil-Nandagopal @mohanarpit @hetunandu @Rishabh-Rathod to research How should the evaluated value work? How do we handle multiple clashing names across pages?
Tools to research Wordpress Excel referencing data across cells and sheets Java / Ruby files have direct access to files in the folder that they are defined inside
Update: After a round of discussion, we've settled on a few things we want to enforce
To explain this, we came up with a mental model of introducing modules. A module is a building block of an application that can exist by itself. A module consists of a canvas file, a group of queries & javascript objects. Modules cannot reference files inside other modules and can have any number of files and folders inside them.
Modules can be made reusable inside an app by defining them as common modules and today will only consist of queries and javascript files. In the future, there will be an option to add the canvas as well to the common modules. Common modules are accessible by all other modules in the application.
JS & Queries inside modules can be abstracted into a common module for better reusability across the application.
Next Steps: @parth-appsmith @dzhenkov to come back with wireframes on these
@parth-appsmith presented wireframes today to show how arguments can be used, and that helped us to imagine the solution around arguments much better.
ApiName.run() is a platform function, and as such cannot be overridden. The platform should have a strictly defined structure to this function, and so in order to give maximum flexibility to the user with this constraint, the run function will accept two parameters:
why JSON object?
The decisions are:
@Nikhil-Nandagopal @mohanarpit @parth-appsmith @Rishabh-Rathod @hetunandu please correct if I've missed something.
cc @Nikhil-Nandagopal @parth-appsmith @Rishabh-Rathod @ajinkyakulkarni @hetunandu
how will we explain to a user that you can now access files inside a page module and utils? But files inside a page module cannot talk to files inside another page module?
how to organise these entities?
how to explain to the user that the query they are accessing are from utils and not from the page module (and vice versa)
navigation between entities in page vs utils
ability to convert existing page-scoped action into utils - out of scope for V1
how will actions be added? When should the user decide whether this should be page-scoped or application level action?
Pages and utils are in the same level in the hierarchy, but:
a. page can access utils
b. page cannot access another page
c. utils cannot access any page
d. so, would it be better to depict utils as a level higher than pages, separated by a "virtual layer". This will help us to add more things to this level, like external libraries
Should "canvas" be an entity within the "Page" folder? Might be needed if/when we introduce tabs. For this discussion, we will stick with Page.
Adding to the above points.
We also discussed how import global actions should work,
import
syntax similar to how the JS module works.
import { fun1, fun2 } from "common_module/util"
import
will work
@hetunandu @Rishabh-Rathod Dear, could you give any update on the planning for this very nice feature? :)
(and preferably, can we find some documentation what the labeling means to infer our self what is going on? e.g. the Actions pod label was removed, added and removed again 2 days ago. What does this mean? :) ) . thanks!!
@unknown1337 we'll be picking this up in a week or 2. The labeling is just an internal way for us to organize issue and doesn't have any bearing on whether an issue is in progress or not. Look forward to updates here where we'll keep you informed with early previews of the feature
A user on a sales call requested this today, is there an official way to upvote it or will this comment do?
@infinitetrooper commenting or adding a reaction works
+1 to this.
@st-trade can you explain your use-case here? We are starting to work on it soon. I would like to know more about the problem you are facing. If you have some time i would like to speak about this, can you please block a slot here - https://calendly.com/dilip_pitchika/30min
I've blocked a slot. It's basically to reuse domain models and use cases along the app.
+1 from my team as well. Our use cases range from checking if the user is logged in on every page to clearing some stored values on every page as well.
@Gabriel-Azevedo thanks for letting us know, could you help me understand the stored value part? Are you talking about removing the store login token or something else?
We use local storage a lot to save IDs and other properties that are used on a single page but our flow might require us to interact with that same object on a different page, so on each page load we clear the local storage of those values. if we had a shared global file, we could just call the same function.
Another use case is string formatting, on multiple places we want to turn a backend status like pending_review
into Pending User Review
, so we need to copy/paste this everywhere, if we want to change that string again, we need to find out all the places that use it.
+1
I would like to use AppSmith to call an internal service with endpoints defined in protobuf (accessible through either grpc or http). Can I enumerate the list of endpoints and valid input parameters + expected output response parameters globally, and enable application authors to use the API without having to define it
+1
If there are global queries and JSObjetcs the left panel gets too full. Whereas I find that it already is now. Maybe it makes sense to reserve the left panel only for the widgets and datasources etc. of the current page and to display global queries and JSObjects and the other data sources in the middle.
+1 please
+1 on this. My team has a huge internal app that we want to port towards appsmith, but the lack of reusable JSObjects / reusable APIs is a dealbreaker.
Hey @geekyme @acron0 we are working on it, i would like to talk to you folks to understand a few things. Will you be available to get on a call with me and discuss how you think this should work? If you are interested please find a slot which works here- https://calendly.com/dilip_pitchika/30min?month=2023-04&date=2023-04-13
📢 Hey everyone! We're working on the reusability feature for widgets, queries, and JavaScript, and we need your help to test the designs for the feature. Please let us know if you are interested. If you are, please go ahead and book a slot here for 1st week of may: Link 🙏
@acron0 @Team-wb @austin-stytch @Gabriel-Azevedo @st-trade @unknown1337
Protected Routes Whenever trying to access a protected page two validations should happen Verify if accessToken is present in the global store If yes then verify the accessToken to be valid by making an API Call Refresh Token The accessToken should be automatically refresh using the refresh token api call whenever an API call fails due to expired accessToken. If the refresh token api call also failed because out refresh token is also expired then the accessToken and refreshToken stored in the global store should be deleted and used must be logged out. Can we write one function and it can be triggered for all pages page load?
Just to chime in on our use-case:
We have a large system where we'd want to re-use a group of components on many pages. Some will be simple, such as menu bars etc. Others will be more complex like re-using a collection of dropdown filters and search boxes for a dataset.
We may also want to have something like "user profile" which groups an image, link and a few strings of text in a nice way that we can use as a profile widget in multiple locations.
@an2cabs this is what we want to deliver with reusable actions, any custom logic/JS you wish to share across pages and apps it should be possible
@spamoom i think you are looking for #5229, which is related to this. Will your widgets also contain data along with them or they are just UI pieces which you want to reuse?
It's a combination of both. There will be cases where ideally we'd have some sort of variable input widget to attach a table to, so the widget group ships with all the necessary logic + data flows/JS, but any "external" relations to other widgets.
Example as above, has auto-disabling UI, data fetched and populated for each widget for dropdowns. And possibly data populated from data sources on each page, not specific to the widget group itself.
Ultimately, I know this is not trivial and complex, so I'm keen to push on both tickets so we can achieve partial progress sooner - rather than copy-paste hell
+1 from my side. I am unable to get this clarity initially but for the help from the support team. Thanks to them for quickly pointing. Ideally I would like to see all the queries under the 'Data' (may be you can provide a drill down for all APIs indicating in which page these APIs are used)
@srichint1 you can block our time to get beta access to the feature https://calendly.com/pedro-appsmith/30min?hide_event_type_details=1&month=2024-01
I'm new to Appsmith, but DRY was one of my first questions after being forced to duplicate code on a page-by-page basis. The discord bot seems to think DRY is a priority in Appsmith, that JSObjects can be globally referenced already and gives a lot of misleading responses. Is there a known release date (cloud) for this functionality, I see the conversation has been going on for some years...
Hi @macasas 👋 This feature will be open as a beta preview in the end of march. It will only be available to self-hosted business edition users.
And another nail in the coffin for the community edition.
@codedmind I'm sorry you feel that way about the community edition but we are heavily investing in the developer experience for it today as we have for over 4 years now. We do have plans for introducing some mechanisms of reusability in the community edition as well but we want to start with the business edition first.
Is that a theme, nails in the coffin, am I wasting my time even playing with Appsmith cos its dead already and I just don't know it?
@Nikhil-Nandagopal don't get me wrong... but that is the feeling, every feature that was request/raised by the community ends "behind" the paywall, audit, SSO, now this...
It wouldn't be the first project that takes ideas from the community ... no names, nothing to see, move along ...
Back on subject, I'm only a month in, I have code functions I know I've written somewhere in Appsmith, and I can't even search to find them so I can copy it and add to my duplicate hell. Coming from a coding background I can't believe page scoping JS Objects was ever thought to be a good idea.
@macasas that is absolutely not true. Appsmith is very much active and well maintained by us. We have taken decisions to put some features behind a paywall to ensure that the project continues to thrive. There are many platforms like openblocks, internal.io and airplane.dev that have disappeared because they could not support the product anymore. We do not wish that future for Appsmith and have to monetise certain features that we believe organisations will pay for. @codedmind I understand your frustration with a request raised by the community being behind a paywall but I hope you understand that
Our business edition is quite reasonably priced with a usage model and I hope you can support the project by paying for it when you see enough value from it.
@Nikhil-Nandagopal i understand that features raised be community don't belong to community, but if you ask the community for feedback and then put that feedback gehinf paywall.. Also how can the community evalute the value of that features if when they are delivered are behind a paywall?
I really understand that the project must live and for that need monetisation, and about that i alredy give my feedback, charge for "simple" users that only access/view and app that was developed for someone don't fit every organizations, and break the "low code/no code" appeal but is just my opinion.
@Nikhil-Nandagopal yeah I get all that, but just so you understand that some of us are using the Community Edition to make sure that the business doesn't go down a dead-end path, so there is a trial and error period to make sure the Business edition is the right decision. With this issue alone, there is a real chance that I'm going to get strangled by duplicate code and suffocate before I even reach that point. DRY is a fundamental, like oxygen, starving users of it prevents growth.
@Nikhil-Nandagopal Hello. While I do agree Appsmith CE is very well equipped, and that financial sustainability is critical, I must say that having common blocks in every page (from widgets like navigation menus, to querys, to js code) is a basic low code functionality. Having this kind of basic, fundamental stuff only on the paid tiers is very disappointing and makes CE a questionable value proposition. I also don't want to hurt the project by being too vocal about this criticism, but in my view it is damaging to pay-wall any basic functionality. What is even more awkward is having in CE much more "advanced" functionality that could be easier to see pay-walled (custom widgets come to mind - not that I'm complaining!). I have discussed this in the past in the discord server, and have been observing in silence how this develops. At this time, the alternative to share code between pages is a complex, high-code approach, involving public github repos with a build step to produce esm modules that are then loaded as 3rd party libs. There is an irony in having to do this in a low code platform... So, let me circle back to the start, Appsmith is amazing, and the CE was fundamental to even consider a paid tier. Even if my main employer can afford the Business Ed (barely), I tend to use CE in other projects. So a single product serves my diverse needs. Things are tied together.
Problem
Today, all actions (API, DB Query, JS functions) created within Appsmith are page-scoped. This means that they are only accessible from within the page and not across pages within the same application. Often, this confuses users. They expect actions to be global to the entire application. This is because in normal coding practices, that is what they expect.
In order to resolve this, we need to create a way for users to create actions that are accessible across pages and are scoped within the application instead of the page.
Why were Actions page-scoped in the first place?
During dynamic binding, we want to give users a consistent experience. If they are able to bind
Input.text
in the UI canvas they should also be able to bindInput.text
in the API pane. If we treat Actions as pure functions, that would mean the action requires variables to be passed into it. This breaks the users mental model of how Appsmith works.Unfortunately, a side-effect of this is that if the user has bound
Input1.text
in the API, they can't re-use this API with other inputs. This leads to:Goals:
Input1.text
in one place, I should be able to logically bindInput1.text
in another place.Potential Solutions:
Have a section called "Global Actions". The user can then create Actions local to the page while also moving them to a global place. The global actions will have variables of the nature "x, y". These actions can now only be invoked via JS that sets the x & y values before calling
Api1.run()
.Any other thoughts here ... ?
Success criteria
A user should be able to define an Action once in the entire application and then re-use it in multiple places with different input parameters.
RACI
| ------------- | ------------- | | Responsible | @arunvjn @Rishabh-Rathod | | Accountable | @arunvjn | | Consulted | @Nikhil-Nandagopal, @mohanarpit @ajinkyakulkarni | | Informed | @parth-appsmith |