[Product Note] Debugging Experience

Nikhil-Nandagopal commented 3 years ago

Problem statement

Developers on appsmith encounter syntactical and logic errors and thus need a way to debug the errors that are created

Success criteria

Developers who see any error on appsmith should be able to understand why the error occurred and what they should do to fix the error. Core areas:

Property Pane Configuration
API / Query Configuration
Data Source Configuration

User story

The user while performing various actions on appsmith enters into an error state. His attention is drawn to the error and is given sufficient information on how to solve the error. He then takes the relevant action and the error disappears

Details

There are largely 2 types of errors that occur in appsmith

Syntax errors
Logical errors

Syntax Errors

These errors occur due to the user entering a value that is not compatible with the systems syntax specification. Since these errors violate a system constraint, they are detectable by the system and can be reported proactively. These errors are largely static and can be immediately reported.

An error in the binding syntax: Usually due to a missing bracket, undefined value, or unknown variable
An error while binding mismatched data types to a widget
An error in the query definition: Usually due to a mismatch in the query input and the db query language specification.
An error on the API: Usually due to invalid JSON being sent
An error in the data source config does not connect to the database correctly: Sometimes the db credentials might change causing existing data sources to stop working.

The core problems of syntax errors are:

Their error messages are poor
They are at times not discoverable
The instructions to fix them are not clear

Logical Errors

These are errors that occur due to the user making a logical mistake in their configuration such as missing a check or not enabling a certain configuration. Their inputs are all valid but do not result in their intended outcome which cannot be detected by the system because the user's intention cannot be gauged.

When a widget does not load the data in view mode because the query / API was not flagged as onPageLoad
When running an API fails due to incorrect values being sent
When running a query updates or fetches the wrong information
When triggering an action from a widget does not update another widget
When a widget is working in most cases when it was built but breaks suddenly in view mode because a null condition was not handled
When a query that was running fine suddenly starts timing out because the data set grew in size or network conditions deteriorated
When a query / DB connection fails to run because the user has not whitelisted our IP

The core problems of logic errors are:

They need sufficient related information to be presented in an easily consumable manner in order to debug them
The information should not overwhelm the developer and should allow drill-down

Solutions

Syntax Linting
Error console with clear messaging and CTAs
Application log with the changes in states
Entity relationships
Improved error messaging

Items not in the scope

This project will not cover ways to avoid making an error and only focus on how to recover from an error. Suggestions on how to avoid the error are welcome and will be logged as a separate task
Performance-related problems will not come under the scope of this project and the focus will remain on 0/1 scenarios i.e working / not working
This project will assume that the developer is able to reproduce 100% of the errors and is not to help developers reproduce an issue

Links

https://miro.com/app/board/o9J_kivj-zU=/ https://www.figma.com/file/EdC0j0XwxyC9Flqzxyoxuf/Debugger?node-id=387%3A670

Nikhil-Nandagopal commented 3 years ago

Console Logs (Dynamic Errors)

Action started execution from - Widget / User
- Request
Action completed execution in 100ms
- Response
Action timeout in 100ms
Action returns with an error in 100ms
- Error message : needs to be copyable
- Search the web for error?
- Request
Action Updated
Widget Updated
- Property changed
- Failed to update widget property
Action created
Widget created
Navigated to page1 with params { “key”: “value” }
StoreValue “accessToken” “asdadsqwqeqwas”
Downloading filename data
Show Message “Hello”, “Success”
Open Modal “Modal1”
Close Modal “Modal1”
Failed to execute Button1 onClick

Problems (static config errors)

[Table1] API1 is not defined
- Quick fix if available
[Dropdown1] options does not evaluate to type array < label: string, value, string>
- Example snippet
- Example data
[API1] post body is not a valid JSON
- , missing in line 2, col 14 : CTA to navigate to error

Nikhil-Nandagopal commented 3 years ago

Things yet to figure out

Relationships: Find Usages & Find References (Who all are updating me and who all am I updating)
Debugger Location: API pane / Query pane / Canvas / Data sources

ofpiyush commented 3 years ago

Updates from conversation with Nikhil:

Console log will show all events as they occur.
Problems tab will show active problems. We'll track problems at the property level.

Updates from conversation with Hetu: Property paths from widgets can be figured out today from the invalidProps, we'll need to add the ability to mark action execution being valid or broken.

Relationships: Find Usages & Find References (Who all are updating me and who all am I updating)

This is present in redux state now, we can pick it up whenever we feel the need to display it.

Nikhil-Nandagopal commented 3 years ago

@akash-codemonk adding a list of issues to solve in the debugger Issues:

[x] Logs messages should be copyable
[x] The [Action] or [Widget] in the log can be replaced by the name of the widget. So the message can read "[Table1] Widget Created". The name of the widget should be clickable in the message itself.
[x] When a widget property fails evaluation, add an error log "[Table1] tableData does not evaluate to type Array
"
[ ] Adding console.log("Hello") to the onClick of a button should display the log in the debugger

[x] Maintain a max queue for the logs like 1000 records and paginate it from the local storage so it does not slow down the app

[x] Widget deletion is not showing up

[x] Need an option to clear logs

[x] Show the logs in the query section also

[x] Action execution response should only the actual response and not the top-level object

[x] on action success, the message can be [Query1] Executed successfully

[x] Instead of Action property updated, the message can be [Query1] Configuration updated

[x] Message for action creation can be [Query1] Action Created

[x] On execution start, the message can be [Query1] Execution started, the body object should have only the final SQL / Query config that gets executed

[x] Display only updates for properties in Property Pane and internal properties (selectedRow)

[x] Internal property updates are not showing (clicking a selected row in a table)

[x] Trigger properties execution should be logged: [Button1] onClick Triggered, [Table1] onRowSelected Triggered

[x] Triggering Framework Actions should display an log of the format "navigateTo("Page1") was triggered"

[x] When the API fails, we should display the request that is shown in the request tab and not the request which went from the client in the log. The status code of the request should be shown in the message: [API1] Execution failed with status 401 200ms @Debsourabh should we also kill the request tab now? Why do we need it if we have logs?

[x] Deleting APIs / Queries should show

[x] Creating / Deleting / Updating / Testing Datasources should show in logs with the messages [MockDb] Datasource created [MockDb] Datasource deleted [MockDb] Datasource deletion failed due to 5 related queries [MockDb] Datasource configuration saved [MockDb] Test Connection successful [MockDb] Test Connection failed [MockDb] Datasource structure retrieved [MockDb] Datasource structure could not be retrieved
Nikhil-Nandagopal commented 3 years ago
Adding more feedback
- [x] The logs should auto-scroll to the latest log entry
- [x] The query pane is missing the debugger
- [x] API errors should be in red
- [x] The entity name should be in the filter when the debugger is opened from the entity
- [x] The debugger should be triggerable from the property pane error message
- [x] The API URL and other information are missing from the API execution request log
- [x] An error tab is needed which should have the current errors of all entities
- [x] If 2 updates are triggered back to back from the same entity, group them
- [ ] Debugger should be triggerable from the view mode if the user has edit access on the application
Nikhil-Nandagopal commented 3 years ago
- [x] API request does not match the format in the request tab
- [x] Kill the request tab on the API
- [x] Error tab is missing in the API pane
- [x] The copy icon can be replaced with a CTA to open the omnibar with the message pre-filled
- [x] Clicking anywhere on the log should expand it
- [x] The entity name should show at the right end of the log
- [x] There should be an X to clear the filter in the debugger
- [x] the errors should only go if the property which had the error got resolved. Now it goes if any property is fixed
- [x] If there is no error, we should open the logs by default
- [x] Logs and error are not showing on the query page
- [x] Empty state for when there are no logs / errors
- [x] Widgets need an error state
- [x] Dropdown remains in an error state saying selectedOptionValues does not match type Array/List
- [x] Widget property error messages don't match the error message in the evaluated value
- [x] The error message for some widgets isn't triggered the first time the widget goes into a bad state it needs 2 updates
- [x] Navigate / Show Alert Actions with invalid values should throw an error. There should be a toast message and it should show in the debugger
- [x] Show alert was triggered message has an extra closing bracket
- [x] There should be a debug request CTA on the Query Page when there is an error (same as on API pane today) which opens the debugger error with the error expanded
- [x] All error messages should have a debug button that will open the debugger log expanded. This will have a CTA to search the documentation in the omnibar with that error message
- [x] The debugger should be resizable
- [x] The success log background should not be green anymore there's a new design
Nikhil-Nandagopal commented 3 years ago
- [x] All filter is not working when you switch between success and error back to All
- [x] Show the entity name in the message
- [x] Request is showing headers as an array
- [x] Table error is disappearing if the table data is empty and the user clicks anywhere outside the table
- [x] Hide the request tab
- [x] Show request error message should take the user to the error tab on API / Query
- [x] The most recent error log should always be expanded
- [x] User should be able to resize the debugger to any height till the top bar
- [x] Opening the omnibar should not close the debugger
- [x] Don’t group error messages in the error tab
- [x] Hitting search from the top error message should search by the relevant human-readable message
- [x] Make query pane and API pane draggable response / logs section resizable and consistent
- [x] Debug button in the toasts are missing
Nikhil-Nandagopal commented 3 years ago
- [x] Add an empty state in omnibar to join discord
- [x] New CTA for the omnibar on the debugger and rest of the app
- [x] Don’t close debugger when navigating to an entity
- [x] Error log for property error should be the same as the log
- [x] Add default values to isRequired, isVisible and isDisabled of all widgets so that the error does not come when dropping it
- [x] Debugger error tab crashes when running this API and navigating to the logs tab https://api.razorpay.com/v1/orders?count=2&skip=1
- [x] When there is an error in action and no response, the UI needs to be updated
- [x] When there is an error in action and response the UI needs to be updated
- [x] Most recent log should be expanded by default
- [x] The debug button in the toast should be right aligned
- [x] The default height of the debugger when opened on the canvas should be 25% of the screen height
- [x] Add analytic events
  
  OPEN_DEBUGGER: { source: CANVAS | TOAST | API | QUERY | EVALUATED_VALUE }
  
  DEBUGGER_TAB_SWITCH: { tabName: LOGS|ERROR }
  
  DEBUGGER_ENTITY_NAVIGATION: { entityType: API|QUERY|WIDGET }
Nikhil-Nandagopal commented 3 years ago
- [x] If the table is deleted, the error does not disappear from the console
- [x] Log is showing showAlert(Label) was triggered instead of showAlert("Label")
- [x] Resize the section above the debugger in API / Query when the debugger is resized
- [x] Needs to be a close button in the datasource and add API page as well
- [x] Empty state in the error tab is missing when there are no errors on the platform
Nikhil-Nandagopal commented 3 years ago
- [x] Relationships disappear even when the entities are selected
- [x] The relationships should show at the top level with only the entity names not the bindings
- [x] The relationships should show in the API / Query pane
- [x] There needs to be a message in the empty state for the user to select an entity
- [x] The text inside needs to be selectable and copyable
- [x] Only errors related to user configuration should have a debug icon in the toast
- [x] When there is an error in the widget data, print the evaluated value in the logs
- © Githubissues.
- Githubissues is a development platform for aggregating issues.

appsmithorg / appsmith