search

apache / superset

Apache Superset is a Data Visualization and Data Exploration Platform
https://superset.apache.org/
Apache License 2.0
61.87k stars 13.55k forks source link

Event flow chart does not display Time section in configuration #26181

Closed JamesGlover closed 3 months ago

JamesGlover commented 9 months ago

When creating an 'event flow' chart in superset, the section labeled 'Time' is not displayed. This prevents the 'time column' select element from rendering, and prevents configuration of the chart. Attempts to configure a chart without filling in this section result in the error message:

Unexpected error Datetime column not provided as part table configuration and is required by this type of chart

Edit: After a bit of investigation, it appears that the time configuration section is rendered by the shared legacyRegularTime section, which is disabled when GENERIC_CHART_AXES is set. This suggests that this is therefor intended behaviour, and the bug instead lies with the propagation of the generic configuration through to the get_sqla_query function.

Charts created in an earlier version of superset (2.0.0) can be imported, and will be rendered successfully, however they result in the same error as above if edited.

How to reproduce the bug

  1. Spin up and configure a superset docker container (I'm using 3.0.2)
  2. Import the examples with docker exec -it superset superset load_examples
  3. Log in to superset
  4. Click on 'Charts'
  5. Click on '+ Chart'
  6. Click on 'Flow' > 'Event Flow'
  7. Select the 'Channels' data set
  8. Click 'Create new chart'
  9. The 'Time' section will not be visible
  10. If you attempt to configure the remainder of the chart, selecting topic__creator for 'Entity id', and name for 'Event names' then click 'Create chart' you will see:

Unexpected error Datetime column not provided as part table configuration and is required by this type of chart

The same set of actions will work for Superset 2.0.0

Expected results

~I expect to have the ability to select created under a 'Time Column' select box, which will allow an event flow chart to be created. This chart will show the flow of channel creation per topic__creator.~

I expect to be able to create an event flow chart. In the absence of an ability to explicitly configure time column settings, then the generic configuration should behave as intended.

Actual results

The time section of the configuration form is not rendered, preventing the selection of a Time column. This results in an error message when an attempt is made to create the chart.

Screenshots

Attached are screenshots of the process in superset 3.0.2, and superset 2.0.0

Superset 2.0.0

Screenshot 2023-12-05 at 08 53 02

Superset 3.0.2

Screenshot 2023-12-05 at 14 38 09

Environment

I'm using the container apache/supereset:3.0.2

Checklist

Make sure to follow these steps before submitting your issue - thank you!

See some issues within the docker logs:

2023-12-05 14:41:46 192.168.65.1 - - [05/Dec/2023:14:41:46 +0000] "GET /api/v1/me/ HTTP/1.1" 200 150 "http://localhost:8082/explore/?form_data_key=DQHTkqDFXtffWzPKYWWOvObk4We9PsomKjRATiDGy3sphErNjopkXo5Q-B40XmeH&viz_type=event_flow&datasource=20__table&datasource_id=20&datasource_type=table" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/118.0.0.0 Safari/537.36"
2023-12-05 14:41:53 192.168.65.1 - - [05/Dec/2023:14:41:53 +0000] "GET /explore/?form_data_key=DQHTkqDFXtffWzPKYWWOvObk4We9PsomKjRATiDGy3sphErNjopkXo5Q-B40XmeH&viz_type=event_flow&datasource=20__table&datasource_id=20&datasource_type=table HTTP/1.1" 200 49991 "http://localhost:8082/superset/welcome/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/118.0.0.0 Safari/537.36"
2023-12-05 14:41:53 HTTPException
2023-12-05 14:41:53 Traceback (most recent call last):
2023-12-05 14:41:53   File "/usr/local/lib/python3.9/site-packages/flask/app.py", line 1823, in full_dispatch_request
2023-12-05 14:41:53     rv = self.dispatch_request()
2023-12-05 14:41:53   File "/usr/local/lib/python3.9/site-packages/flask/app.py", line 1799, in dispatch_request
2023-12-05 14:41:53     return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args)
2023-12-05 14:41:53   File "/usr/local/lib/python3.9/site-packages/flask/scaffold.py", line 331, in send_static_file
2023-12-05 14:41:53     return send_from_directory(
2023-12-05 14:41:53   File "/usr/local/lib/python3.9/site-packages/flask/helpers.py", line 590, in send_from_directory
2023-12-05 14:41:53     return werkzeug.utils.send_from_directory(  # type: ignore[return-value]
2023-12-05 14:41:53   File "/usr/local/lib/python3.9/site-packages/werkzeug/utils.py", line 578, in send_from_directory
2023-12-05 14:41:53     raise NotFound()
2023-12-05 14:41:53 werkzeug.exceptions.NotFound: 404 Not Found: The requested URL was not found on the server. If you entered the URL manually please check your spelling and try again.
2023-12-05 14:41:53 2023-12-05 14:41:53,194:WARNING:superset.views.base:HTTPException
2023-12-05 14:41:53 Traceback (most recent call last):
2023-12-05 14:41:53   File "/usr/local/lib/python3.9/site-packages/flask/app.py", line 1823, in full_dispatch_request
2023-12-05 14:41:53     rv = self.dispatch_request()
2023-12-05 14:41:53   File "/usr/local/lib/python3.9/site-packages/flask/app.py", line 1799, in dispatch_request
2023-12-05 14:41:53     return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args)
2023-12-05 14:41:53   File "/usr/local/lib/python3.9/site-packages/flask/scaffold.py", line 331, in send_static_file
2023-12-05 14:41:53     return send_from_directory(
2023-12-05 14:41:53   File "/usr/local/lib/python3.9/site-packages/flask/helpers.py", line 590, in send_from_directory
2023-12-05 14:41:53     return werkzeug.utils.send_from_directory(  # type: ignore[return-value]
2023-12-05 14:41:53   File "/usr/local/lib/python3.9/site-packages/werkzeug/utils.py", line 578, in send_from_directory
2023-12-05 14:41:53     raise NotFound()
2023-12-05 14:41:53 werkzeug.exceptions.NotFound: 404 Not Found: The requested URL was not found on the server. If you entered the URL manually please check your spelling and try again.
2023-12-05 14:41:53 HTTPException
2023-12-05 14:41:53 Traceback (most recent call last):
2023-12-05 14:41:53   File "/usr/local/lib/python3.9/site-packages/flask/app.py", line 1823, in full_dispatch_request
2023-12-05 14:41:53     rv = self.dispatch_request()
2023-12-05 14:41:53   File "/usr/local/lib/python3.9/site-packages/flask/app.py", line 1799, in dispatch_request
2023-12-05 14:41:53     return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args)
2023-12-05 14:41:53   File "/usr/local/lib/python3.9/site-packages/flask/scaffold.py", line 331, in send_static_file
2023-12-05 14:41:53     return send_from_directory(
2023-12-05 14:41:53   File "/usr/local/lib/python3.9/site-packages/flask/helpers.py", line 590, in send_from_directory
2023-12-05 14:41:53     return werkzeug.utils.send_from_directory(  # type: ignore[return-value]
2023-12-05 14:41:53   File "/usr/local/lib/python3.9/site-packages/werkzeug/utils.py", line 578, in send_from_directory
2023-12-05 14:41:53     raise NotFound()
2023-12-05 14:41:53 werkzeug.exceptions.NotFound: 404 Not Found: The requested URL was not found on the server. If you entered the URL manually please check your spelling and try again.
2023-12-05 14:41:53 2023-12-05 14:41:53,213:WARNING:superset.views.base:HTTPException
2023-12-05 14:41:53 Traceback (most recent call last):
2023-12-05 14:41:53   File "/usr/local/lib/python3.9/site-packages/flask/app.py", line 1823, in full_dispatch_request
2023-12-05 14:41:53     rv = self.dispatch_request()
2023-12-05 14:41:53   File "/usr/local/lib/python3.9/site-packages/flask/app.py", line 1799, in dispatch_request
2023-12-05 14:41:53     return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args)
2023-12-05 14:41:53   File "/usr/local/lib/python3.9/site-packages/flask/scaffold.py", line 331, in send_static_file
2023-12-05 14:41:53     return send_from_directory(
2023-12-05 14:41:53   File "/usr/local/lib/python3.9/site-packages/flask/helpers.py", line 590, in send_from_directory
2023-12-05 14:41:53     return werkzeug.utils.send_from_directory(  # type: ignore[return-value]
2023-12-05 14:41:53   File "/usr/local/lib/python3.9/site-packages/werkzeug/utils.py", line 578, in send_from_directory
2023-12-05 14:41:53     raise NotFound()
2023-12-05 14:41:53 werkzeug.exceptions.NotFound: 404 Not Found: The requested URL was not found on the server. If you entered the URL manually please check your spelling and try again.
2023-12-05 14:41:53 192.168.65.1 - - [05/Dec/2023:14:41:53 +0000] "GET /static/appbuilder/select2/select2.css HTTP/1.1" 404 0 "http://localhost:8082/explore/?form_data_key=DQHTkqDFXtffWzPKYWWOvObk4We9PsomKjRATiDGy3sphErNjopkXo5Q-B40XmeH&viz_type=event_flow&datasource=20__table&datasource_id=20&datasource_type=table" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/118.0.0.0 Safari/537.36"
2023-12-05 14:41:53 HTTPException
2023-12-05 14:41:53 Traceback (most recent call last):
2023-12-05 14:41:53   File "/usr/local/lib/python3.9/site-packages/flask/app.py", line 1823, in full_dispatch_request
2023-12-05 14:41:53     rv = self.dispatch_request()
2023-12-05 14:41:53   File "/usr/local/lib/python3.9/site-packages/flask/app.py", line 1799, in dispatch_request
2023-12-05 14:41:53     return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args)
2023-12-05 14:41:53   File "/usr/local/lib/python3.9/site-packages/flask/scaffold.py", line 331, in send_static_file
2023-12-05 14:41:53     return send_from_directory(
2023-12-05 14:41:53   File "/usr/local/lib/python3.9/site-packages/flask/helpers.py", line 590, in send_from_directory
2023-12-05 14:41:53     return werkzeug.utils.send_from_directory(  # type: ignore[return-value]
2023-12-05 14:41:53   File "/usr/local/lib/python3.9/site-packages/werkzeug/utils.py", line 578, in send_from_directory
2023-12-05 14:41:53     raise NotFound()
2023-12-05 14:41:53 werkzeug.exceptions.NotFound: 404 Not Found: The requested URL was not found on the server. If you entered the URL manually please check your spelling and try again.
2023-12-05 14:41:53 2023-12-05 14:41:53,219:WARNING:superset.views.base:HTTPException
2023-12-05 14:41:53 Traceback (most recent call last):
2023-12-05 14:41:53   File "/usr/local/lib/python3.9/site-packages/flask/app.py", line 1823, in full_dispatch_request
2023-12-05 14:41:53     rv = self.dispatch_request()
2023-12-05 14:41:53   File "/usr/local/lib/python3.9/site-packages/flask/app.py", line 1799, in dispatch_request
2023-12-05 14:41:53     return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args)
2023-12-05 14:41:53   File "/usr/local/lib/python3.9/site-packages/flask/scaffold.py", line 331, in send_static_file
2023-12-05 14:41:53     return send_from_directory(
2023-12-05 14:41:53   File "/usr/local/lib/python3.9/site-packages/flask/helpers.py", line 590, in send_from_directory
2023-12-05 14:41:53     return werkzeug.utils.send_from_directory(  # type: ignore[return-value]
2023-12-05 14:41:53   File "/usr/local/lib/python3.9/site-packages/werkzeug/utils.py", line 578, in send_from_directory
2023-12-05 14:41:53     raise NotFound()
2023-12-05 14:41:53 werkzeug.exceptions.NotFound: 404 Not Found: The requested URL was not found on the server. If you entered the URL manually please check your spelling and try again.
2023-12-05 14:41:53 192.168.65.1 - - [05/Dec/2023:14:41:53 +0000] "GET /static/appbuilder/datepicker/bootstrap-datepicker.css HTTP/1.1" 404 0 "http://localhost:8082/explore/?form_data_key=DQHTkqDFXtffWzPKYWWOvObk4We9PsomKjRATiDGy3sphErNjopkXo5Q-B40XmeH&viz_type=event_flow&datasource=20__table&datasource_id=20&datasource_type=table" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/118.0.0.0 Safari/537.36"
2023-12-05 14:41:53 192.168.65.1 - - [05/Dec/2023:14:41:53 +0000] "GET /static/appbuilder/select2/select2-bootstrap-theme.css HTTP/1.1" 404 0 "http://localhost:8082/explore/?form_data_key=DQHTkqDFXtffWzPKYWWOvObk4We9PsomKjRATiDGy3sphErNjopkXo5Q-B40XmeH&viz_type=event_flow&datasource=20__table&datasource_id=20&datasource_type=table" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/118.0.0.0 Safari/537.36"
2023-12-05 14:41:54 192.168.65.1 - - [05/Dec/2023:14:41:54 +0000] "GET /api/v1/database/?q=(filters:!((col:database_name,opr:neq,value:examples))) HTTP/1.1" 200 521 "http://localhost:8082/explore/?form_data_key=DQHTkqDFXtffWzPKYWWOvObk4We9PsomKjRATiDGy3sphErNjopkXo5Q-B40XmeH&viz_type=event_flow&datasource=20__table&datasource_id=20&datasource_type=table" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/118.0.0.0 Safari/537.36"
2023-12-05 14:41:54 192.168.65.1 - - [05/Dec/2023:14:41:54 +0000] "GET /api/v1/database/?q=(filters:!((col:allow_file_upload,opr:upload_is_enabled,value:!t))) HTTP/1.1" 200 521 "http://localhost:8082/explore/?form_data_key=DQHTkqDFXtffWzPKYWWOvObk4We9PsomKjRATiDGy3sphErNjopkXo5Q-B40XmeH&viz_type=event_flow&datasource=20__table&datasource_id=20&datasource_type=table" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/118.0.0.0 Safari/537.36"
2023-12-05 14:41:54 192.168.65.1 - - [05/Dec/2023:14:41:54 +0000] "GET /api/v1/explore/?form_data_key=DQHTkqDFXtffWzPKYWWOvObk4We9PsomKjRATiDGy3sphErNjopkXo5Q-B40XmeH&viz_type=event_flow&datasource_id=20&datasource_type=table HTTP/1.1" 200 2389 "http://localhost:8082/explore/?form_data_key=DQHTkqDFXtffWzPKYWWOvObk4We9PsomKjRATiDGy3sphErNjopkXo5Q-B40XmeH&viz_type=event_flow&datasource=20__table&datasource_id=20&datasource_type=table" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/118.0.0.0 Safari/537.36"
2023-12-05 14:41:54 192.168.65.1 - - [05/Dec/2023:14:41:54 +0000] "GET /api/v1/database/1/table_extra/channels/main/ HTTP/1.1" 200 3 "http://localhost:8082/explore/?form_data_key=DQHTkqDFXtffWzPKYWWOvObk4We9PsomKjRATiDGy3sphErNjopkXo5Q-B40XmeH&viz_type=event_flow&datasource=20__table&datasource_id=20&datasource_type=table" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/118.0.0.0 Safari/537.36"
2023-12-05 14:41:54 Superset.explore_json This API endpoint is deprecated and will be removed in version 4.0.0
2023-12-05 14:41:54 2023-12-05 14:41:54,705:WARNING:superset.views.base:Superset.explore_json This API endpoint is deprecated and will be removed in version 4.0.0
2023-12-05 14:41:54 2023-12-05 14:41:54,721:DEBUG:parsedatetime:eval - with context - False, False
2023-12-05 14:41:54 Cache key: db81d61dbf5d30dce5e2f4f1609d33fd
2023-12-05 14:41:54 2023-12-05 14:41:54,743:INFO:superset.viz:Cache key: db81d61dbf5d30dce5e2f4f1609d33fd
2023-12-05 14:41:54 192.168.65.1 - - [05/Dec/2023:14:41:54 +0000] "POST /superset/explore_json/ HTTP/1.1" 400 1175 "http://localhost:8082/explore/?form_data_key=DQHTkqDFXtffWzPKYWWOvObk4We9PsomKjRATiDGy3sphErNjopkXo5Q-B40XmeH&viz_type=event_flow&datasource=20__table&datasource_id=20&datasource_type=table" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/118.0.0.0 Safari/537.36"
2023-12-05 14:41:54 192.168.65.1 - - [05/Dec/2023:14:41:54 +0000] "GET /api/v1/chart/0 HTTP/1.1" 404 24 "http://localhost:8082/explore/?form_data_key=DQHTkqDFXtffWzPKYWWOvObk4We9PsomKjRATiDGy3sphErNjopkXo5Q-B40XmeH&viz_type=event_flow&datasource=20__table&datasource_id=20&datasource_type=table" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/118.0.0.0 Safari/537.36"
2023-12-05 14:41:55 192.168.65.1 - - [05/Dec/2023:14:41:55 +0000] "POST /api/v1/explore/form_data?tab_id=1 HTTP/1.1" 201 75 "http://localhost:8082/explore/?form_data_key=DQHTkqDFXtffWzPKYWWOvObk4We9PsomKjRATiDGy3sphErNjopkXo5Q-B40XmeH&viz_type=event_flow&datasource=20__table&datasource_id=20&datasource_type=table" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/118.0.0.0 Safari/537.36"
2023-12-05 14:41:55 192.168.65.1 - - [05/Dec/2023:14:41:55 +0000] "POST /superset/log/?explode=events HTTP/1.1" 200 1 "http://localhost:8082/explore/?form_data_key=DQHTkqDFXtffWzPKYWWOvObk4We9PsomKjRATiDGy3sphErNjopkXo5Q-B40XmeH&viz_type=event_flow&datasource=20__table&datasource_id=20&datasource_type=table" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/118.0.0.0 Safari/537.36"

Additional context

Add any other context about the problem here.

gauravruhela07 commented 9 months ago

Hey @JamesGlover I want to work on this issue. can you please assign it to me? Thank you!

JamesGlover commented 9 months ago

@gauravruhela07 I'm not a maintainer, just an end-user reporting the issue, so am not able to assign issues.

gauravruhela07 commented 9 months ago

@JamesGlover can you please tell me whom should I tag/ask to assign it to me?

unnyns-307 commented 9 months ago

Hello @rusackas, I'm sorry to bother you again. Could you kindly suggest us on assigning @gauravruhela07 as assignee to this issue?

JamesGlover commented 9 months ago

Done a bit of probing into this one.

It appears that this chart uses the legacyRegularTime section, which gets disabled when the feature flag GENERIC_CHART_AXES is enabled.

Which suggests that the missing time section is intentional, and actually the problem here is that the generic axes options aren't being passed through to get_sqla_query where the error message we see is actually generated. I'll update the report to reflect this.

JamesGlover commented 9 months ago

Following a bit more analysis: EventFlowViz.query_object has a granularity of None when not present in the form object, which triggers the error message when handles by get_sqla_query. Adding:

if query["granularity"] is None:
          query["granularity"] = ''

To this will resolve the issue, resulting in a fallback to the default timestamp. However I'm not familiar enough with the architecture of Superset to know what the correct fix actually is. I notice that a lot of this is going through legacy endpoints, so I'm assuming that the general direction is to update or replace these visualisations with versions using newer interfaces.

JamesGlover commented 9 months ago

This PR fixes a similar issue on another chart by restoring the time section of the form in an ad-hoc manner:

https://github.com/apache/superset/pull/23865

cdr4321 commented 4 months ago

anybody knows where this issue stands? I am unable to use the event flow chart at all

rusackas commented 4 months ago

TBH, I'm going to propose deprecating the event flow chart as part of Superset 5.0. If anyone wants to make a modern replacement, that's great, but it seems like it's not used by many, and it's built with dependencies that are no longer maintained.

mdeshmu commented 3 months ago

@villebro @michael-s-molina @Vitor-Avila Seems like 26705 didn't solve this issue. We are still facing the same issue in 3.1.2. Any work around would be much appreciated.

michael-s-molina commented 3 months ago

@mdeshmu There's a proposal in 5.0 to remove this chart as it's not compatible with React 17. Check here for more details.

Vitor-Avila commented 3 months ago

thanks, @michael-s-molina! That's really good context 🙏 Seems like solving this specific bug doesn't make sense then, and instead a better option would be to recreate/upgrade this plugin to meet current needs/standards (leaving this comment here just as a potential suggestion for next steps if someone wants to tackle it)

michael-s-molina commented 3 months ago

Closing as won't fix.

mdeshmu commented 3 months ago

@michael-s-molina @rusackas @Vitor-Avila We do need an alternate chart for event flow. Any suggestions?

michael-s-molina commented 3 months ago

@mdeshmu I don't think ECharts has anything similar. Maybe one could be built using a combination of ECharts charts or another library. Currently, we don't have anyone willing to create/maintain this chart as it's not frequently used.

rusackas commented 3 months ago

I think it's even fair to say that the existing chart could be refactored... if it's updated to use the newer chart/data endpoint, converted to typescript, and stripped of the problematic dependencies (data-ui, if memory serves) then we can probably just keep it! Of course, I have NO idea how deep that rabbit hole goes...