Cannot get back asyncquery result

[oOhyeahh]

Bug description

I try to use global async query feature flag but I keep getting this error on the frontend.

Cannot read properties of undefined (reading 'forEach')

Here's part of my set up in superset config.py

FEATURE_FLAGS = {
    "GLOBAL_ASYNC_QUERIES": True,
}

# ----------------------------------------------------
# CACHE CONFIG
# ----------------------------------------------------
twelve_hour_seconds = 60 * 60 * 12
REDIS_HOST = os.environ.get("REDIS_HOST")
REDIS_PORT = 6379
CACHE_CONFIG = {
    "CACHE_TYPE": "RedisCache",
    "CACHE_DEFAULT_TIMEOUT": twelve_hour_seconds,
    "CACHE_KEY_PREFIX": "s_results_",
    "CACHE_REDIS_URL": f"redis://{REDIS_HOST}:6379/0",
}
# Cache for datasource metadata and query results
DATA_CACHE_CONFIG = {
    "CACHE_TYPE": "RedisCache",
    "CACHE_DEFAULT_TIMEOUT": twelve_hour_seconds,
    "CACHE_KEY_PREFIX": "s_data_",
    "CACHE_REDIS_URL": f"redis://{REDIS_HOST}:6379/1",
}

GLOBAL_ASYNC_QUERIES_JWT_SECRET = os.environ.get("GLOBAL_ASYNC_QUERIES_JWT_SECRET")

GLOBAL_ASYNC_QUERIES_REDIS_CONFIG = {
    "port": REDIS_PORT,
    "host": REDIS_HOST,
    "db": 0,
    "ssl": False,
}

How to reproduce the bug

Use the same config I post and go to any chart/dashboard. You should see the error

Screenshots/recordings

No response

Superset version

3.1.3

Python version

3.11

Node version

Not applicable

Browser

Chrome

Additional context

No response

Checklist

• [X] I have searched Superset docs and Slack and didn't find a solution to my problem.
• [X] I have searched the GitHub issue tracker and didn't find a similar bug report.
• [X] I have checked Superset's logs for errors and if I found a relevant Python stacktrace, I included it here as text in the "additional context" section.

[dosubot[bot]]

@oOhyeahh Hello there! I'm here to help you with your Superset issue. I can assist with bug troubleshooting, answer questions, and guide you on becoming a contributor. Let's work together to resolve your concerns.

To resolve the "Cannot read properties of undefined (reading 'forEach')" error on the frontend when enabling the global async query feature flag in Superset, ensure that the events parameter passed to the processEvents function is defined and is an array.

The error occurs because events is undefined or not an array, and the forEach method cannot be called on it. You can add a check to ensure events is defined and is an array before calling forEach.

Here is the modified processEvents function with the necessary check:

export const processEvents = async (events: AsyncEvent[]) => {
  if (!Array.isArray(events)) {
    logging.warn('Expected events to be an array, but got:', events);
    return;
  }

  events.forEach((asyncEvent: AsyncEvent) => {
    const jobId = asyncEvent.job_id;
    const listener = listenersByJobId[jobId];
    if (listener) {
      listener(asyncEvent);
      delete retriesByJobId[jobId];
    } else {
      // handle race condition where event is received
      // before listener is registered
      if (!retriesByJobId[jobId]) retriesByJobId[jobId] = 0;
      retriesByJobId[jobId] += 1;

      if (retriesByJobId[jobId] <= MAX_RETRIES) {
        setTimeout(() => {
          processEvents([asyncEvent]);
        }, RETRY_DELAY * retriesByJobId[jobId]);
      } else {
        delete retriesByJobId[jobId];
        logging.warn('listener not found for job_id', asyncEvent.job_id);
      }
    }
    setLastId(asyncEvent);
  });
};

Additionally, you can refer to the updateComponentParentsList function in superset-frontend/src/dashboard/util/updateComponentParentsList.test.js for handling invalid parameters gracefully. This function includes checks to ensure that parameters are valid and does not throw errors when they are not, which can be a useful pattern to follow:

describe('updateComponentParentsList with bad inputs', () => {
  it('should handle invalid parameters and not throw error', () => {
    updateComponentParentsList({
      currentComponent: undefined,
      layout: undefined,
    });

    expect(() =>
      updateComponentParentsList({
        currentComponent: undefined,
        layout: undefined,
      }),
    ).not.toThrow();

    expect(() =>
      updateComponentParentsList({
        currentComponent: {},
        layout: undefined,
      }),
    ).not.toThrow();

    expect(() =>
      updateComponentParentsList({
        currentComponent: {},
        layout: null,
      }),
    ).not.toThrow();

    expect(() =>
      updateComponentParentsList({
        currentComponent: { id: 'id3', children: ['id1', 'id2'] },
        layout: { id3: {} },
      }),
    ).not.toThrow();

    expect(() =>
      updateComponentParentsList({
        currentComponent: { id: 'id3' },
        layout: { id3: {} },
      }),
    ).not.toThrow();

    expect(() =>
      updateComponentParentsList({
        currentComponent: { id: 'id3' },
        layout: {},
      }),
    ).not.toThrow();
  });
});

This approach ensures that your code handles unexpected inputs gracefully and avoids runtime errors ^[1][2].

To continue talking to Dosu, mention @dosu.

[oOhyeahh]

@dosu Unfortunately it doesn't solve my problem

[dosubot[bot]]

To further address the issue, you can use the ensureIsArray utility function from @superset-ui/core to make sure that events is always an array. This function handles various input types and ensures that the input is always converted to an array.

Here is the updated processEvents function using ensureIsArray:

import { ensureIsArray } from '@superset-ui/core';

export const processEvents = async (events: AsyncEvent[]) => {
  events = ensureIsArray(events); // Ensure events is always an array

  events.forEach((asyncEvent: AsyncEvent) => {
    const jobId = asyncEvent.job_id;
    const listener = listenersByJobId[jobId];
    if (listener) {
      listener(asyncEvent);
      delete retriesByJobId[jobId];
    } else {
      if (!retriesByJobId[jobId]) retriesByJobId[jobId] = 0;
      retriesByJobId[jobId] += 1;

      if (retriesByJobId[jobId] <= MAX_RETRIES) {
        setTimeout(() => {
          processEvents([asyncEvent]);
        }, RETRY_DELAY * retriesByJobId[jobId]);
      } else {
        delete retriesByJobId[jobId];
        logging.warn('listener not found for job_id', asyncEvent.job_id);
      }
    }
    setLastId(asyncEvent);
  });
};

This should help resolve the issue by ensuring that events is always an array, thus preventing the forEach method from encountering undefined or non-array inputs ^[1].

To continue talking to Dosu, mention @dosu.