When I export data, it only gives me a limited number of rows

[geroter]

Expected result When I press export from Reps Events, it either gives the option for selecting how much data (date range?), or just gives me all of the data.

Actual result I press export and get a CSV with about 35 rows.

I'm specifically looking at this for Reps Events, but I've noticed this elsewhere.

[jgbarah]

This is because there are limits to the information shown in tables, for performance reasons. However, the current limit (30 in most cases) is probably too conservative. We can enlarge it. I'm opining an internal thread at Bitergia to do that.

[acs]

@geroter do you want a quick fix now for Reps Events until we deploy the general approach we are discussing right now?

[geroter]

No quick fix needed.

For personal purposes, being able to choose the number of records to export (better even is date-range) would be ideal, especially if we are doing more detailed analysis of qualitative data about the events (which right now can't be done in Kibana).

[jgbarah]

That could be done likely more easily with direct queries to ElasticSearch. In fact, you can get the base query from the table (in the editing dashboard you can see the query done to produce the table), maybe complement it, and query ElasticSearch directly.

In any case, for date-ranges, you can use the time filter (top right on the dashboard), and filler by the date-range you want. The whole dashboard will get filtered, including the table. Anything you get from the table later will as well be filtered by that time range.

[hmitsch]

@jgbarah and @acs do you have an update concerning the internal thread mentioned above in https://github.com/mozilla/participation-metrics-org/issues/72#issuecomment-270647032?

[hmitsch]

Bitergia concluded that they keep the current configuration. For an undefined point in the future, there might be the possibility to extract data from the database.

[jgbarah]

I'm reopening because I had not noticed that we could use a solution that is working with other customers. It consists in offering special panels with "long" tables. In this case, for example, with all (or a large number) of events, so that those could be exported to CSV. Since they are in special panels, even if they are slow the don't harm the "usual" panels. By using sticky filters and date selectors, the selection of rows in the table can be tailored.

@hmitsch do you think this could be a solution?

[hmitsch]

Let's create one new panel (aka Kibana dashboard) with only one widget (aka Kibana panel) which allows to download all Reps Events into a CVS file. Please also link it in the top menu.

[canasdiaz]

Hi @hmitsch and @geroter. About the last events, does this fit your needs? I've moved the "Last Events" table to a new panel and increased the limit to 10K docs.

https://analytics.mozilla.community/edit/app/kibana#/dashboard/Reps-All-Past-Events

[geroter]

yes this is excellent for my purposes

if I gave a list of other "fields" I'd like to export, would it be rather easy to include them in this panel/table?

[canasdiaz]

@geroter Give us the list and let's see what we can do ;)

[canasdiaz]

Guys, I have this assigned by I don't know what it is needed to be done so I'm closing it. If you miss something that should be done please 1) reopen it and 2) specify what is missing