dolthub / hosted-issues

Issues for hosted.doltdb.com
https://hosted.doltdb.com
2 stars 0 forks source link

Hosted Dolt: CSV export issues #103

Closed fulghum closed 1 year ago

fulghum commented 1 year ago

While testing Hosted Dolt's new CSV export feature, a customer ran into two problems:

  1. Timestamps do not roundtrip – the export format for timestamp and datetime types (e.g. Tue Jun 13 2023 21:37:02 GMT+0000 (Coordinated Universal Time)) does not round trip back into Dolt or MySQL.
  2. Unique filenames – currently, the filename of the export is just the database name. Making the filenames more unique would help customers who are frequently using the export feature so they can easily tell info about what source generated the export. The customer mentioned that including the short SHA of the current branch head, a timestamp, and the first X chars of the query would be the ideal format for them.
  3. Request timeouts for large result sets – the customer was unable to export a result set of ~300k rows. I know our first version of CSV export has limitations on how large of a result set we can handle. In addition to working towards larger result set sizes, it would be nice to have a better error message when we can't process a result set (current error UI included below). Ideally, the error message would 1) explain to customers that we can only support a limited result set size currently, 2) mention any workarounds like using LIMIT, and potentially also 3) encouraging users to ping a GitHub issue or reach out to us if they need larger result set size support. That last point is probably not necessary if we're already planning to prioritize larger result set sizes soon, but if we aren't, it could provide a good signal from affected customers.
hosted-dolt-csv-export-timeout
tbantle22 commented 1 year ago

Addressed these issues:

  1. Fixed
  2. Users can now specify their own file name. It will default to [database]_[ref]_[date].csv
  3. I added more info to timeout errors that we currently only support a limited result set size and using limit might help. There's another issue here for workbench queries timing out too quickly which I will address next and should help with supporting larger result sets