Feature: Export Logs

[ajwalks]

Include the ability to export viewable filtered logs to a CSV file. If no filters are set export all available logs.

[muhakh]

I have a problem with the export. I made a button in the view as in this https://github.com/muhakh/gsoc16_recording-action-logs/blob/d0a38892fa8352fd511c89ea60bef965cb1b8c35/administrator/components/com_userlogs/views/userlogs/view.html.php#L69 This button executes this function in the controller https://github.com/muhakh/gsoc16_recording-action-logs/blob/d0a38892fa8352fd511c89ea60bef965cb1b8c35/administrator/components/com_userlogs/controllers/userlogs.php#L45 In it there is a helper function called which is written in https://github.com/muhakh/gsoc16_recording-action-logs/blob/d0a38892fa8352fd511c89ea60bef965cb1b8c35/administrator/components/com_userlogs/helpers/userlogs.php

The problem is with this function. I want to know what is the problem with it.

[yvesh]

Hey Mohamed, sorry to say that, but there are a lot of issues in that code ;-)

Headers should be send before anything is written out, so your output (which should be echo and not print) should come after them.

And instead of using php://temp, you can just build a CSV string (text!!!) $data = ...; and echo this after the headers are sent. This would make things much easier.

After the output you should exit!

P.S.: You can also use JFactory::getDate() and skip the 'now' (default) to get the current date P.P.S.: UTF-8 in csv files is mostly not a good idea, as for example Microsoft Excel fails with that very bad (Charsets in CSV are a real big issue with Microsoft products) P.P.S.: Codestyle should be improved

[roland-d]

You can do UTF-8 for Excel but you must add a BOM to the file, in that case Excel reads the file just fine is my experience.

[yvesh]

@roland-d interesting, in the Excel versions i have (like the Mac one), the UTF-8 BOM is printed out and completely ignored.. I had so many support tickets and issues with it in the last years, really mad at MS because of that. There are even more things to take into account, like regional versions: On a european version a semicolon (;) separated excel file works, on an US not etc.

[roland-d]

@yvesh I feel your pain, in my 10 years of doing CSVI I think I have seen all of what you describe :) Not even counting users inventing their own separators and enclosures.

It also depends on how you open the file with Excel either directly or via the Load from text option. CSV and Excel is not a happy marriage. Even till today the support for CSV by Excel lacks. OpenOffice/LibreOffice are much stronger in that area.

[chrisdavenport]

Note that the correct MIME type for CSV is "text/csv" and not "application/cvs". See: https://tools.ietf.org/html/rfc4180

[muhakh]

Thanks guys for your help, but when I add $app->close() or exit(), the $data is shown on the browser in csv format but not downloaded.

[johanjanssens]

If a browser recognises the mime-type he will try to display the file, most browser recognise text/csv as it's a text format. You could prevent that by using application/csv which tells the browser to look for a specific application to handle the csv file, if none can be found it would trigger a save as.

If you do wish to use text/csv you need to set the Content-Disposition header to 'attachment'. https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Content-Disposition

Tip: be careful with the "filename" parameter, different browsers handle this differently, especially when UTF-8 characters are used.

[johanjanssens]

Another note regarding this feature. Exporting logs - the way you implemented it - will work if the logs table has a limited amount of data, if the table has a lot of data the process will likely timeout.

To prevent that you require an export process that is driven by a client side driven finite state machine that triggers the server to export data from the database x amount of rows at a time defined and append them to a temp file. Once all data has been appended to the file the file would be send to the client.

When sending the file you also need to take care of timeouts, sending files to clients depends on bandwidth, server performance, php timeout etc.

To do this reliably you need to either implement Byte Serving or Chunked transfer encoding

Tip: Byte serving has the added benefit that the download can be re-sumed if the connection is dropped.

[yvesh]

@muhakh As Johan said this CSV file could get very big within time, but i am not sure if you manage to resolve that in the time left (this could also be added later). The same goes for your view - looks like you are missing pagination in the view?

It's really easy to implement and a must have for the component. See this (it's for 2.5, but there is not much difference in the pagination part)