search

tierpod / dmarc-report-converter

Convert dmarc reports from xml to human-readable formats
MIT License
242 stars 25 forks source link

Improve UX of browsing through reports #43

Open eloydegen opened 8 months ago

eloydegen commented 8 months ago

Hi, thanks for the cool tool! I was looking for something like this, most other projects use way more elaborate dependencies I do not need. It fits my needs quite good :D

Currently it is pretty tedious to browse through all the reports that are put in a subfolder. I do like the idea of using the webserver built-in feature of open directories, but the UX isn't great at the moment. Let me know your thoughts on this!

Possible options:

tierpod commented 7 months ago

Thank you for the report.

I would like to improve UX, but on the other hand - keep this project as simple as possible (for my use case, current situation suits very well).

Merging all the pages to a single page.

For now, we use this key for grouping

https://github.com/tierpod/dmarc-report-converter/blob/b4ccc490fe4e3877663758ed018a798dfb8b52eb/cmd/dmarc-report-converter/merge.go#L23

Maybe we can make group key configurable.

Rename the actual report file to 'index.html' so clicking on a sub-folder makes you enter the report directly. If there are multiple for a single day, they could be merged (maybe as a setting for people who do not want this)

In this case, all reports from different email providers will be merged into one big html file. With current merging logic, we'll lost information about who send us this report (I think it's important). We can implement another logic like "generate all reports not in individual html file each, but in one index.html keeping provider information".

Not merge pages, but adding 'next' buttons to report pages so it is easier to browse

I assume, it's possible by adding new dependency (like database), or writing html template with some javascript: read list of files, and render "next" button, but not sure (I'm not good in JS).

As you can see, there are ways to improve situation, just need some time to experiment and choose more suitable way.

eloydegen commented 7 months ago

I assume, it's possible by adding new dependency (like database), or writing html template with some javascript: read list of files, and render "next" button, but not sure (I'm not good in JS).

At the time of rendering the whole thing, it is known in-memory what all the pages are, right? Then it would be possible to add an HTML button on each page, without any need for a database or Javascript.

moorereason commented 7 months ago

Merging all the pages to a single page.

For now, we use this key for grouping

https://github.com/tierpod/dmarc-report-converter/blob/b4ccc490fe4e3877663758ed018a798dfb8b52eb/cmd/dmarc-report-converter/merge.go#L23

Maybe we can make group key configurable.

I added this feature locally to see what it could do and then got distracted with something else and haven't really tested it out much. Here's the idea:

  1. Create a new merge_key config option.
  2. The merge_key is a Go template string.
  3. The default value is {{ .ReportMetadata.OrgName }}!{{ .ReportMetadata.Email }}!{{ .PolicyPublished.Domain }}.
  4. Render the merge_key for each report and use it to test for equality.

I can submit a PR if you're interested.

Not merge pages, but adding 'next' buttons to report pages so it is easier to browse

I assume, it's possible by adding new dependency (like database), or writing html template with some javascript: read list of files, and render "next" button, but not sure (I'm not good in JS).

As you can see, there are ways to improve situation, just need some time to experiment and choose more suitable way.

One of the things I got distracted with was using dmarc-report-converter to generate data files (JSON) and content files (Markdown with just frontmatter) for use with Hugo (a Go static site generator). I don't know if it's a viable solution or not, but I want to keep playing with it. If it turns into something useful, I'll be sure to share it.

tierpod commented 7 months ago

I can submit a PR if you're interested.

Yes, of course, I'll take a look at it. I had the same idea, but busy at work to play with it.

I don't know if it's a viable solution or not

Why not? Nice idea. I added json output for such cases (integrating with external software), so feel free to experiment with it :+1: