domainaware / parsedmarc

A Python package and CLI for parsing aggregate and forensic DMARC reports
https://domainaware.github.io/parsedmarc/
Apache License 2.0
962 stars 210 forks source link

:Feat: allow user to write again an again their report on the same index #476

Open kawaegle opened 4 months ago

kawaegle commented 4 months ago

it can be cool to let the user specify the name of the report index and not generate a bunch of index each day of each report ?

it only take the user to set the function save_aggregate_report_to_elasticsearch() or save_forensic_report_to_elasticsearch() or save_smtp_tls_report_to_elasticsearch() with a new variable index and in the funtion just add

if index is None:
   index = "whatever is default"

and based on that, we can also simplyfied the code from:

if index_suffix is not None:
    search = Search(index="dmarc_forensic_{0}*".format(index_suffix))
else:
    search = Search(index="dmarc_forensic*")
    ///  
    /// code not related
    ///    
index = "dmarc_forensic"
if index_suffix:
    index = "{0}_{1}".format(index, index_suffix)
    if monthly_indexes:
        index_date = arrival_date.strftime("%Y-%m")
    else:
        index_date = arrival_date.strftime("%Y-%m-%d")
index = "{0}-{1}".format(index, index_date)        

to something like this:

if index is None or index == "":
    index = "dmarc_forensic"
if index_suffix:
    index = "{0}_{1}".format(index, index_suffix)
else:
    index = "{0}-{1}".format(index, index_date)
search = Search(index="{}¨*".format(index))
    ///  
    /// code not related
    ///
create_indexes([index], index_settings)

so after in kabana the user can update his discover data view to use whaterver he want and using that we preserve the user to have a bunch of index based on the date but with custom name
it change mostly nothing but let other user the choice to name their index as they want. I can provide a patch for the function save_aggregate_report_to_elasticsearch() if wanted/needed.