MISP / misp-warninglists

Warning lists to inform users of MISP about potential false-positives or other information in indicators
http://misp.github.io/misp-warninglists/
520 stars 170 forks source link

Make disposable-email list reference-able from Earthly builds #258

Closed idelvall closed 10 months ago

idelvall commented 10 months ago

Having this Earthfile (see https://github.com/earthly/earthly) in your folder would help all Earthly users referencing this list in their build scripts (benefiting from the Earthly build cache).

This is how a referencing Earthfile would look like:

VERSION 0.7
FROM alpine:3.18.4

get-disposable-emails:
  COPY github.com/MISP/misp-warninglists/lists/disposable-email+export/list.json ./
  RUN cat list.json | head -n 10

And an example of its execution:

$ earthly +get-disposable-emails
 Init 🚀
————————————————————————————————————————————————————————————————————————————————

           buildkitd | Found buildkit daemon as docker container (earthly-buildkitd)

Streaming logs to https://cloud.earthly.dev/builds/a9cb7865-d4dd-4123-9715-9c8886e79480

 Build 🔧
————————————————————————————————————————————————————————————————————————————————

       alpine:3.18.4 | --> Load metadata alpine:3.18.4 linux/amd64
g/e/m/l/disposable-email+base | --> FROM alpine:3.18.4
g/e/m/l/disposable-email+base | [----------] 100% FROM alpine:3.18.4
g/e/m/l/disposable-email+export | *cached* --> COPY list.json ./
+get-disposable-emails | *cached* --> COPY github.com/earthly/misp-warninglists/lists/disposable-email+export/list.json ./
+get-disposable-emails | --> RUN cat list.json | head -n 10
+get-disposable-emails | {
+get-disposable-emails |   "description": "List of disposable email domains",
+get-disposable-emails |   "list": [
+get-disposable-emails |     "0-mail.com",
+get-disposable-emails |     "027168.com",
+get-disposable-emails |     "0815.ru",
+get-disposable-emails |     "0815.ry",
+get-disposable-emails |     "0815.su",
+get-disposable-emails |     "0845.ru",
+get-disposable-emails |     "0box.eu",
              output | --> exporting outputs

 Push Summary ⏫ (disabled)
————————————————————————————————————————————————————————————————————————————————

To enable pushing use earthly --push

 Local Output Summary 🎁
————————————————————————————————————————————————————————————————————————————————

========================== 🌍 Earthly Build  ✅ SUCCESS ==========================

View logs at https://cloud.earthly.dev/builds/a9cb7865-d4dd-4123-9715-9c8886e79480
adulau commented 10 months ago

Thank you for the pull request. I'm not entirely sure about the advantage of having a kind-of Makefile in a specific directory for a single list of MISP warning lists.

If there's a specific need for deployment scripts, I would suggest placing an Earth file at the top of the project directory. The directory structure for warning lists expects a single JSON file in the specification.

idelvall commented 10 months ago

Hi Alexandre, the goal is to make our builds explicit about which are the sources of truth involved in it, and the same time get your updates in a timely and performant way. I focused on the solving our immediate needs in the less intrusive way, that is why I placed the Earthfile there, but placing at the top level have more sense now that you mentioned it.
Let me work on that

adulau commented 10 months ago

Thank you for the updates and contribution.

idelvall commented 10 months ago

Thanks to you for maintaining these lists. They are useful!