Log Management

[thebiblelover7]

Hey! I've been trying to get moroz to output data that promtail will be able to interpret and use for logs in Grafana, but the only way I've found so far is to make moroz's output to append to a file and then let promtail tail that file.

Has anyone found any other better way of managing logs? I'm open to other services as well

Thanks for this project!

[clreinki]

So I'm not sure if this will help you or not, but I made a small modification to the svc_upload_event.go file that enables logging to an HTTP endpoint - in my case, a Logstash agent -> Elastic. Here's the steps I did:

1. Clone the github repo

2. Ensure Golang has been downloaded and installed (link)

3. Modify ./moroz/svc_upload_event.go to add HTTP endpoint logging. Basically replace the main function at the top with this code (don't forget to update the url to reflect your endpoint!):

func (svc *SantaService) UploadEvent(ctx context.Context, machineID string, events []santa.EventPayload) error {
        if !svc.flPersistEvents {
                return nil
        }
        for _, ev := range events {
                eventDir := filepath.Join(svc.eventDir, ev.FileSHA, machineID)
                if err := os.MkdirAll(eventDir, 0700); err != nil {
                        return errors.Wrapf(err, "create event directory %s", eventDir)
                }
                eventPath := filepath.Join(eventDir, fmt.Sprintf("%f.json", ev.UnixTime))
                eventInfoJSON, err := json.Marshal(ev.EventInfo)
                if err != nil {
                        return errors.Wrap(err, "marshal event info to json")
                }
                // Decode JSON data into a map[string]interface{}
                var eventInfoMap map[string]interface{}
                if err := json.Unmarshal(eventInfoJSON, &eventInfoMap); err != nil {
                        return errors.Wrap(err, "unmarshal eventInfoJSON")
                }
                // Add machineID to the map
                eventInfoMap["serial"] = machineID
                // Marshal the modified map back into JSON format
                updatedEventInfoJSON, err := json.Marshal(eventInfoMap)
                if err != nil {
                        return errors.Wrap(err, "marshal updated eventInfoJSON")
                }
                if err := os.WriteFile(eventPath, updatedEventInfoJSON, 0644); err != nil {
                        return errors.Wrapf(err, "write event to path %s", eventPath)
                }
                req, err := http.NewRequest("POST", "http://<YOUR IP HERE>:8080", bytes.NewReader(updatedEventInfoJSON))
                req.Header.Set("Content-Type", "application/json")
                client := &http.Client{Timeout: time.Minute}
                // Execute the HTTP request asynchronously in a goroutine
                go func() {
                        resp, err := client.Do(req)
                        if err != nil {
                                // Handle error if occurred during HTTP request
                                // (such as connection error, timeout, etc.)
                                return
                        }
                        defer resp.Body.Close()
                        // Optionally, you can process the response here if needed
                }()
                return nil
        }
        return nil
}

4. Compile by running cd cmd/moroz; go build

There's probably a better way to do this, but I'm not a Go coder so it's the quick and dirty way I came up with.

[thebiblelover7]

@clreinki That might be useful! Thanks! What do you feed these logs into? Grafana?

[clreinki]

Sending it to Elasticsearch and visualizing in Kibana