hashicorp-forge / grove

A Software as a Service (SaaS) log collection framework.
https://hashicorp-forge.github.io/grove/
Mozilla Public License 2.0
130 stars 11 forks source link

github_log fails to pull logs with default scope #58

Closed ismajl-ramadani closed 1 month ago

ismajl-ramadani commented 1 month ago

When trying to setup a github connector to fetch audit logs for organization scope, its failing.

This is the configuration that I'm using:

{
  "key": "$TOKEN$",
  "identity": "$ORG_NAME$",
  "delay": 10,
  "name": "github-orgs-audit-log-example",
  "connector": "github_audit_log"
}

I can see the following error on the logs:

{"exception": "404 Client Error: Not Found for url: https://api.github.com/orgs/$ORG_NAME$/audit-log?phrase=created%3A%3E%3D2024-09-24T12%3A59%3A43Z+created%3A%3C%3D2024-10-01T12%3A49%3A43Z&include=all&order=asc&per_page=100", "operation": "all", "connector": "grove.connectors.github.audit_log", "identity": "$ORG_NAME$"}, "context": {"runtime": "/usr/local/lib/python3.9/site-packages/grove/entrypoints/local_process.py", "runtime_id": "1", "runtime_host": ""}}

I wanted to know if someone else had the same issue, or managed to get a working example using the gh connector. What I managed to observe is that the /audit-log is not available for organizations that are not on the enterprise plan, but I'm not sure about this as I can get the logs if I use Webhooks.

hcpadkins commented 1 month ago

Hey there,

Thanks for raising this issue. Unfortunately, according to the Github documentation, it appears that the audit logging REST API which Grove uses for log collection is only supported by Github Enterprise Cloud accounts, where it appears webhook style delivery may be supported by other subscription types.

"Organizations that use GitHub Enterprise Cloud can interact with the audit log using the GraphQL API and REST API. For more information, see the GitHub Enterprise Cloud documentation."

Per using the audit log api.

ismajl-ramadani commented 1 month ago

Thank you @hcpadkins for confirming this.