github / ghec-audit-log-cli

Query the GitHub Audit Log for your organization to send it over to other services like elastic, splunk or sentinel for visualization and security
https://github.com/github/ghec-audit-log-cli
MIT License
63 stars 11 forks source link

Cursor check is failing "Invalid Cursor" #32

Open ghost opened 3 years ago

ghost commented 3 years ago

Describe the bug The cursor the CLI is commiting is causing the next run to fail because the cursor is not passing the regex expression.

The CLI is commiting the id of the Type.

Here are some example, of the new "cursor" it is commiting

"event_type": "cursor that is commited"
{
   "RepoAccessAuditEntry":"RAAE_kgC2dmxGYVowTlpYQlRUTzJsaE5VMDNFQQ",
   "OrgInviteMemberAuditEntry":"OIMAE_kgC2OGVGWlJlTGh5OWMyMTMxdURLWmVNUQ",
   "TeamAddMemberAuditEntry":"TAMAE_kgC2aEJOV3M1Tk5vQmkxSkd6LWJwSzhBZw",
   "TeamAddRepositoryAuditEntry":"TARAE_kgC2VVBWeFdHbWpNVHZLMVdOQzBJUWFRQQ",
   "RepoCreateAuditEntry":"RCAE_kgC2eU1jSUM2NkJrWXVHNnZXQXFVMnItdw",
   "RepoAddMemberAuditEntry":"RAMAE_kgC2TXpObGdGV21ISUxlTEttcmNreGtZZw",
   "OrgAddMemberAuditEntry":"OAMAE_kgC2aFl0YmFfaHdUdUJ4c1Rlbkl1X1U3Zw",
   "RepoRemoveMemberAuditEntry":"RRMAE_kgC2NVpuYlhLOW9kOHNOaDNzcWMyT2stQQ",
   "TeamChangeParentTeamAuditEntry":"TCPTAE_kgC2SWRzNTlwd2hnY0tWN0ZGZHBNb2cwQQ",
   "RepoArchivedAuditEntry":"RARAE_kgC2aVV3RjVsRlpTbC1VQUszNDVnM3hhZw",
   "OrgRemoveMemberAuditEntry":"ORMAE_kgC2VXQ0Y3JlV2xFbFBsTjBHTk1kOExzZw",
   "TeamRemoveRepositoryAuditEntry":"TRRAE_kgC2S0lXcUVYSGVmNW14T2xvWFRYaE0xZw",
   "TeamRemoveMemberAuditEntry":"TRMAE_kgC2SXVDZVBlY0xXTUNtSlRNeERmWG51Zw",
   "RepoDestroyAuditEntry":"RDAE_kgC2SmI4cXNYVGVOcFFlNWpMVlhsLThCQQ",
   "RepoChangeMergeSettingAuditEntry":"RCMSAE_kgC2Q1JVMjF6OTFhOVZYU2pWbkx1NkRZUQ",
   "RepoAddTopicAuditEntry":"RATAE_kgC2Y200cE5HSnU1VjdBdHNkVm4zdzhDQQ",
   "OrgRestoreMemberAuditEntry":"ORSMAE_kgC2ODhMQ1NkYk4zcktKWXR6NHh5ZXlpZw"
}

Removing the regex check, and letting the cli use the newly commited cursor looks like it allows the cli to gather new events, and it looks like it is not missing any events.

However, the graphql, always says there are new pages after the last event pulled, I'd assume this should be false when we pull the last event.

{
    "organization": {
        "auditLog": {
            "pageInfo": {
                "endCursor": "MS42MzI0MTMwNTQxNzZlKzEyfEsyNnFzbjktaHpaVm1ycFphU2s2THc=",
                "hasNextPage": true
            },
            "nodes": [{
                "__typename": "RepoCreateAuditEntry",
                "id": "RCAE_kgC2bE5VWGotUThlMll0ckZ4S2hfbG8tZw", #<---- this is what gets commited as the last cursor pulled
                "action": "repo.create",

To Reproduce Steps to reproduce the behavior: Run the CLI - check the output of the cursor it saves

Expected behavior The CLI will output a valid cursor that it will use the next run to get all new events

Other

I'm not sure the reason for the change in behavior, but if we can confirm that the new formats are valid we can update the validation.

droidpl commented 3 years ago

I believe the reason may be some change in the cursor format. Bypassing the validation shouldn't be the solution , but supporting the new format. I don't have permissions anymore in this repo to be able to contribute/approve contributions, and I am not sure why a new format was introduced. @ppremk may be able to.help us here with internal details.

@ffalorjr with the error you are seeing can you reproduce it and create a PR with a fix? It requires an enterprise for it. 🐱