search

petergtz / pegomock

Pegomock is a powerful, yet simple mocking framework for the Go programming language
Apache License 2.0
253 stars 28 forks source link

Add debug methods to mocks #41

Closed petergtz closed 5 years ago

petergtz commented 7 years ago

These methods are only meant to be used to debug tests, not for any assertions or similar.

lkysow commented 6 years ago

It would be awesome to have these called automatically when a Verify call fails. For example:

method Foo() was not called with "bar"
it was called with "baz"
petergtz commented 6 years ago

@lkysow Agree, that would be very helpful. Will see if I can make some time to add this. I'm happy to take this as a PR as well of course :-).

petergtz commented 6 years ago

@lkysow Is this satisfying your request?

lkysow commented 6 years ago

yes! It's definitely better. Only way I could see to improve it would be to format the output in a more readable way.

This is what I saw before:

--- FAIL: TestExecuteCommand_FullRun (0.00s)
    command_handler_test.go:141: when running a plan, apply or help should comment
    testing_t_support.go:41:
            ***/go/src/github.com/hootsuite/atlantis/vendor/github.com/petergtz/pegomock/testing_t_support.go:40 +0x59
        github.com/hootsuite/atlantis/vendor/github.com/petergtz/pegomock.(*GenericMock).Verify(0xc42000e370, 0x0, 0x160d500, 0xc4200e9920, 0x141f792, 0x6, 0xc42005fa40, 0x4, 0x4, 0x0, ...)
            ***/go/src/github.com/hootsuite/atlantis/vendor/github.com/petergtz/pegomock/dsl.go:111 +0x71d
        github.com/hootsuite/atlantis/server/events/mocks.(*VerifierGHStatusUpdater).Update(0xc42010b930, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
            ***/go/src/github.com/hootsuite/atlantis/server/events/mocks/mock_github_status.go:65 +0x2c6
        github.com/hootsuite/atlantis/server/events_test.TestExecuteCommand_FullRun(0xc4201685a0)
            ***/go/src/github.com/hootsuite/atlantis/server/events/command_handler_test.go:171 +0xca7
        testing.tRunner(0xc4201685a0, 0x14388d0)
            /usr/local/Cellar/go/1.9/libexec/src/testing/testing.go:746 +0xd0
        created by testing.(*T).Run
            /usr/local/Cellar/go/1.9/libexec/src/testing/testing.go:789 +0x2de

        Mock invocation count for method "Update" with params [{    } {1 16ca62f65c18ff456c6ef4cacc8d4826e264bb17 8ed0280678d49d42cd286610aabcfceb5bb673c6 url branch ****} pending 0xc42005f600] does not match expectation.

            Expected: 1; but got: 0

After your changes it's much better

--- FAIL: TestExecuteCommand_FullRun (0.00s)
    command_handler_test.go:141: when running a plan, apply or help should comment
    testing_t_support.go:41:
            ***/go/src/github.com/hootsuite/atlantis/vendor/github.com/petergtz/pegomock/testing_t_support.go:40 +0x59
        github.com/hootsuite/atlantis/vendor/github.com/petergtz/pegomock.(*GenericMock).Verify(0xc42000e390, 0x0, 0x160f500, 0xc4200e99a0, 0x1420812, 0x6, 0xc420065a80, 0x4, 0x4, 0x0, ...)
            ***/go/src/github.com/hootsuite/atlantis/vendor/github.com/petergtz/pegomock/dsl.go:115 +0x7a9
        github.com/hootsuite/atlantis/server/events/mocks.(*VerifierGHStatusUpdater).Update(0xc420109930, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
            ***/go/src/github.com/hootsuite/atlantis/server/events/mocks/mock_github_status.go:65 +0x2c6
        github.com/hootsuite/atlantis/server/events_test.TestExecuteCommand_FullRun(0xc42016c5a0)
            ***/go/src/github.com/hootsuite/atlantis/server/events/command_handler_test.go:171 +0xca7
        testing.tRunner(0xc42016c5a0, 0x14399b0)
            /usr/local/Cellar/go/1.9/libexec/src/testing/testing.go:746 +0xd0
        created by testing.(*T).Run
            /usr/local/Cellar/go/1.9/libexec/src/testing/testing.go:789 +0x2de

        Mock invocation count for method "Update" with params [{    } {1 16ca62f65c18ff456c6ef4cacc8d4826e264bb17 8ed0280678d49d42cd286610aabcfceb5bb673c6 url branch ****} pending 0xc420065640] does not match expectation.

            Expected: 1; but got: 0

            But other interactions with this mock were:
            Update({hootsuite/atlantis hootsuite atlantis https://user:password@github.com/hootsuite/atlantis.git https://github.com/hootsuite/atlantis.git}, {1 16ca62f65c18ff456c6ef4cacc8d4826e264bb17 8ed0280678d49d42cd286610aabcfceb5bb673c6 url branch ****}, pending, &{plan env false []})
            UpdateProjectResult(&{{hootsuite/atlantis hootsuite atlantis https://user:password@github.com/hootsuite/atlantis.git https://github.com/hootsuite/atlantis.git} {hootsuite/atlantis hootsuite atlantis https://user:password@github.com/hootsuite/atlantis.git https://github.com/hootsuite/atlantis.git} {1 16ca62f65c18ff456c6ef4cacc8d4826e264bb17 8ed0280678d49d42cd286610aabcfceb5bb673c6 url branch ****} {****} 0xc420065640 0xc420114870}, {<nil>  []})

But it's still hard to read the invocations.

If I change to Sprintf("+%v", param) it looks like

--- FAIL: TestExecuteCommand_FullRun (0.00s)
    command_handler_test.go:141: when running a plan, apply or help should comment
    testing_t_support.go:41:
            ***/go/src/github.com/hootsuite/atlantis/vendor/github.com/petergtz/pegomock/testing_t_support.go:40 +0x59
        github.com/hootsuite/atlantis/vendor/github.com/petergtz/pegomock.(*GenericMock).Verify(0xc42013c300, 0x0, 0x160f500, 0xc4201313c0, 0x1420832, 0x6, 0xc420132ec0, 0x4, 0x4, 0x0, ...)
            ***/go/src/github.com/hootsuite/atlantis/vendor/github.com/petergtz/pegomock/dsl.go:115 +0x7a9
        github.com/hootsuite/atlantis/server/events/mocks.(*VerifierGHStatusUpdater).Update(0xc420109930, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
            ***/go/src/github.com/hootsuite/atlantis/server/events/mocks/mock_github_status.go:65 +0x2c6
        github.com/hootsuite/atlantis/server/events_test.TestExecuteCommand_FullRun(0xc42016e5a0)
            ***/go/src/github.com/hootsuite/atlantis/server/events/command_handler_test.go:171 +0xca7
        testing.tRunner(0xc42016e5a0, 0x14399d0)
            /usr/local/Cellar/go/1.9/libexec/src/testing/testing.go:746 +0xd0
        created by testing.(*T).Run
            /usr/local/Cellar/go/1.9/libexec/src/testing/testing.go:789 +0x2de

        Mock invocation count for method "Update" with params [{    } {1 16ca62f65c18ff456c6ef4cacc8d4826e264bb17 8ed0280678d49d42cd286610aabcfceb5bb673c6 url branch ****} pending 0xc420132a80] does not match expectation.

            Expected: 1; but got: 0

            But other interactions with this mock were:
            Update({FullName:hootsuite/atlantis Owner:hootsuite Name:atlantis CloneURL:https://user:password@github.com/hootsuite/atlantis.git SanitizedCloneURL:https://github.com/hootsuite/atlantis.git}, {Num:1 HeadCommit:16ca62f65c18ff456c6ef4cacc8d4826e264bb17 BaseCommit:8ed0280678d49d42cd286610aabcfceb5bb673c6 URL:url Branch:branch Author:****}, pending, &{Name:plan Environment:env Verbose:false Flags:[]})
            UpdateProjectResult(&{BaseRepo:{FullName:hootsuite/atlantis Owner:hootsuite Name:atlantis CloneURL:https://user:password@github.com/hootsuite/atlantis.git SanitizedCloneURL:https://github.com/hootsuite/atlantis.git} HeadRepo:{FullName:hootsuite/atlantis Owner:hootsuite Name:atlantis CloneURL:https://user:password@github.com/hootsuite/atlantis.git SanitizedCloneURL:https://github.com/hootsuite/atlantis.git} Pull:{Num:1 HeadCommit:16ca62f65c18ff456c6ef4cacc8d4826e264bb17 BaseCommit:8ed0280678d49d42cd286610aabcfceb5bb673c6 URL:url Branch:branch Author:****} User:{Username:****} Command:0xc420132a80 Log:0xc42016c7e0}, {Error:<nil> Failure: ProjectResults:[]})

Which is nice. I like this improvement. I then wanted to see what it would look like formatted. I used https://github.com/davecgh/go-spew but it looks too busy now I think.

--- FAIL: TestExecuteCommand_FullRun (0.00s)
    command_handler_test.go:141: when running a plan, apply or help should comment
    testing_t_support.go:41:
            ***/go/src/github.com/hootsuite/atlantis/vendor/github.com/petergtz/pegomock/testing_t_support.go:40 +0x59
        github.com/hootsuite/atlantis/vendor/github.com/petergtz/pegomock.(*GenericMock).Verify(0xc420144338, 0x0, 0x161d660, 0xc4201474e0, 0x142b317, 0x6, 0xc420133200, 0x4, 0x4, 0x0, ...)
            ***/go/src/github.com/hootsuite/atlantis/vendor/github.com/petergtz/pegomock/dsl.go:116 +0x7a9
        github.com/hootsuite/atlantis/server/events/mocks.(*VerifierGHStatusUpdater).Update(0xc420109930, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
            ***/go/src/github.com/hootsuite/atlantis/server/events/mocks/mock_github_status.go:65 +0x2c6
        github.com/hootsuite/atlantis/server/events_test.TestExecuteCommand_FullRun(0xc4201925a0)
            ***/go/src/github.com/hootsuite/atlantis/server/events/command_handler_test.go:171 +0xca7
        testing.tRunner(0xc4201925a0, 0x1444580)
            /usr/local/Cellar/go/1.9/libexec/src/testing/testing.go:746 +0xd0
        created by testing.(*T).Run
            /usr/local/Cellar/go/1.9/libexec/src/testing/testing.go:789 +0x2de

        Mock invocation count for method "Update" with params [{    } {1 16ca62f65c18ff456c6ef4cacc8d4826e264bb17 8ed0280678d49d42cd286610aabcfceb5bb673c6 url branch ****} pending 0xc420132dc0] does not match expectation.

            Expected: 1; but got: 0

            But other interactions with this mock were:
            Update((models.Repo) {
         FullName: (string) (len=18) "hootsuite/atlantis",
         Owner: (string) (len=9) "hootsuite",
         Name: (string) (len=8) "atlantis",
         CloneURL: (string) (len=55) "https://user:password@github.com/hootsuite/atlantis.git",
         SanitizedCloneURL: (string) (len=41) "https://github.com/hootsuite/atlantis.git"
        }
        , (models.PullRequest) {
         Num: (int) 1,
         HeadCommit: (string) (len=40) "16ca62f65c18ff456c6ef4cacc8d4826e264bb17",
         BaseCommit: (string) (len=40) "8ed0280678d49d42cd286610aabcfceb5bb673c6",
         URL: (string) (len=3) "url",
         Branch: (string) (len=6) "branch",
         Author: (string) (len=6) "****"
        }
        , (events.Status) 0
        , (*events.Command)({
         Name: (events.CommandName) 1,
         Environment: (string) (len=3) "env",
         Verbose: (bool) false,
         Flags: ([]string) <nil>
        })
        )
            UpdateProjectResult((*events.CommandContext)({
         BaseRepo: (models.Repo) {
          FullName: (string) (len=18) "hootsuite/atlantis",
          Owner: (string) (len=9) "hootsuite",
          Name: (string) (len=8) "atlantis",
          CloneURL: (string) (len=55) "https://user:password@github.com/hootsuite/atlantis.git",
          SanitizedCloneURL: (string) (len=41) "https://github.com/hootsuite/atlantis.git"
         },
         HeadRepo: (models.Repo) {
          FullName: (string) (len=18) "hootsuite/atlantis",
          Owner: (string) (len=9) "hootsuite",
          Name: (string) (len=8) "atlantis",
          CloneURL: (string) (len=55) "https://user:password@github.com/hootsuite/atlantis.git",
          SanitizedCloneURL: (string) (len=41) "https://github.com/hootsuite/atlantis.git"
         },
         Pull: (models.PullRequest) {
          Num: (int) 1,
          HeadCommit: (string) (len=40) "16ca62f65c18ff456c6ef4cacc8d4826e264bb17",
          BaseCommit: (string) (len=40) "8ed0280678d49d42cd286610aabcfceb5bb673c6",
          URL: (string) (len=3) "url",
          Branch: (string) (len=6) "branch",
          Author: (string) (len=6) "****"
         },
         User: (models.User) {
          Username: (string) (len=6) "****"
         },
         Command: (*events.Command)({
          Name: (events.CommandName) 1,
          Environment: (string) (len=3) "env",
          Verbose: (bool) false,
          Flags: ([]string) <nil>
         }),
         Log: (*logging.SimpleLogger)({
          Source: (string) (len=25) "hootsuite/atlantis/pull/1",
          History: (bytes.Buffer) {
           buf: ([]uint8) <nil>,
           off: (int) 0,
           lastRead: (bytes.readOp) 0,
           bootstrap: ([64]uint8) (len=64) {
            00000000  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
            00000010  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
            00000020  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
            00000030  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
           }
          },
          Logger: (*log.Logger)({
           mu: (sync.Mutex) {
            state: (int32) 0,
            sema: (uint32) 0
           },
           prefix: (string) "",
           flag: (int) 0,
           out: (ioutil.devNull) 0,
           buf: ([]uint8) <nil>
          }),
          KeepHistory: (bool) true,
          Level: (logging.LogLevel) 1
         })
        })
        , (events.CommandResponse) {
         Error: (error) <nil>,
         Failure: (string) "",
         ProjectResults: ([]events.ProjectResult) <nil>
        }
        )
petergtz commented 6 years ago

@lkysow Agree, Sprintf("%+v", param) definitely makes it more readable for structs. I'll change it. (BTW, feel free to open small PRs for stuff like this.)

I think it also makes sense to change it in the Mock invocation count for method... part, which is not at all readable for now.

For your idea to further format the parameters including line breaks etc. I start to wonder if that should be addressed differently. E.g. in your case I'm wondering if this formatted output is mostly desirable while writing tests and implementation, more or less to make that process less painful. Once it works, and only occasionally fails when making future code changes, it might be acceptable to get the less formatted output.

And if the above is true, then it might be more pragmatic to use GetAllCapturedArguments() and print that out using go-spew while developing the tests.

Would that work for you?

Another idea I had was to introduce a debug mode in which mocks would simply print out whenever they're invoked from somewhere, including file and line numbers and the parameters. I'm tracking ideas in https://github.com/petergtz/pegomock/issues/32.

lkysow commented 6 years ago

You're right that the formatted output is probably only worth using during debugging tests. Good call on using GetAllCaptured for debugging. I think this might be worth a section on the README called "Debugging Mocks".

And yes :) I know I should just submit this as a PR but ⌛️ is always the issue! Thanks for your work here, I think your library can become the best mocking library.

petergtz commented 6 years ago

@lkysow I've just pushed a change which now actually uses Sprintf("%#v", param). With that, you not only get names in the structs, but the full Go-syntax representation of the value. I think that makes it even more transparent what values are in your params. Let me know if you see problems with that.

Good call on using GetAllCaptured for debugging. I think this might be worth a section on the README called "Debugging Mocks".

Good point. I will try to write something up.

I think your library can become the best mocking library.

Thank you for these kind words. Would love to see that!