Add logging and recording of requests

[tateexon]

Added two new command line arguments:

• -l for logging level, defaults to 0, 1 adds requests, 2 adds the request body
• -d to dump requests into a the specified file

This is needed for cases when a test needs access to what is posted as a request to the mock but we don't know the exact data, like a hash that was posted but need that hash later on in the test to post to another service. We don't need the responses dumped because the test should have access to the imposters they are using and thus already have access to that data.

I think this also covers what was originally wanted in https://github.com/friendsofgo/killgrave/issues/100 but maybe not exactly how it was imagined there.

I added a few changes to the Makefile and Dockerfile to make local testing a bit easier.

I made the request writer use a channel and run in a go function so it wouldn't run into issues of multiple concurrent requests trying to write into the dump file at once.

I fully realize this could cause the dump file to get very very large if this is used incorrectly in something like a load test but I am not sure how to handle that better outside of doing some kind of rotating log. I would hope people wouldn't use it for a case like that but I don't know what all the kinds of users of the tool are. I welcome all suggestions.

[tateexon]

@joanlopez I would like your thoughts on this please!

[iljapavlovs]

upvote for this feature, really missing this essential feature! @joanlopez

[tateexon]

@aperezg @joanlopez is there anything else I need to do before this can get reviewed?

[joanlopez]

@aperezg @joanlopez is there anything else I need to do before this can get reviewed?

It's fine for now, @tateexon - just trying to book some time to review it carefully! Thanks! 🙇🏻

[tateexon]

Sounds good. I will continue with my fork for now to unblock my QA team.

[tateexon]

I think I have hit all the review comments so far. Anything else @asad-urrahman ? Thanks for the reviews so far!

[bigmeech]

when is this getting merged in?

[joanlopez]

when is this getting merged in?

TBH, I'd love to see this merged sooner rather than later. However, the set of changes here is very large, and some are trickier to review, discuss and approve than others. That's why I just suggested @tateexon to split this into smaller PRs.

From your side @bigmeech, is there anything you're more interested about? Logging? Dumping? The minor fixes?

Thanks!

[tateexon]

@joanlopez Thank you very much for the review!

I broke out a PR for the makefile and dockerfile changes.

I think I have hit most of your review comments and that brought a lot of change. I would like another review before I go in and add more tests just in case it needs a lot more change.

Some of the things that have changed:

• the copied gorilla code is gone
• we just output in json format now and we can add an option later to output in other formats as needed
• used a context and wait group to cleanly handle shutdown for the request writer go routine
• the request writer now uses an io.Writer instead of a file so the file handling and deferred close can be handled at higher layers where they make sense
• reversed the order for when we base 64 encode
• added a serverconfig package to functionally configure and add more options in the future using a With* syntax

[bigmeech]

when is this getting merged in?

TBH, I'd love to see this merged sooner rather than later. However, the set of changes here is very large, and some are trickier to review, discuss and approve than others. That's why I just suggested @tateexon to split this into smaller PRs.

From your side @bigmeech, is there anything you're more interested about? Logging? Dumping? The minor fixes?

Thanks!

Hi @joanlopez

Thanks for the work you have done in the PR. To answer your question actually there are a few stuffs I'd be interested in but I think the most important for me personally is responding to the client with a 400 bad request when the schema validation fails instead of only reporting the failed validation on the server logs. (at least this is what happens in the version i am using). Not sure how big of a change that would be.

Thanks.