Update examples

[varunsh-xilinx]

Summary of Changes

• Redo examples

Closes #84

Motivation

The examples are intended to show new users a simplified sample of how to interact with the server, not as a tool to benchmark and profile the different backends. The edited examples are structured similarly to each other to show users how similar making inferences to different models can be. Future examples can be added to demonstrate different features.

Implementation

The examples are now sorted into directories by type rather than by language. The ResNet50 examples in particular span the gamut of C++ and Python across all the backends. They are intentionally repetitive to be standalone and complete. The ResNet50 example is accompanied by a document on the online documentation that shows the different steps. The other models are currently not online as I'm unsure what needs to be highlighted about them and how it should be done.

The validation script for resnet50 is moved to tests/validation. Currently, it only works with migraphx but should be generalized to all backends as well as add timing information.

[gbuildx]

Build failed!

[varunsh-xilinx]

retest this please

[gbuildx]

Build failed!

[gbuildx]

Build failed!

[varunsh-xilinx]

retest this please

[gbuildx]

Build failed!

[varunsh-xilinx]

These repeated failures are due to the pipeline timing out because of increased load on the pipeline server. The tests pass locally so I'm merging it to address #86. If there are any issues, they'll be patched in a new PR.

[gbuildx]

Build failed!

[bpickrel]

General feedback:

• the main reader takeaways for the yolo and bert models (besides how to do model-specific pre and post processing, which is available elsewhere) are how to populate multi-channel inputs and parse multi-channel outputs. One difference between the two, for migraphx, is that inputs are referenced by name while outputs don't support names (an migraphx API limitation) and must be read in order.
• migraphx resnet model can also serve as an example or self-test for bombing the server with lots of requests at a time.
• Does the modified resnet script try to submit all 50,000 imagenet images at once? I didn't try it. If so, it may crash and it may be necessary to change the script to submit a few at a time, as the original did.
• Is it possible to interrupt the resnet request set with Ctrl-C etc, or does the server try to process all the images once requested? Users will very rarely want to really wait for the entire set, which will take hours via REST interface. Another reason to provide a way to submit just a few requests at a time.
• I don't see any explanation of the meaning of batch size and timeout. Examples should illustrate these too.

[varunsh-xilinx]

I can add a bit more about the multi-channel input/output in the comments. Same thing for the names

Your migraph_resnet script is in tests/validation and it still sends requests in batches of 64. I did try running it for a few iterations. It runs that far at least. Killing it with Ctrl+C doesn't work (or maybe it takes a while, not sure) but you can background the process with Ctrl+Z first and then kill it.

I did think about explaining batch size and timeout but if I'm assuming someone brand new is looking at this example, I don't know if they have enough context to really know what that means. I can put a little bit but I think it has to be a pointer to somewhere else for more details. I was thinking of having these parameters listed somewhere else (e.g. each worker having a page listing its specific quirks like how the migraphx worker is compiling to mxr under the hood) since they're worker-specific. The batcher level ones like timeout can be elsewhere since those are shared

[gbuildx]

Build failed!