Document a few use cases for submitting tests to the Orchestrator

[SidHarder]

@dgrisham I wanted to touch base with you about yesterdays call. Will you have some time this week to experiment with the Kubernetes Minikube, or is there something else you were planning to work on?

[dgrisham]

@SidHarder My main task was to get more familiar with using IPFS and come up with ideas for test cases to use once the basic Kubernetes orchestrator + VM distribution is set up. However, I'd be happy to experiment with the Minikube if that would also be useful. After my exam tomorrow morning I'll have quite a bit of time to devote to all of this, so let me know if that'd be helpful (and what exactly to look into aside from getting familiar with it, if anything).

[SidHarder]

@dgrisham I think that you are on the right track. I think we probably have enough people working on Kubernetes. I think it would be good to take some time and get yourself familiar with IPFS. I would stick with the GO implementation for now. I would start with experimenting with the command line. You could start with https://ipfs.io/docs/commands/

I think you will find it's pretty amazing. If you could think about some use cases and document them here in this issue that would be awesome.

Thanks for your help.

[dgrisham]

@SidHarder Sounds good, and thanks for the link! I've done some of the basics, but this list of commands will be a nice way to discover even more. I'll document what I come up with on this page, as you mentioned.

[SidHarder]

@dgrisham Another Thought I had is that if we could come up with a super simple test and then package that test inside a Docker image that could be helpful. The test could be as simple as add 10 file to ipfs and then cat them back. Could be written using a simple batch script. Let me know what you think.

[dgrisham]

@SidHarder ah, yeah I could make that happen. I went through a Docker tutorial the other day and it seemed straightforward enough. I'll start with simple cases like the one you suggested. should I maintain my test-case progress through the repo or just locally for now?

Edit: Also, would it be useful to focus my tests on self-contained swarms (not sure if that use case is well-established yet) or just the global ipfs network?

[SidHarder]

@dgrisham If you could make a few notes in the issue about your progress that would be great. With respect to a "self-contained swarm", I actually have a real world need to test this use case. So I would say go for it.

[dgrisham]

@SidHarder Sounds good. My initial idea for implementing that would be to set up the swarms/bootstrap nodes/etc. in the config file in the appropriate manner, is there an idiomatic approach to setting that up that you could direct me to? Or would it just require me to play with it and set it up myself?

[SidHarder]

@dgrisham The only way I no to do it is in the config. You might as @whyrusleeping or @Kubuxu

[dgrisham]

Gotcha, any input here @whyrusleeping or @Kubuxu ? If not I'll proceed as we discussed @SidHarder.

[Kubuxu]

@dgrisham you can either modify the json in config file directly or use ipfs config ..... It allows to modify the json in a form ipfs config --json Addresses.Swarm '["/ip4/...."]'.

[dgrisham]

Ah, very nice, makes it easily scriptable. Thanks @Kubuxu!

[Kubuxu]

Also for bootstrap you have ipfs bootstrap (add|rm|list).

[dgrisham]

I'm digging the design philosophy here, simple base commands that are easy to leverage for various use cases. Thanks again Jakub.

[dgrisham]

@SidHarder I have a working Docker image (for reference, it's on dockerhub at dgrisham/ipfs-test). It's packaged with a few short files and a script that adds those files to ipfs, then cats them back.

Let me know if you have any recommended next steps. Otherwise, I'm going to start working on a self-contained swarm example.

[dgrisham]

@SidHarder Frank showed me how a self-contained swarm would work for the purposes of our tests (simple example). I'd like to get an automated means of setting up a 'private' cluster going because this is only useful when we have the Kubernetes orchestrator, but for the purpose of testing I'll stick with this for now.

You mentioned a use-case with a self-contained swarm, would you mind detailing a bit about that? Might make for a good test.

[dgrisham]

Test ideas

General

• Variations on simple file exchange (these are all finished)
  • [x] 1 user add's a file, 10 users cat that file
  • [x] 1 users add's 10 files, 10 users each cat one of the files
  • [x] 1 user add's 10 files, 10 users each cat all of the files
  • [x] 10 users each add a file, each of the users cat's a single file added by one of the other users
  • [x] 10 users each add a file, each user cat's all of the files

Other/future

• Streaming
  • e.g. ipfs cat <hash> | mpv
  • vary number of nodes with content
  • easy way to measure buffering/waiting time?
  • Metrics of interest: Network load, memory working set, CPU usage
• Single-file updating
  • 'Server' node public ID tied to current file version
  • 'Client' nodes take turns updating/adding to file, server updates pointer
  • could extend to something with/like git ?
  • Metrics of interest: Network load, filesystem usage

Metrics

• Topology measurement
  • Start with specific set of connections (initial topology), exchange one or more files, measure final topology (ipfs swarm peers).
  • Simplest case: Three peers, 'line' topology (1 <-> 2 <-> 3), peer 1 downloads something from peer 3.
  • Visualization with graphviz?
• Anything kubernetes can give us
  • Network load
  • Working set in memory
  • CPU usage
  • Filesystem usage

[dgrisham]

@SidHarder Here are some simple tests, let me know if you have any feedback/recommendations . I'll keep updating the above list as I come up with/complete tests.