Gather topology/request samples and tests in one place

sajith commented 1 year ago

We have topology files in three places:

in sdx-controller/samples/topologies
in sdx-controller/tests/data
in pce/tests/data/topologies

With topology files under sdx-controller/samples/topologies and the sample test request that we've been testing with (namely test_sample.json), TESolver fail to find a solution. With the topology files in pce/tests/data/topologies, things work as expected (a solution is found, and it breaks down to links in three domains). This has been quite confusing.

I am proposing four steps to avoid future confusion (once the current release target is met):

[ ] pce/tests/data/topologies: This works. In order to avoid confusion, it would be helpful to have only one version around. Let's make sure to use just these topology files when running tests "live" on the test bed.
[ ] sdx-controller/samples/topologies: Let us remove this altogether.
[ ] sdx-controller/tests/data: The tests in tests/test_Solver.py are now basically unit tests for pce. Let us move the tests and test data to pce/tests.
[ ] Further, it would be helpful to document the provenance, "canonicalness", and the process for creating/updating these topology files. (This probably has already been done, and I'm just unaware of it?)

Related to:

YufengXin commented 1 year ago

make sense to me

YufengXin commented 1 year ago

The current topology and request json files were all generated by me manually for testing. I'm waiting for the completion of the Kytos-LC integration so we can generate a suite of realistic testing files.

sajith commented 1 year ago

I compared the topology files with some "better" diffing tools (plain old diff is not very helpful) such as difftastic and jsondiff.com. The big difference is that the the topology that broke during the demo had some really high latency values.

https://github.com/atlanticwave-sdx/sdx-controller/blob/cb46d516926e53fb55ceb1fe9dc82e399dba971a/samples/topologies/amlight.json#L12

https://github.com/atlanticwave-sdx/sdx-controller/blob/cb46d516926e53fb55ceb1fe9dc82e399dba971a/samples/topologies/amlight.json#L46

https://github.com/atlanticwave-sdx/sdx-controller/blob/cb46d516926e53fb55ceb1fe9dc82e399dba971a/samples/topologies/amlight.json#L80

https://github.com/atlanticwave-sdx/sdx-controller/blob/cb46d516926e53fb55ceb1fe9dc82e399dba971a/samples/topologies/amlight.json#L114

... and so on. You get the idea. :-)

sajith commented 1 year ago

Perhaps this will work: https://jsoncompare.com/#!/diff/id=3eb161670c9ab0c9934c33dbfc53d5aa/

YufengXin commented 1 year ago

Great! Oh, poor TE solver.... So, need to add the realistic upper bound on the latency and lower bound on the bandwidth as part of the validation....

atlanticwave-sdx / sdx-controller

Gather topology/request samples and tests in one place #168