Open italovalcy opened 2 years ago
Another test showed that this is not specific to the environment I'm running at. Steps to reproduce:
amlight/kytos:latest
(Ex: docker run -d --name k99 --privileged amlight/kytos:latest
)docker exec -it k99 tmux new-sess -d "mn --topo=linear,10 --controller=remote,ip=127.0.0.1"
docker exec -it k99 bash
IP=127.0.0.1; for sw in $(curl -s http://$IP:8181/api/kytos/topology/v3/switches | jq -r '.switches[].id'); do curl -H 'Content-type: application/json' -X POST http://$IP:8181/api/kytos/topology/v3/switches/$sw/enable; curl -H 'Content-type: application/json' -X POST http://$IP:8181/api/kytos/topology/v3/interfaces/switch/$sw/enable; done
IP=127.0.0.1; for l in $(curl -s http://$IP:8181/api/kytos/topology/v3/links | jq -r '.links[].id'); do curl -H 'Content-type: application/json' -X POST http://$IP:8181/api/kytos/topology/v3/links/$l/enable; done
for vlan in $(seq 401 500); do curl -H 'Content-type: application/json' -X POST http://127.0.0.1:8181/api/kytos/mef_eline/v2/evc -d '{"name": "evc-vlan-'$vlan'", "dynamic_backup_path": true, "enabled": true, "uni_z": {"tag": {"value": '$vlan', "tag_type": 1}, "interface_id": "00:00:00:00:00:00:00:01:1"}, "uni_a": {"tag": {"value": '$vlan', "tag_type": 1}, "interface_id": "00:00:00:00:00:00:00:0a:1"}}'; done
load-from-storehouse.py
script above, try to load the saved data from storehouse:
# ./load-from-storehouse.py /var/tmp/kytos/storehouse/kytos.flow.persistence/*
Traceback (most recent call last):
File "//./load-from-storehouse.py", line 20, in <module>
data = _load_from_file(filename).data
File "//./load-from-storehouse.py", line 12, in _load_from_file
data = pickle.load(load_file)
EOFError: Ran out of input
Continuing with the debugging of this issue, I managed to save the flow persistence box using Kytos console and the results are very different:
tmux a -t kytosserver
kytos $> import pickle
kytos $> f = open("/tmp/bla", "wb")
kytos $> pickle.dump(controller.napps[ ('kytos', 'flow_manager')].storehouse.box, f)
kytos $> f.close()
Then, from the command line:
# ls -l /tmp/bla
-rw-r--r-- 1 root root 433062 Feb 4 14:48 /tmp/bla
# ls -l /var/tmp/kytos/storehouse/kytos.flow.persistence/2baf479b304348bea24ae830a3801659
-rw-r--r-- 1 root root 196638 Feb 4 14:24 /var/tmp/kytos/storehouse/kytos.flow.persistence/2baf479b304348bea24ae830a3801659
# /load-from-storehouse.py /tmp/bla > /dev/null
# /load-from-storehouse.py /var/tmp/kytos/storehouse/kytos.flow.persistence/2baf479b304348bea24ae830a3801659 > /dev/null
Traceback (most recent call last):
File "/load-from-storehouse.py", line 20, in <module>
data = _load_from_file(filename).data
File "/load-from-storehouse.py", line 12, in _load_from_file
data = pickle.load(load_file)
EOFError: Ran out of input
Looking into the size of the file, it seems to be a problem in the way flow_manager
is persisting data. I'll open another issue over kytos-ng/flow_manager.
Anyway, kytos-ng/storehouse
should have a way to deal with failures like the one we just observed. One approach could be to backup the current file and create an empty one? These along with an exception handler when loading a file, of course.
For future readers, I'm setting this as wontfix
since this NApp has been deprecated see PR #15
Hi,
I'm not sure the conditions that led to the following error, but after stopping Kytos and starting it again, the following exception was raised:
As a consequence, many stored data was not available after that error, including the one being loaded and the ones that should be loaded right after.
Trying to load some of the stored boxes manually showed the following output:
The scenario in which this error happened was: 1) running Kytos-ng with docker image
amlight/kytos-prod:2022.2rc1
2) Kytos is running with the console enable (kytosd -f) inside a tmux session (default behavior for the docker init script) 3) Kytos is killed with the following command:This issue seems to be quite common in the above sequence of steps. Out of 10 executions, all of them resulted in the same error: