canonical / seldonio-rocks

ROCKs for Seldon Core
Apache License 2.0
0 stars 1 forks source link

Kf 2271 feat tensorflow serving rock #32

Closed i-chvets closed 1 year ago

i-chvets commented 1 year ago

Design and implementation details are in https://github.com/canonical/seldon-core-operator/issues/133

Summary of changes:

NOTE: This is heavy ROCK. It builds in around 2 hours on 32 core machine.

Manual testing shows that all artefacts are in place:

$ docker run tensorflow-serving:2.13.0_20.04_1 exec pebble exec bash -c 'tensorflow_model_server --port=8500 --rest_api_port=8501 --model_name=${MODEL_NAME} --model_base_path=${MODEL_BASE_PATH}/${MODEL_NAME} "$@"'
2023-07-26T13:31:39.690Z [pebble] Started daemon.
2023-07-26T13:31:39.699Z [pebble] POST /v1/exec 8.818296ms 202
2023-07-26T13:31:39.712Z [pebble] GET /v1/tasks/1/websocket/control 12.437858ms 200
2023-07-26T13:31:39.713Z [pebble] GET /v1/tasks/1/websocket/stdio 121.315µs 200
2023-07-26T13:31:39.713Z [pebble] GET /v1/tasks/1/websocket/stderr 52.198µs 200
2023-07-26T13:31:39.727Z [pebble] POST /v1/exec 8.69893ms 202
2023-07-26T13:31:39.735Z [pebble] GET /v1/tasks/2/websocket/control 8.076374ms 200
2023-07-26T13:31:39.736Z [pebble] GET /v1/tasks/2/websocket/stdio 56.035µs 200
2023-07-26T13:31:39.736Z [pebble] GET /v1/tasks/2/websocket/stderr 54.31µs 200
2023-07-26 13:31:39.801036: I external/org_tensorflow/tensorflow/core/util/port.cc:110] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.
2023-07-26 13:31:39.802675: I tensorflow_serving/model_servers/server.cc:74] Building single TensorFlow model file config:  model_name:  model_base_path: /
2023-07-26 13:31:39.802808: I tensorflow_serving/model_servers/server_core.cc:465] Adding/updating models.
2023-07-26 13:31:39.802814: I tensorflow_serving/model_servers/server_core.cc:594]  (Re-)adding model: 
i-chvets commented 1 year ago

Fails to be started by workload in integration tests:

 microk8s.kubectl -n test-seldon-servers-nsoi logs tfserving-default-0-classifier-7ddd9bd9dc-twk67
Defaulted container "classifier" out of: classifier, seldon-container-engine, classifier-model-initializer (init)
error: unknown flag `port'

Port is part of config for tensorflow/seving. Needs more debugging.

i-chvets commented 1 year ago

@SamirPS Please review. Integration tests will be addressed in another work item that is planned next. For now just a review of the ROCK itself.

i-chvets commented 1 year ago

@SamirPS Installing python3.8 via packages worked fine. Ready for review.