gramineproject / examples

Sample applications configs for Gramine
BSD 3-Clause "New" or "Revised" License
29 stars 22 forks source link

Code works properly, but an error occurred when I tried to run it in gramine. #61

Closed Kikyou5473 closed 1 year ago

Kikyou5473 commented 1 year ago

This is an error message It seems that the program wants to download the model, but I have already downloaded the model before. Does anyone know how to solve this problem

Downloading: "https://download.pytorch.org/models/vgg19-dcbb9e9d.pth" to /root/.cache/torch/hub/checkpoints/vgg19-dcbb9e9d.pth ./checkpoint/1024/2/epoch_78-best-acc_0.8823530077934265.pth Traceback (most recent call last): File "/usr/lib/python3.8/urllib/request.py", line 1354, in do_open h.request(req.get_method(), req.selector, req.data, headers, File "/usr/lib/python3.8/http/client.py", line 1256, in request self._send_request(method, url, body, headers, encode_chunked) File "/usr/lib/python3.8/http/client.py", line 1302, in _send_request self.endheaders(body, encode_chunked=encode_chunked) File "/usr/lib/python3.8/http/client.py", line 1251, in endheaders self._send_output(message_body, encode_chunked=encode_chunked) File "/usr/lib/python3.8/http/client.py", line 1011, in _send_output self.send(msg) File "/usr/lib/python3.8/http/client.py", line 951, in send self.connect() File "/usr/lib/python3.8/http/client.py", line 1418, in connect super().connect() File "/usr/lib/python3.8/http/client.py", line 922, in connect self.sock = self._create_connection( File "/usr/lib/python3.8/socket.py", line 787, in create_connection for res in getaddrinfo(host, port, 0, SOCK_STREAM): File "/usr/lib/python3.8/socket.py", line 918, in getaddrinfo for res in _socket.getaddrinfo(host, port, family, type, proto, flags): socket.gaierror: [Errno -3] Temporary failure in name resolution

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "./Get_feature.py", line 59, in model = Load_model(opt.model_path) File "./Get_feature.py", line 31, in Load_model model = vgg_model.VGG19() File "/vgg_model.py", line 8, in init self.features = models.vgg19(pretrained=True).features File "/usr/lib/python3/dist-packages/torchvision/models/_utils.py", line 142, in wrapper return fn(*args, *kwargs) File "/usr/lib/python3/dist-packages/torchvision/models/_utils.py", line 228, in inner_wrapper return builder(args, kwargs) File "/usr/lib/python3/dist-packages/torchvision/models/vgg.py", line 485, in vgg19 return _vgg("E", False, weights, progress, kwargs) File "/usr/lib/python3/dist-packages/torchvision/models/vgg.py", line 105, in _vgg model.load_state_dict(weights.get_state_dict(progress=progress)) File "/usr/lib/python3/dist-packages/torchvision/models/_api.py", line 89, in get_state_dict return load_state_dict_from_url(self.url, progress=progress) File "/usr/lib/python3/dist-packages/torch/hub.py", line 746, in load_state_dict_from_url download_url_to_file(url, cached_file, hash_prefix, progress=progress) File "/usr/lib/python3/dist-packages/torch/hub.py", line 611, in download_url_to_file u = urlopen(req) File "/usr/lib/python3.8/urllib/request.py", line 222, in urlopen return opener.open(url, data, timeout) File "/usr/lib/python3.8/urllib/request.py", line 525, in open response = self._open(req, data) File "/usr/lib/python3.8/urllib/request.py", line 542, in _open result = self._call_chain(self.handle_open, protocol, protocol + File "/usr/lib/python3.8/urllib/request.py", line 502, in _call_chain result = func(*args) File "/usr/lib/python3.8/urllib/request.py", line 1397, in https_open return self.do_open(http.client.HTTPSConnection, req, File "/usr/lib/python3.8/urllib/request.py", line 1357, in do_open raise URLError(err) urllib.error.URLError: <urlopen error [Errno -3] Temporary failure in name resolution> Error in sys.excepthook: Traceback (most recent call last): File "/usr/lib/python3/dist-packages/apport_python_hook.py", line 72, in apport_excepthook from apport.fileutils import likely_packaged, get_recent_crashes File "/usr/lib/python3/dist-packages/apport/init.py", line 5, in from apport.report import Report File "/usr/lib/python3/dist-packages/apport/report.py", line 32, in import apport.fileutils File "/usr/lib/python3/dist-packages/apport/fileutils.py", line 27, in from apport.packaging_impl import impl as packaging File "/usr/lib/python3/dist-packages/apport/packaging_impl.py", line 23, in import apt File "/usr/lib/python3/dist-packages/apt/init.py", line 36, in apt_pkg.init_system() apt_pkg.Error: E:Error reading the CPU table

Original exception was: Traceback (most recent call last): File "/usr/lib/python3.8/urllib/request.py", line 1354, in do_open h.request(req.get_method(), req.selector, req.data, headers, File "/usr/lib/python3.8/http/client.py", line 1256, in request self._send_request(method, url, body, headers, encode_chunked) File "/usr/lib/python3.8/http/client.py", line 1302, in _send_request self.endheaders(body, encode_chunked=encode_chunked) File "/usr/lib/python3.8/http/client.py", line 1251, in endheaders self._send_output(message_body, encode_chunked=encode_chunked) File "/usr/lib/python3.8/http/client.py", line 1011, in _send_output self.send(msg) File "/usr/lib/python3.8/http/client.py", line 951, in send self.connect() File "/usr/lib/python3.8/http/client.py", line 1418, in connect super().connect() File "/usr/lib/python3.8/http/client.py", line 922, in connect self.sock = self._create_connection( File "/usr/lib/python3.8/socket.py", line 787, in create_connection for res in getaddrinfo(host, port, 0, SOCK_STREAM): File "/usr/lib/python3.8/socket.py", line 918, in getaddrinfo for res in _socket.getaddrinfo(host, port, family, type, proto, flags): socket.gaierror: [Errno -3] Temporary failure in name resolution

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "./Get_feature.py", line 59, in model = Load_model(opt.model_path) File "./Get_feature.py", line 31, in Load_model model = vgg_model.VGG19() File "/vgg_model.py", line 8, in init self.features = models.vgg19(pretrained=True).features File "/usr/lib/python3/dist-packages/torchvision/models/_utils.py", line 142, in wrapper return fn(*args, *kwargs) File "/usr/lib/python3/dist-packages/torchvision/models/_utils.py", line 228, in inner_wrapper return builder(args, kwargs) File "/usr/lib/python3/dist-packages/torchvision/models/vgg.py", line 485, in vgg19 return _vgg("E", False, weights, progress, kwargs) File "/usr/lib/python3/dist-packages/torchvision/models/vgg.py", line 105, in _vgg model.load_state_dict(weights.get_state_dict(progress=progress)) File "/usr/lib/python3/dist-packages/torchvision/models/_api.py", line 89, in get_state_dict return load_state_dict_from_url(self.url, progress=progress) File "/usr/lib/python3/dist-packages/torch/hub.py", line 746, in load_state_dict_from_url download_url_to_file(url, cached_file, hash_prefix, progress=progress) File "/usr/lib/python3/dist-packages/torch/hub.py", line 611, in download_url_to_file u = urlopen(req) File "/usr/lib/python3.8/urllib/request.py", line 222, in urlopen return opener.open(url, data, timeout) File "/usr/lib/python3.8/urllib/request.py", line 525, in open response = self._open(req, data) File "/usr/lib/python3.8/urllib/request.py", line 542, in _open result = self._call_chain(self.handle_open, protocol, protocol + File "/usr/lib/python3.8/urllib/request.py", line 502, in _call_chain result = func(*args) File "/usr/lib/python3.8/urllib/request.py", line 1397, in https_open return self.do_open(http.client.HTTPSConnection, req, File "/usr/lib/python3.8/urllib/request.py", line 1357, in do_open raise URLError(err) urllib.error.URLError: <urlopen error [Errno -3] Temporary failure in name resolution>

dimakuv commented 1 year ago

It seems that the program wants to download the model, but I have already downloaded the model before.

Looks like the model is downloaded under /root/.cache/torch/hub/checkpoints/vgg19-dcbb9e9d.pth. Are you sure you specified this path/file in the Gramine manifest file? Probably Gramine simply doesn't "know" about this directory/file because you didn't expose them in the manifest.

socket.gaierror: [Errno -3] Temporary failure in name resolution

This error probably happens because you didn't expose the DNS logic in Gramine manifest. Please add it as described here: https://gramine.readthedocs.io/en/stable/manifest-syntax.html#domain-names-configuration

Kikyou5473 commented 1 year ago

i add "/root/.cache/torch/hub/checkpoints/vgg19-dcbb9e9d.pth" in trusted_files and [sys] enable_extra_runtime_domain_names_conf = true in manifest

I meet a new error.It seems to be a certificate authentication issue. Should I add any more information to the manifest

Downloading: "https://download.pytorch.org/models/vgg19-dcbb9e9d.pth" to /root/.cache/torch/hub/checkpoints/vgg19-dcbb9e9d.pth ./checkpoint/1024/2/epoch_78-best-acc_0.8823530077934265.pth Traceback (most recent call last): File "/usr/lib/python3.8/urllib/request.py", line 1354, in do_open h.request(req.get_method(), req.selector, req.data, headers, File "/usr/lib/python3.8/http/client.py", line 1256, in request self._send_request(method, url, body, headers, encode_chunked) File "/usr/lib/python3.8/http/client.py", line 1302, in _send_request self.endheaders(body, encode_chunked=encode_chunked) File "/usr/lib/python3.8/http/client.py", line 1251, in endheaders self._send_output(message_body, encode_chunked=encode_chunked) File "/usr/lib/python3.8/http/client.py", line 1011, in _send_output self.send(msg) File "/usr/lib/python3.8/http/client.py", line 951, in send self.connect() File "/usr/lib/python3.8/http/client.py", line 1425, in connect self.sock = self._context.wrap_socket(self.sock, File "/usr/lib/python3.8/ssl.py", line 500, in wrap_socket return self.sslsocket_class._create( File "/usr/lib/python3.8/ssl.py", line 1040, in _create self.do_handshake() File "/usr/lib/python3.8/ssl.py", line 1309, in do_handshake self._sslobj.do_handshake() ssl.SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:1131)

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "./Get_feature.py", line 59, in model = Load_model(opt.model_path) File "./Get_feature.py", line 31, in Load_model model = vgg_model.VGG19() File "/vgg_model.py", line 8, in init self.features = models.vgg19(pretrained=True).features File "/usr/lib/python3/dist-packages/torchvision/models/_utils.py", line 142, in wrapper return fn(*args, *kwargs) File "/usr/lib/python3/dist-packages/torchvision/models/_utils.py", line 228, in inner_wrapper return builder(args, kwargs) File "/usr/lib/python3/dist-packages/torchvision/models/vgg.py", line 485, in vgg19 return _vgg("E", False, weights, progress, kwargs) File "/usr/lib/python3/dist-packages/torchvision/models/vgg.py", line 105, in _vgg model.load_state_dict(weights.get_state_dict(progress=progress)) File "/usr/lib/python3/dist-packages/torchvision/models/_api.py", line 89, in get_state_dict return load_state_dict_from_url(self.url, progress=progress) File "/usr/lib/python3/dist-packages/torch/hub.py", line 746, in load_state_dict_from_url download_url_to_file(url, cached_file, hash_prefix, progress=progress) File "/usr/lib/python3/dist-packages/torch/hub.py", line 611, in download_url_to_file u = urlopen(req) File "/usr/lib/python3.8/urllib/request.py", line 222, in urlopen return opener.open(url, data, timeout) File "/usr/lib/python3.8/urllib/request.py", line 525, in open response = self._open(req, data) File "/usr/lib/python3.8/urllib/request.py", line 542, in _open result = self._call_chain(self.handle_open, protocol, protocol + File "/usr/lib/python3.8/urllib/request.py", line 502, in _call_chain result = func(*args) File "/usr/lib/python3.8/urllib/request.py", line 1397, in https_open return self.do_open(http.client.HTTPSConnection, req, File "/usr/lib/python3.8/urllib/request.py", line 1357, in do_open raise URLError(err) urllib.error.URLError: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:1131)> Error in sys.excepthook: Traceback (most recent call last): File "/usr/lib/python3/dist-packages/apport_python_hook.py", line 72, in apport_excepthook from apport.fileutils import likely_packaged, get_recent_crashes File "/usr/lib/python3/dist-packages/apport/init.py", line 5, in from apport.report import Report File "/usr/lib/python3/dist-packages/apport/report.py", line 32, in import apport.fileutils File "/usr/lib/python3/dist-packages/apport/fileutils.py", line 27, in from apport.packaging_impl import impl as packaging File "/usr/lib/python3/dist-packages/apport/packaging_impl.py", line 23, in import apt File "/usr/lib/python3/dist-packages/apt/init.py", line 36, in apt_pkg.init_system() apt_pkg.Error: E:Error reading the CPU table

Original exception was: Traceback (most recent call last): File "/usr/lib/python3.8/urllib/request.py", line 1354, in do_open h.request(req.get_method(), req.selector, req.data, headers, File "/usr/lib/python3.8/http/client.py", line 1256, in request self._send_request(method, url, body, headers, encode_chunked) File "/usr/lib/python3.8/http/client.py", line 1302, in _send_request self.endheaders(body, encode_chunked=encode_chunked) File "/usr/lib/python3.8/http/client.py", line 1251, in endheaders self._send_output(message_body, encode_chunked=encode_chunked) File "/usr/lib/python3.8/http/client.py", line 1011, in _send_output self.send(msg) File "/usr/lib/python3.8/http/client.py", line 951, in send self.connect() File "/usr/lib/python3.8/http/client.py", line 1425, in connect self.sock = self._context.wrap_socket(self.sock, File "/usr/lib/python3.8/ssl.py", line 500, in wrap_socket return self.sslsocket_class._create( File "/usr/lib/python3.8/ssl.py", line 1040, in _create self.do_handshake() File "/usr/lib/python3.8/ssl.py", line 1309, in do_handshake self._sslobj.do_handshake() ssl.SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:1131)

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "./Get_feature.py", line 59, in model = Load_model(opt.model_path) File "./Get_feature.py", line 31, in Load_model model = vgg_model.VGG19() File "/vgg_model.py", line 8, in init self.features = models.vgg19(pretrained=True).features File "/usr/lib/python3/dist-packages/torchvision/models/_utils.py", line 142, in wrapper return fn(*args, *kwargs) File "/usr/lib/python3/dist-packages/torchvision/models/_utils.py", line 228, in inner_wrapper return builder(args, kwargs) File "/usr/lib/python3/dist-packages/torchvision/models/vgg.py", line 485, in vgg19 return _vgg("E", False, weights, progress, kwargs) File "/usr/lib/python3/dist-packages/torchvision/models/vgg.py", line 105, in _vgg model.load_state_dict(weights.get_state_dict(progress=progress)) File "/usr/lib/python3/dist-packages/torchvision/models/_api.py", line 89, in get_state_dict return load_state_dict_from_url(self.url, progress=progress) File "/usr/lib/python3/dist-packages/torch/hub.py", line 746, in load_state_dict_from_url download_url_to_file(url, cached_file, hash_prefix, progress=progress) File "/usr/lib/python3/dist-packages/torch/hub.py", line 611, in download_url_to_file u = urlopen(req) File "/usr/lib/python3.8/urllib/request.py", line 222, in urlopen return opener.open(url, data, timeout) File "/usr/lib/python3.8/urllib/request.py", line 525, in open response = self._open(req, data) File "/usr/lib/python3.8/urllib/request.py", line 542, in _open result = self._call_chain(self.handle_open, protocol, protocol + File "/usr/lib/python3.8/urllib/request.py", line 502, in _call_chain result = func(*args) File "/usr/lib/python3.8/urllib/request.py", line 1397, in https_open return self.do_open(http.client.HTTPSConnection, req, File "/usr/lib/python3.8/urllib/request.py", line 1357, in do_open raise URLError(err) urllib.error.URLError: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:1131)>

dimakuv commented 1 year ago

It looks like you'll need to also expose the root certificates directory to Gramine. You can do loader.log_level = "trace" and see what your application tries to do. It will probably try to open some system-wide directory with certificates that is not mentioned in your manifest file.

But also -- if you already downloaded the model, then your application shouldn't even try to download it once more. Which means that you won't need all the stuff I mentioned above. You just need to understand what is missing from your manifest file that makes your app think that there is no model available on your file system and to download it again.

Kikyou5473 commented 1 year ago

yes, i add "/root/.cache/torch/hub/checkpoints/vgg19-dcbb9e9d.pth" in [fs]mount,trusted_files and allowed_filee.then it work. thank you ! thank you very much !