Cannot access file path - Error Uploading to S3

[india081947]

For the version: ERPNext: v14.7.0 Frappe Framework: v14.15.0 Storage Integration: v0.0.1

Have fresh installation for the said version.

Steps to Reproduce:

1. Storage integration Settings for S3
2. Try uploading any file and it will throw below error.

Cannot access file path /api/method/storage_integration.controller.download_from_s3?doc_name=bc16370ed6&local_file_url=/files/test1.png

Traceback (most recent call last): File "apps/frappe/frappe/app.py", line 69, in application response = frappe.api.handle() File "apps/frappe/frappe/api.py", line 54, in handle return frappe.handler.handle() File "apps/frappe/frappe/handler.py", line 45, in handle data = execute_cmd(cmd) File "apps/frappe/frappe/handler.py", line 83, in execute_cmd return frappe.call(method, **frappe.form_dict) File "apps/frappe/frappe/__init__.py", line 1581, in call return fn(*args, **newargs) File "apps/frappe/frappe/handler.py", line 221, in upload_file return frappe.get_doc( File "apps/frappe/frappe/model/document.py", line 303, in save return self._save(*args, **kwargs) File "apps/frappe/frappe/model/document.py", line 325, in _save return self.insert() File "apps/frappe/frappe/model/document.py", line 274, in insert self.run_method("after_insert") File "apps/frappe/frappe/model/document.py", line 908, in run_method out = Document.hook(fn)(self, *args, **kwargs) File "apps/frappe/frappe/model/document.py", line 1260, in composer return composed(self, method, *args, **kwargs) File "apps/frappe/frappe/model/document.py", line 1244, in runner add_to_return_value(self, f(self, method, *args, **kwargs)) File "apps/storage_integration/storage_integration/controller.py", line 126, in upload_to_s3 conn.upload_file() File "apps/storage_integration/storage_integration/controller.py", line 38, in upload_file self.file.save() File "apps/frappe/frappe/model/document.py", line 303, in save return self._save(*args, **kwargs) File "apps/frappe/frappe/model/document.py", line 337, in _save self.run_before_save_methods() File "apps/frappe/frappe/model/document.py", line 1039, in run_before_save_methods self.run_method("validate") File "apps/frappe/frappe/model/document.py", line 908, in run_method out = Document.hook(fn)(self, *args, **kwargs) File "apps/frappe/frappe/model/document.py", line 1260, in composer return composed(self, method, *args, **kwargs) File "apps/frappe/frappe/model/document.py", line 1242, in runner add_to_return_value(self, fn(self, *args, **kwargs)) File "apps/frappe/frappe/model/document.py", line 905, in fn return method_object(*args, **kwargs) File "apps/frappe/frappe/core/doctype/file/file.py", line 92, in validate self.validate_file_on_disk() File "apps/frappe/frappe/core/doctype/file/file.py", line 281, in validate_file_on_disk full_path = self.get_full_path() File "apps/frappe/frappe/core/doctype/file/file.py", line 480, in get_full_path frappe.throw(_("Cannot access file path {0}").format(file_path)) File "apps/frappe/frappe/__init__.py", line 522, in throw msgprint( File "apps/frappe/frappe/__init__.py", line 490, in msgprint _raise_exception() File "apps/frappe/frappe/__init__.py", line 442, in _raise_exception raise raise_exception(msg) frappe.exceptions.ValidationError: Cannot access file path /api/method/storage_integration.controller.download_from_s3?doc_name=bc16370ed6&local_file_url=/files/test1.png

As far as I have checked this is happening at below line: https://github.com/frappe/frappe/blob/6fb27b47a5dd3405396545479bbfdd7e39ed9327/frappe/core/doctype/file/file.py#L479

Due to line no 458 & 459 if "/files/" in file_path and file_path.startswith(site_url): file_path = file_path.split(site_url, 1)[1]

file path gets split, removing the site url and prefix http from the path and hence in later stage is_safe_path(file_path) will fail.

This will not allow saving the File Doc. Though files gets uploaded in S3 successfully.

Problem is file is checking in site url, which is always set from storage integration, and that will spilt the file, also path contains "files" and path separators "/" also making it difficult to handle.

Was thinking of using flags here which can be set from storage application and validation can be skipped based on that.

Dont know how to handle the situation here. Please help.

[zsnmwy]

I have the same question in Frappe Framework: v14.25.1 (version-14). But the repo seems to have no more maintenance.