Open TheRealHaoLiu opened 1 year ago
What code is trying to access that directory?
https://github.com/ansible/awx/blob/devel/awx/main/middleware.py#L41
It's used by TimingMiddleware
but that self.dest
seems to be defunct, it calls out to a utils method
https://github.com/ansible/awx/blob/devel/awx/main/utils/profiling.py#L73
I believe that's the functional place we set that location.
Is that the right place for it? We removed /var/log/tower
in https://github.com/ansible/awx/commit/6d11003975948855033605624dfe9f6521c963a9. I'd hate to add it back.
@shanemcd - I get a similar error when enabling debugging via web ui. AWX web responds 'A server error has occurred.'
Log from the web container shows;
PermissionError: [Errno 13] Permission denied: '/var/log/tower'
^ Interesting...
https://github.com/ansible/awx/pull/14211
The error I pasted in that isn't the same as above, but it makes sense that it would be a file-does-not-exist in the awx_devel container, but a Permission denied error in OCP, with both having the same root cause. There was a change some time ago when we had OCP set the environment variable AWX_LOGGING_MODE=stdout. I'm interested to ask if you could check to see if that's absent in your case, and that might explain the problem, which may come from a mismatch of operator/AWX versions.
I've no clue why the app suddenly stopped working but I'm seeing the a similar error in the awx-web pod.
Traceback (most recent call last):
78
File "/var/lib/awx/venv/awx/lib64/python3.9/site-packages/django/core/handlers/exception.py", line 55, in inner
79
response = get_response(request)
80
File "/var/lib/awx/venv/awx/lib64/python3.9/site-packages/django/utils/deprecation.py", line 136, in __call__
81
response = self.process_response(request, response)
82
File "/var/lib/awx/venv/awx/lib64/python3.9/site-packages/awx/main/middleware.py", line 58, in process_response
83
response['X-API-Profile-File'] = self.prof.stop()
84
File "/var/lib/awx/venv/awx/lib64/python3.9/site-packages/awx/main/utils/profiling.py", line 129, in stop
85
res = self.output_results()
86
File "/var/lib/awx/venv/awx/lib64/python3.9/site-packages/awx/main/utils/profiling.py", line 90, in output_results
87
super().output_results()
88
File "/var/lib/awx/venv/awx/lib64/python3.9/site-packages/awx/main/utils/profiling.py", line 22, in output_results
89
os.makedirs(self.dest)
90
File "/usr/lib64/python3.9/os.py", line 215, in makedirs
91
makedirs(head, exist_ok=exist_ok)
92
File "/usr/lib64/python3.9/os.py", line 225, in makedirs
93
mkdir(name, mode)
94
PermissionError: [Errno 13] Permission denied: '/var/log/tower'
95
2023-07-07 17:28:13,533 ERROR [686d5b6a4fa04b478fc50be5ea43977e] django.request Internal Server Error: /
I added AWX_LOGGING_MODE as an environment variable but still see the same error, I've installed the application via the awx operator 2.3.0
I think there is a bug in the Troubleshooting settings page. At least that was the path that I hit and got stuck with the above error.
I edited the page to set "Debug Web requests" as ON but then started seeing the error
I removed the database records for keys AWX_CLEANUP_PATHS, AWX_REQUEST_PROFILE and RECEPTOR_RELEASE_WORK, cleared the web cache and was able to login again.
I think there is a bug in the Troubleshooting settings page. At least that was the path that I hit and got stuck with the above error.
I edited the page to set "Debug Web requests" as ON but then started seeing the error
I removed the database records for keys AWX_CLEANUP_PATHS, AWX_REQUEST_PROFILE and RECEPTOR_RELEASE_WORK, cleared the web cache and was able to login again.
Yes i think its a bug too, we had the same error, reported it here https://github.com/ansible/awx-operator/issues/1485
Just a short help for us non sql literal Run this sql on the awx db:
DELETE FROM conf_setting
WHERE key = 'AWX_CLEANUP_PATHS';
DELETE FROM conf_setting
WHERE key = 'AWX_REQUEST_PROFILE';
DELETE FROM conf_setting
WHERE key = 'RECEPTOR_RELEASE_WORK';
Thanks to @prabhjotbawa for the hint!
@janorn Thanks for the mention, and also sharing the command to delete those entries, I forgot to mention the name of the table :). We could also remove them in a single command:
DELETE from conf_setting where key in ('AWX_CLEANUP_PATHS','AWX_REQUEST_PROFILE','RECEPTOR_RELEASE_WORK');
Please confirm the following
security@ansible.com
instead.)Bug Summary
bash-4.4$ ls /var/log/tower/profile/ ls: cannot access '/var/log/tower/profile/': Permission denied
unable to access profile for debugging
AWX version
devel
Select the relevant components
Installation method
kubernetes
Modifications
no
Ansible version
No response
Operating system
No response
Web browser
No response
Steps to reproduce
exec into container and do
ls /var/log/tower/profile/
Expected results
able to access '/var/log/tower/profile/'
Actual results
unable to access '/var/log/tower/profile/'
Additional information
No response