search

databricks-industry-solutions / security-analysis-tool

Security Analysis Tool (SAT) analyzes customer's Databricks account and workspace security configurations and provides recommendations that help them follow Databrick's security best practices. When a customer runs SAT, it will compare their workspace configurations against a set of security best practices and delivers a report.
Other
74 stars 36 forks source link

PermissionError: [Errno 13] Permission denied: '/var/log/dbrprofiler.log' #105

Closed saravankrish closed 1 month ago

saravankrish commented 2 months ago

Hi,

I would like to express my gratitude to you for developing and maintaining this exceptional tool. As a data engineering team, we often struggle to identify security vulnerabilities in our infrastructure setup. Your tool has been instrumental in pinpointing these issues accurately.

Regarding the setup of the SAT tool in our Databricks environment on Azure, I followed the documentation meticulously. I created a service principal (SP) and granted it contributor permissions at the subscription level. Additionally, I added the SP to Databricks and assigned it admin privileges for the workspace. However, when I run the SAT Initializer Notebook, I encounter the following permission error:

PermissionError: [Errno 13] Permission denied: '/var/log/dbrprofiler.log'

Can you please help me by pointing out what permission this SP might be missing? Also, It appears that the log file location is hardcoded in the core Python module. Is there a way to customize this path? If so, could you please guide me on how to do so?

Thank you in advance for your assistance.

Best regards, Saravanan

arunpamulapati commented 2 months ago

Sorry for the delayed response. Thank you for your kind words.

Are you running this on a High concurrency or single-user cluster? Did someone vi or open /var/log/dbrprofiler.log? this file ? Can you try removing the file by running this in a notebook cell :

%sh rm -rf /var/log/dbrprofiler.log

and retry to run. We are here to help you. Please reach out to us if you need further assistance.

saravankrish commented 2 months ago

Hello @arunpamulapati,

Thank you for your timely response. I attempted to delete the log file, but unfortunately, I continue to encounter the same error:

rm: cannot remove '/var/log/dbrprofiler.log': Permission denied

Please advise on how to proceed.

Also, I am running a Multi-node cluster and I didn't try to modify the log file.

arunpamulapati commented 2 months ago

Saravanan,

Did you use a different cluster first and then change to a new cluster to run SAT? If so, you need to be on the original cluster and delete the logs file. Looks like the log file was created by that cluster user and now the new cluster user is unable to delete/use that file due to the permission lock on the log file.

saravankrish commented 2 months ago

@arunpamulapati,

I noticed that the log file location is currently hardcoded in the logging_utils class of the core Python module. Is there a way to make it customizable? It would be beneficial if we could specify a different location for the log files.

arunpamulapati commented 2 months ago

Not at this time, as this is an internal log and hasn't been an issue so far, as usually, this should not happen if the right cluster is used and the subsequent runs are done with the same cluster. We will consider making this configurable in future releases.

Is your issue fixed now?

saravankrish commented 2 months ago

@arunpamulapati, This is the first cluster I am running on this workspace and I still see the same issue.

arunpamulapati commented 2 months ago

Hi Saravanan,

Are you using a single-user cluster or a shared cluster? The SAT requirement is to use a single-user cluster. It's likely you don't have access to /var/log on your cluster. Please use a single-user cluster as stated here. https://github.com/databricks-industry-solutions/security-analysis-tool/blob/main/docs/setup.md If the problem persists. Send an email to ramdas.murali@databricks.com and arun@databricks.com with days and times best for you for the next few days and one of us will help you.

saravankrish commented 2 months ago

Hi @arunpamulapati,

I have sent an email to the specified email addresses. Please review the proposed meeting times, and if they don’t work for you, kindly let me know your availability.

Thank you Saravanan

arunpamulapati commented 1 month ago

The cluster issue was resolved.