Open harsh288 opened 3 years ago
Hello @kenhys Sorry for replying late, I thought this is a dead forum, wasn't expecting any reply over here, the environment is the last option to me but still, it is a security threat.
Also one more issue I faced recently was if you specify a host like hostName/sqlInstanceName it doesn't work, in one server there is a possibility of having more than one server instances.
@repeatedly @kenhys @frsyuki @cosmo0920 @ashie @ganmacs
@kenhys
Can we write some custom encryption logic inside fluent-plugin-sql for the password where the password will be in an encrypted format in the config file and the custom logic will decrypt first and then it will pass it to the SQL DB.
Hello @repeatedly @kenhys @frsyuki @cosmo0920 @ashie @ganmacs
Can you please help, this is such a big security loophole.
@kenhys Do we have any working solution? This issue has been escalated so need some workaround ASAP.
Generally speaking, encrypting the password here doesn't make sense. You should set the permission of reading the config file only for the file owner. Even when encryption is implemented, if an attacker can read the file, the attacker will be also able to decrypt the password same way with the fluentd process. This is the why no one reply to your question.
Writing plain text credentials in the config file isn't fluent-plugin-sql specific, most application use same way such as Rails applications.
If you have a special reason that you are able to make the encryption meaningful, please describe it. For example, if entering password manually is allowed on every starting your Fluentd process, encrypting the credentials make sense.
@ashie Yes we do have to enter the password manually at every start of the Fluentd process, more than that, for us, no matter if it is a system password, DB password, or any other passwords, it can't be in plain text, that is our company security policy, we don't care about attackers right now or any other scenario you described, it just that it has to be encrypted, so I request you to please help me with a possible solution as soon as possible.
Rails 5.2 or later supports encrypted credentials by ActiveSupport::EncryptedConfiguration
.
(A master key is required even though using it.)
We might be able to introduce similar feature if its really required by many users.
@ashie Request you to please introduce that.
I was under the impression this feature would be available, I started browsing for some syntax to enable encryption and unfortunately I didn't find anything so when I checked the code it was completely missing.
I upvote for this feature, can we have in next version soon?
I think we can use "config/credentials.yml.enc" of Rails for encrypted password, though looking for a solution, elastic cache and other plugins have the same feature but not for SQL and Oracle plugin.
Hello @ashie Could you please share details, will this feature be implemented? if yes, then what will be the deadline? We have just one week left and most of the testing needs to be completed mid-week so any help on this will be of great help.
We don't have actual plan for it yet. Although I'm positive for implementing this feature than before, not yet have higher priority. In addition, if we decide to implement it, we'll add this feature to Fluentd core, not only for this plugin.
BTW I think your requirement can be realized without modifying both this plugin and Fluentd core. For example:
#!/usr/bin/env ruby
require 'openssl' require 'fileutils'
class MyCredential PATH = ENV["MY_CREDENTIAL_PATH"] || "/etc/fluentd/credential.txt" ENCRYPTED_PATH = PATH + ".enc" KEY = ENV["MY_CREDENTIAL_KEY"] SALT = ENV["MY_CREDENTIAL_SALT"] || "a9bab8f6-5db7-4693-81e8-93951d2c2468"
def self.encrypt data = File.read(PATH)
enc = OpenSSL::Cipher.new("AES-256-CBC")
enc.encrypt
key_iv = OpenSSL::PKCS5.pbkdf2_hmac_sha1(KEY, SALT, 2000,
enc.key_len + enc.iv_len)
enc.key = key_iv[0, enc.key_len]
enc.iv = key_iv[enc.key_len, enc.iv_len]
encrypted_data = ""
encrypted_data << enc.update(data)
encrypted_data << enc.final
encrypted_data
end
def self.decrypt(field=nil) data = File.read(ENCRYPTED_PATH)
dec = OpenSSL::Cipher.new("AES-256-CBC")
dec.decrypt
key_iv = OpenSSL::PKCS5.pbkdf2_hmac_sha1(KEY, SALT, 2000,
dec.key_len + dec.iv_len)
dec.key = key_iv[0, dec.key_len]
dec.iv = key_iv[dec.key_len, dec.iv_len]
decrypted_data = ""
decrypted_data << dec.update(data)
decrypted_data << dec.final
if field == :user
decrypted_data.split(/\R/)[0]
elsif field == :pass
decrypted_data.split(/\R/)[1]
else
decrypted_data
end
end
def self.encrypt_file data = encrypt File.open(ENCRYPTED_PATH, "wb") do |f| f.write(data) end FileUtils.chmod(0660, ENCRYPTED_PATH) FileUtils.rm_f(PATH) end
def self.decrypt_file data = decrypt File.open(PATH, "wb") do |f| f.write(data) end FileUtils.chmod(0660, PATH) FileUtils.rm_f(ENCRYPTED_PATH) end end
if $0 == FILE case ARGV[0] when "encrypt" MyCredential.encrypt_file when "decrypt" MyCredential.decrypt_file else puts "Unknown command: #{ARGV[0]}" end end
2. Prepare /etc/fluentd/credential.txt like this
myuser mypassword
3. Encrypt the credential.txt (/etc/fluentd/credential.txt.enc will be created and /etc/fluentd/credential.txt will be deleted):
$ MY_CREDENTIAL_KEY=mykey ruby /etc/fluentd/mycredential.rb encrypt
4. Prepare fluent.conf like this:
username "#{require('/etc/fluentd/mycredential'); MyCredential.decrypt(:user)}" password "#{MyCredential.decrypt(:pass)}" ...
5. Run fluentd with the environment variable `MY_CREDENTIAL_KEY=mykey`
@ashie Thanks a lot for taking efforts and providing a solution, that worked for me.
Hello @ashie
username "#{require('C:/opt/td-agent/etc/td-agent/mycredential'); MyCredential.decrypt(:user)}" password "#{MyCredential.decrypt(:pass)}"
I have the above configuration in the .conf file
When I run td-agent from the command prompt it works however when I run from Windows service, it doesn't work, any idea?? It is not even generating logs in td-agent.log file
Although I don't try it by myself, if storing the master key to registry is acceptable for you, it seems that you can set environment variables for Fluentd service by registry.
HKLM\SYSTEM\CurrentControlSet\Services\flunentdwinsvc\Environment
with REG_MULTI_SZ
typeMY_CREDENTIAL_KEY=mykey
as the value of itHi @ashie I missed to update, the above issue was resolved, ruby was expecting the path in forward slashes instead of backward, as if now I have kept the key in a static format, and I'm just running the command once "ruby /etc/fluentd/mycredential.rb encrypt" from Powershell in our custom build setup.
Thanks for the suggestion for storing in registry however we are providing our custom setup to the customer where I don't think we can edit permission, even though we save in registry how to get it either in fluentd or ruby .rb file
Hello @ashie
We are going to use the latest version of td-agent & fluentd, so are you planning to add this feature?
How to encrypt the password, because of security reasons we can't keep in a text file