HicServices / RDMP

Research Data Management Platform (RDMP) is an open source application for the loading,linking,anonymisation and extraction of datasets stored in relational databases.
https://github.com/HicServices/RDMP#research-data-management-platform
GNU General Public License v3.0
37 stars 16 forks source link

unable to decrypt Password stored in YamlRepo #1907

Closed rkm closed 3 weeks ago

rkm commented 3 months ago

Describe the bug

I've not been able to reproduce this in a local environment yet, but when running ImportTableInfo with a YamlRepo which has been setup with an encryption key the following error is produced:

CryptographicException: The length of the data to decrypt is not valid for the size of this key.
  at byte[[]] Decrypt(byte[] rgb, bool fOAEP)                                                                                                                          
  at string Decrypt(string toDecrypt) in SimpleStringValueEncryption.cs:77                                                                                             
  at string GetDecryptedPassword() in EncryptedPasswordHost.cs:83                                                                                                      
  at string GetDecryptedPassword() in DataAccessCredentials.cs:136                                                                                                     
  at bool <GetCredentialByUsernameAndPasswordIfExists>b__0(DataAccessCredentials c) in MemoryCatalogueRepository.cs:280                                               
  at TSource TryGetFirst<TSource>(IEnumerable<TSource> source, Func<TSource, bool> predicate, out bool found)                                                         
  at DataAccessCredentials GetCredentialByUsernameAndPasswordIfExists(string username, string password) in MemoryCatalogueRepository.cs:279                           
  at DataAccessCredentials Create(ITableInfo tableInfoCreated, string username, string password, DataAccessContext usageContext) in DataAccessCredentialsFactory.cs:40
  at void DoImport(out ITableInfo tableInfoCreated, out ColumnInfo[] columnInfosCreated) in TableInfoImporter.cs:158                                                  
  at void Execute() in ExecuteCommandImportTableInfo.cs:57                                                                                                             
  at void ExecuteCommand(ConstructorInfo constructorInfo, CommandLineObjectPicker picker) in CommandInvoker.cs:298                                                    
  at void RunCommand(string command) in ExecuteCommandRunner.cs:104                                                                                                    
  at void RunCommandExecutionLoop(IRDMPPlatformRepositoryServiceLocator repositoryLocator) in ExecuteCommandRunner.cs:142    

To Reproduce

This reproduction cases the failure on an Ubuntu 22.04 VM in the NSH:

#!/usr/bin/env bash

set -euxo pipefail

rdmp=/mnt/nfs_temp/ansible/software/HicServices/RDMP/v8.2.1/rdmp-cli/rdmp
rdmpdir=$(readlink -f test)
rm -rf $rdmpdir key.xml
mkdir $rdmpdir
$rdmp --dir $rdmpdir CreatePrivateKey $(pwd)/key.xml
$rdmp --dir $rdmpdir CreateNewExternalDatabaseServer None 'DatabaseType:PostgreSql:Server=localhost;Uid=rdmp_cli;Pwd=...;Database=smi'

for t in Study Series Image; do
    $rdmp --dir $rdmpdir ImportTableInfo "Table:IO_${t}Table:Schema:dicom:DatabaseType:PostgreSql:Server=localhost;Database=smi;Uid=rdmp_cli;Pwd=..." True
done

echo "ok"

The same reproduction on an external Ubuntu 22.04 VM works fine however.

Expected behavior

The key is valid.

Screenshots

N/A.

RDMP Version

v8.2.1

Error with Stack Trace

See above.

Database Engine

N/A.

Additional context

N/A.

rkm commented 3 months ago

Let me know if there's anything I can do to debug this further.

JFriel commented 3 months ago

@rkm do you have a private key set up on your instance? I've an into issues with long passwords requiring a decryption certificate. You can create on using the command ./rdmp CreatePrivateKey <location to put key> (see https://github.com/HicServices/RDMP/blob/develop/Rdmp.Core/CommandExecution/AtomicCommands/ExecuteCommandCreatePrivateKey.cs)

rkm commented 3 months ago

I'm using a key, yeah :)

$rdmp --dir $rdmpdir CreatePrivateKey $(pwd)/key.xml

JFriel commented 3 months ago

Similarly, am unable to reproduce the issue following the steps provided. Don't have a solution for you right now unfortunately, but leave it with us and we'll get it sorted

tznind commented 3 months ago

Maybe the specific password that you are using ?when you are trying to reproduce on other machine do you use same db password?

That will be what it is trying to encrypt at that point.

On Fri, 26 Jul 2024, 13:37 James Friel, @.***> wrote:

Similarly, am unable to reproduce the issue following the steps provided. Don't have a solution for you right now unfortunately, but leave it with us and we'll get it sorted

— Reply to this email directly, view it on GitHub https://github.com/HicServices/RDMP/issues/1907#issuecomment-2252675323, or unsubscribe https://github.com/notifications/unsubscribe-auth/AHO3C5BTETQKHUYQ3PZHMVTZOI7ILAVCNFSM6AAAAABLQMM5HSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDENJSGY3TKMZSGM . You are receiving this because you are subscribed to this thread.Message ID: @.***>

JFriel commented 3 months ago

Think you're onto something here Thomas, have managed to recreate the issue using a user with certain passwords. Looks like the issue is with the decryption method and the size of byte[] we're trying to decrypt being longer than the key size. Investigating a fix now

rkm commented 3 months ago

Pretty sure I used the same password when trying to reproduce it, though I've just realised the encryption key would be different so it wouldn't have been a like-for-like test. Will have another go at reproducing it next week.

jas88 commented 3 months ago

Hm, that's quite baffling - that exception being thrown at that location means the ephemeral (AES) key was either truncated or had junk on the end, but the overall encrypted block still contained the correct number of $ symbols. Does this happen consistently on the NSH Ubuntu VM but never happen on the non-NSH one?

tznind commented 3 months ago

Is it possible there is \r\n vs \n? Maybe when writing encrypted value it puts newline on?

Shooting in the dark here.

JFriel commented 3 months ago

Looks like the YAML repository was attempting to decrypt the DataAccessCredentials before loading in the encryption key. Small reordering of the imports seems to fix the issue. Should have PR up for this later today