Add test files/branch name encryption

[ArturT]

about the feature

Allow user to enable encryption of sensitive data like:

• test file paths
• branch names

why we need this feature?

Some of the customers requested encryption for Jest.

idea how to implement it

Let's keep it simple.

Here is some info about the existing encryption feature for Ruby which might be too complex. We could do it simpler for Knapsack Pro in JavaScript (feature should be implemented in @knapsack-pro/core npm package so other packages like @knapsack-pro/jest or @knapsack-pro/cypress could use it).

In knapsack_pro ruby gem there is an encryption feature but it is a bit overcomplicated. It allows encrypting separately test file paths and branch names. It does not allow decryption so that's a downside for the development and performance of some of the features like RSpec split by examples because there is no simple way to decrypt test files on CI without encrypting all test files first to get SHA2 key (SHA2 value is used to compare SHA2 values from API response to identify test file paths).

In case of @knapsack-pro/core library I would recommend:

• use some JS library (TODO: research what library would be a good fit) that allows to encrypt and decrypt string based on a key. If env var likeKNAPSACK_PRO_ENCRYPTION_KEY has value then it could mean encryption is enabled and the key will be used to encrypt and decrypt string like test file path & branch names.
• There are some branch names that are exceptions and they should not be encrypted because Knapsack Pro API uses them to better split tests in some edge cases situation. Here is list of branch names that should never be encrypted.

story

https://trello.com/c/f494kFmB

[ArturT]

Here are some tips on how to implement this feature. It might be helpful if someone would like to help us develop this feature.

• First, it's good to understand how Queue Mode works in Knapsack Pro. https://docs.knapsackpro.com/2020/how-to-speed-up-ruby-and-javascript-tests-with-ci-parallelisation

• Then it's good to understand the general architecture of @knapsack-pro/core library and how other clients for Jest (@knapsack-pro/jest) or Cypress (@knapsack-pro/cypress) uses it. This article explains it: https://docs.knapsackpro.com/2020/how-to-build-native-integration-with-knapsack-pro-api-to-run-tests-in-parallel-for-any-test-runner-testing-framework

• You can setup @knapsack-pro/core in development https://github.com/KnapsackPro/knapsack-pro-core-js#development

• You can setup @knapsack-pro/jest in development https://github.com/KnapsackPro/knapsack-pro-jest#development

• You can setup jest-example-test-suite project in development https://github.com/KnapsackPro/jest-example-test-suite

  • It contains a few example test files. We use it to test the above packages in development. Thanks to that when you make code change in development for @knapsack-pro/core or @knapsack-pro/jest you can run the script bin/knapsack_pro_jest. The script connects with Knapsack Pro Staging API. Sometimes you need to run it twice to get our staging API up and running (the staging API can get turn off when no requests to it).
  • There are more examples of bin scripts. You could create a new one bin/knapsack_pro_jest_encryption to test the encryption feature :)

Here are a few tips on what places in code @knapsack-pro/core would need to have encryption and decryption added.

Encryption:

• allTestFiles should be encrypted before we send a list of test files existing on the disk to the API. We could do something like knapsackProCipher.encryptArray(allTestFiles). It would be good to have a separate service/class like KnapsackProCipher which does encrypt/decrypt string to separate encryption logic from API logic.
  • https://github.com/KnapsackPro/knapsack-pro-core-js/blob/0f44c6a3daa369cd4353e315abbf5539295289ea/src/knapsack-pro-api.ts#L39
  • we should also encrypt recorded test files before sending them to API https://github.com/KnapsackPro/knapsack-pro-core-js/blob/0f44c6a3daa369cd4353e315abbf5539295289ea/src/knapsack-pro-api.ts#L53
• branch should be encrypted before sending to API
  • https://github.com/KnapsackPro/knapsack-pro-core-js/blob/0f44c6a3daa369cd4353e315abbf5539295289ea/src/knapsack-pro-api.ts#L35
  • https://github.com/KnapsackPro/knapsack-pro-core-js/blob/0f44c6a3daa369cd4353e315abbf5539295289ea/src/knapsack-pro-api.ts#L50

Decryption:

• queueTestFiles will be an encrypted list of test files returned by Knapsack Pro API. You should decrypt it. You could call service/class to decrypt it const queueTestFiles = knapsackProCipher.decryptArray(response.data.test_files); https://github.com/KnapsackPro/knapsack-pro-core-js/blob/0f44c6a3daa369cd4353e315abbf5539295289ea/src/knapsack-pro-core.ts#L58

It's up to you if encrypt/decrypt function takes array or string. Feel free to figure out what would be a good enough way:

• you could have knapsackProCipher.decryptArray(response.data.test_files)
• or you could have knapsackProCipher.decrypt(test_file) and you would need to run map on response.data.test_files to decrypt each string.

summary

If you would like to help us with this feature feel free to do it. I can answer any questions and help later one with the documentation, code review, and final testing and releasing new versions of all libraries like @knapsack-pro/core, @knapsack-pro/jest, @knapsack-pro/cypress.

Please add me rights to push to your PR code so I can help :)