HTTP response info saved in the HTTP file after each execution

[alekdavisintel]

I'm new here, so I may be doing something silly, but I just noticed that every time I run the test, HTTP response (version, status code, headers, but not the body) gets saved in the HTTP file. I did not realize until I opened the file in Notepad++. I reloaded the file after each execution and the timestamp was updated after each run. Also, on the first run, there is response from the OAuth 2.0 call, but on the subsequent calls it gets removed. Not really sure whay the responses get saved, but is see nothing but trouble with this feature (it would mean that the after each run Git would see tha the file was updated, even though no changes were made to the request or scripts. Anyway, is there a way to turn this off?

[AnWeber]

Do you use httpbook? Notebooks support this feature, to provide the same View after closing and reopening a http file. I am not a fan of the feature myself, so it is deactivable.

settings.json

httpbook.saveWithOutputs: false

I do not use notebooks in general. Notebooks are not working for my work style. The missing response seems to be a bug.

[alekdavisintel]

Ah, I didn't realize httpbook was optional. I uninstalled it and I like it without httpbook a lot more (for starters, I prefer to see the text version of the file, not what httpbook shows). Also dicovered that you can run tests by folders from the TESTING window. THIS IS AWESOME!!! And the results look more compact and usable. I'm so excited. After trying a half dozen alternatives, I have a slimmer of hope that it will work. Our use case is pretty complex (https://docs.google.com/document/d/10Yvd_X-2j8JpdPymqd8mW9y5sG0oMuxyoRf0zGzRj2I) but from what I've seen so far, it looks like it should be possible. Thank you so much for such amazing work. And you will most likely hear from me again (I apologize in advance).

[AnWeber]

I took the liberty of reading through your requirements. A few comments

The system must provide a capability to import data (collections and environments) from the common applications (Postman, Insomnia+).

There is no import script at the moment. Intellij Idea supports convertierung Postman collection to http files with Intellij Style scripts. Should be compatible, but without guarantee. I solved it by writing a 200 line script myself.

It would be nice to be capable of generating tests from the OpenAPI data (Swagger).++ I'm on board with the feature, but I don't have the time right now.

Allow saving and omitting sensitive authorization settings (passwords, client secrets, tokens, keys) in exported files.

Secrets are transferred during execution. I have no idea how and where, i.e. I can't clean them up properly. For this reason, the user himself is responsible for ensuring his needs in the loggingRepsonseHook.

Secrets

I don't currently have anything in the way of secret management on purpose. I treat secrets similarly to other variables. I don't store any variables on the hard disk, but keep them in memory if necessary. Variables that have been added in requests must be removed independently using loggingResponseHook.

Integration with secret storage facility (Azure Key Vault, PAM, etc) would be nice (if not implicitly, could be achieved programmatically, via a script)

I once had a connection to Consul Vault, but left the code behind at the last company. In other words, it would work.

Test scripts should be able to use common frameworks, such as chai.js, etc.

No ESM support. chai.js@5.x uses ESM.

[alekdavisintel]

The import feature would be important for the Postman/Insomnia-like apps, which is not the case here, so I would not worry about it. I actually like the httpYac architecture (file based, .http/.rest standard) better than Postman's/Insomnia's, so I would not mind the time and effort spent on converting. No idea what IntelliJ is, so cannot appreaciate the compatibility at this time, but I will take a look.

I don't know what ESM is (I'm sorry, but all things JavaScript are not my specialty, I am a complete novice here), but it should not matter. My point is that the tool should be able to consume some sort of assertion framework and other libraries required for running tests and performing validation (e.g. generating timestamps, guids, checking returned results against expectations, making HTTP calls). As long as there is a way to do this using whatever, it is good. The built-in option is great, but in case it lacks a particular feature, there should be a way to consume either a public or a custom library. Does not need to be chai.js, as long as there is an alternative (although, chai.js is sweet).

Secret handling is important. I probably did not explain it well, so let me re-try.

First, don't care about ad-hoc test secrets. I'm talking about test that are used by the team and files being checked into source control. The thing we want to prevent is checking in a file with secrets because once you push the commit (intentionally or not), you pretty much need to wipe off the repo and start from scratch because by the time security scanning tools find the secrets in the repo and report them to management, there is no way to revert the commits. So, it is critical to prevent this possibility.

There are other workarounds (like manually removing sensitive values from the exported files in Postman, but the whole idea of needing to export projects, so they can be checked into the source control is bad from the start, so let's just look at the better file-based solutions like httpYac), but there are two basic options that can be used to handle secrets.

Option 1: Store secrets in some sort of vault (PAM, Azure KeyVault, etc). Assuming there is a programmatic way to retrieve the secrets, it would require a startup script to retrieve them before the test run and setting the corresponding environment valiables. For example, to use this option with CyberArk Password Vault, you place the secret in the appropriate PAM container, register a local cert in the vault, and the script can make a REST API call to restrieve the secret in the beginning of the test (assuming it can supply the cert). The headache here is the original setup and cert updates, but that's what CI/CD tests must use (I cannot think of any other approach).

Option 2: This is mostly for local runs (i.e. during development, not for CI/CD). First, you isolate sensitive settings in the separate files and mark these files in the .gitignore file, so they do not get checked into the repo. Say, I have a brand new machine, so I do the initial checkout. I get the repo without the sensitive files. There would be moniker files with the settings, so we know what we need, checked into source control, but these would have no secrets, just names of the settings/variables and empty values. You copy the moniker files to the required files (that are git ignored) and set the values (manually copy them from some secure storage, co-worker, post-it note, or whatever). Now, you can run the tests, but the secrets will be isolated on your machine and will not leak into source control.

Option 3. I said that that there are two option because since these are simple, but the ideal way would be to have something like what Microsoft does when the secret values are initialized and stored locally in special location encrypted (via the dotnet user-secret set command). I undertsand that it would be somewhat complicated, so not something that I'd expect.

[alekdavisintel]

Just figured out how to set up IntelliJ environments (as I mentioned, I'm a novice). That's exatly what we need for option 2. Really cool. Thank you.

[AnWeber]

.env ist also supported to configure Environments. See https://github.com/httpyac/httpyac.github.io/tree/main/examples/project/environments.

[alekdavis]

I think there are three versions of config files in documentation, but only the IntelliJ version shows how to separate sensitive values from non-sensitive values. Maybe it's possible in alternatives, just missing examples.

[AnWeber]

Yes, there are several possibilities. dotenv exists to be compatible with vscode.rest-client. Intellij Config for compatibility with Jetbrains and .httpyac.config.js to be similar to other NodeJS tools. My favorite is dotenv. .env Files are searched in different locations. I store my public .env files in the repository and use a global location for my secrets (configured with HTTPYAC_ENV). You also use dev.env for public files and .env.dev for private. see docs

[alekdavisintel]

That's cool. I will take a closer look at dotenv, but for now, it does not seem like either one has any advantages over others (assuming they can all handle separation of secrets from non-secrets). I'm really impressed with the capabilities. Very appretiated.