Unable to access files outside of archive

eloo-abi commented 6 months ago

Brief summary

Hi, we are trying to use k6 in a kubernetes environment together with the k6-operator (not sure if its a k6 or operator issue) Therefore we want to mount our test configurations as configmap into the runner pods and then load it at runtime like this

const configFile = `${__ENV.CONFIG_FILES_DIRECTORY}/test-config.json`;

const testConfig = JSON.parse(open(configFile));

but as we are bundling our tests as k6 archive it looks like we are running into a bug that we are not able to load file from the filesystem.

k6 is complaining that the file is not existing as long as it is not bundled into the tarball under "file"

At least i can not found anything in the docs for such a limitation so i think its some kind of bug that the filesystem is not reachable.

Best regards

k6 version

0.50.0

OS

kubernetes

Docker version and image (if applicable)

No response

Steps to reproduce the problem

Create a simple test with


const configFile = `${__ENV.CONFIG_FILES_DIRECTORY}/test-config.json`;

const testConfig = JSON.parse(open(configFile));


- add some random file here  `/tmp/test-config.json`
- add some random file next to the test js file `./test-config.json`
- run `k6 archive --env CONFIG_FILES_DIRECTORY=.`
- try to run the archive `k6 run my-test.tar  --env CONFIG_FILES_DIRECTORY=/tmp`

### Expected behaviour

i can access the filesystem in the same way as without a k6 archive

### Actual behaviour

looks like the k6 archive has a different way to load files as plain script executions

codebien commented 5 months ago

Hi @eloo-abi, thanks for reporting the issue.

i can access the filesystem in the same way as without a k6 archive

Unfortunately, this is not how it is expected to work. The archive feature allows the script to be sharable which means all the resources needed for running the test have to be in the archive. k6 resolves and stores the files in the archive when k6 archive command runs. So, if you attempt to access a resource not mapped in the archive then k6 returns an error.

The steps reported in the Steps to reproduce the problem section led to a working script for me. Isn't it for you? Or did you omit the pre-requisite to run them through k8s or k6-operator?

eloo-abi commented 5 months ago

Hi @codebien thanks for the response...

Ah yes for sure i have missed a step.. or better said a detail. The configFile is using a env variable to resolve.. i will update the steps to reproduce.

The archive feature allows the script to be sharable which means all the resources needed for running the test have to be in the archive

yes i totally agree with that. but this brings some more downsides with may could be eliminated by adding something like loadFileFromOutside or open(configFile, true) where the addition parameter is includeFIlesystemOutsideTarball or something like that

because in our scenario we want to share the code for our tests between our stages. but for sure we don't want to share our secrets and configurations between our environments. packing all the live secrets into the same unencrypted tarball and push it into a non-live environment is pretty bad :D

so our idea was we have the secrets and configurations provided by our environments and then share the test code between stages

maybe this could be improved in future to be more flexible here

thanks

edit: steps to reproduce updated

codebien commented 5 months ago

because in our scenario we want to share the code for our tests between our stages. but for sure we don't want to share our secrets and configurations between our environments. packing all the live secrets into the same unencrypted tarball and push it into a non-live environment is pretty bad :D

@eloo-abi --exclude-env-vars is an option for k6 archive that serves specifically this case. Of course, you have to pass them when you do k6 run via --env flags. If you are using a few env vars that should be fine, instead, if you are using a lot of env vars then it could be not the best UX.

eloo-abi commented 5 months ago

--exclude-env-vars

sounds to me like i do NOT get system envs into my current run. but thats NOT what i want..

i build the tarball on a CI system (decoupled from our applications) with a default test config then in the specific environments (different kubernetes clusters) i need to exchange the configs because these are environment specific for sure.

so exclude env vars is not what helps me here

If you are using a few env vars that should be fine, instead, if you are using a lot of env vars then it could be not the best UX

we have plenty of data here.. sometime more than 100 keys so env vars is not an option here

grafana / k6