Use requirements.txt inside conda.yaml file for enviroment config

[LucianCrainic]

Hello, currently I am using a conda.yaml file for my env dependecies when building with rcc. Having the following structure:

channels:
  - conda-forge
dependencies:
  - python=3.9.18
  - pip=23.2.1
  - pip:
    - cryptography

I would love to know from the maintainers if there is a way to also use requirements.txt inside the conda.yaml file, i have tried the following approach but it fails:

channels:
  - conda-forge
dependencies:
  - python=3.9.18
  - pip=23.2.1
  - pip:
    - -r file:requirements.txt

with the requirements.txt file being at the same level of the conda.yaml and robot.yaml file. This approach results in a fail while building with RCC since it does not find the requirements.txt file. I also tried to use a full path without any success.

My guess after reading a part of the RCC source code is that rcc takes the conda.yaml file and creates a requirements.txt file from it that it passes to pip.

I am asking this since i would preefer to have a requirements.txt file that other operators can use to build their enviroment without RCC, leaving the decisions to the operator itself if to use RCC or not.

Thank you.

[slalomsk8er]

Not quite the requirements.txt but have a look at https://github.com/robocorp/rcc/blob/master/docs/recipes.md#how-to-freeze-dependencies also isn't the stuff under pip in conda.yaml easy to morph into an requirements.txt?

[LucianCrainic]

Not quite the requirements.txt but have a look at https://github.com/robocorp/rcc/blob/master/docs/recipes.md#how-to-freeze-dependencies.

Not what i am looking for, my goal is to use the requirements.txt in the conda.yaml file for RCC and also use it if i want to build the enviroment by myself without using RCC.

To make a clear example, if someone in my working team does not want to use RCC to build the Test Suite Enviroment i want to leave the option of using the requirements.txt file for the pip installation but at the same time i want to use that same requirements.txt for the RCC enviroment without the need of creating a separate conda.yaml file.

[slalomsk8er]

My guess, the best would be a helper script to generate the requirements.txt from the conda.yaml or investigate, if using conda without rcc is a viable option.

[kariharju]

Hi, I bit of a longer response, but hope to explain why this is not a a thing and what options you may be looking for that are already in RCC.

The reason why the requirements.txt is not an option in conda.yaml has to do with the ability to cache the environments. To be able to create, isolate and recreate the environment a "unique key" is needed. That is calculated from the conda.yaml content. If there would be only requirements.txt that would always be unchanging. There are also a lot of things users can do inside requirements.txt and cause ambiguous dependencies and just pure supply chain attacks. RCC is mainly made to provide repeatable environments (not just Python) in way that those environments can be packaged, scanned, validated and recreated on another machine or another OS even. In short being able to use Python on enterprise level production environments.

However RCC can be used in multiple ways. So you could use RCC and conda.yaml just to get your base environment. Then "activate" the environment by setting the env. variables that rcc ht vars command returns and jumping in that Python environment. There is also the rcc venv command which may be what you are looking for and if you do not want RCC making sure the base environment is unchanged check out the --unmanaged flag.

And yeah the dependency freezing over the whole environment, not just the python deps. is a huuuge thing when you actually need to know every single file on the executing environment. Dependency without a strict version is a just call for surprise breakages when the package eventually releases breaking changes.

Hope these help.

[LucianCrainic]

Hi @kariharju, sorry for the late reply. I totally got your point and explanation, my only doubt is in regard to this part

So you could use RCC and conda.yaml just to get your base environment.

what do you mean with base environment ?