Docs: Instruction on how to build houdini-openjd conda package

[paulgolter]

Documentation Issue

Hi everyone,

we have the issue that we want to run houdini 20 and above on deadline cloud.

The current houdini-openjd package raises an exception that it only supports houdini 19.5.

I already saw there is a commit in this repository updating the supported houdini version so I assume the conda package houdini-openjd will be updatet soon (Is thera already a rough date?)

But I was wondering if we need to build our own houdini-openjd conda package and upload it to our custom channel how that would work?

Is it a matter of building this python module and wrapping it in a similiar structure that is templatet in: https://github.com/aws-deadline/deadline-cloud-samples/tree/mainline/conda_recipes

I would appreciate some instructions :)

[epmog]

Yea, the recipes folder is a good place to start looking if you'd like to build your own conda package. We have some examples in there that should help, specifically maya-openjd which builds the conda package using pypi (or optionally your own git url).

An alternative that exists with the submitters, is to override the adaptors by submitting your own built packages (.whls) with the jobs. This is a built-in workflow you can use right now: https://github.com/aws-deadline/deadline-cloud-for-houdini/blob/mainline/DEVELOPMENT.md#application-interface-adaptor-development-workflow

Option is located in the job-specific settings tab:

If you want to learn more about conda package development with deadline cloud, then we have some docs in a few places as well. Namely the developer guide and a blog post:

1. https://docs.aws.amazon.com/deadline-cloud/latest/developerguide/configure-jobs-s3-channel.html
2. https://docs.aws.amazon.com/deadline-cloud/latest/developerguide/conda-package.html
3. https://aws.amazon.com/blogs/media/create-a-conda-package-and-channel-for-aws-deadline-cloud/

[jusiskin]

Hi @paulgolter, thanks for reaching out with this valuable feedback.

The current houdini-openjd package raises an exception that it only supports houdini 19.5.

This error you're seeing is likely coming from conda when using Deadline Cloud Service-managed fleets (SMF) and trying to specify houdini-openjd with a houdini conda package version > 19.5. This is because the conda package in SMF has a pinned dependency to houdini 19.5 currently.

You could look into a bring-your-own conda package solution where you relax the pinned version of houdini. As @epmog said, the best sample would be https://github.com/aws-deadline/deadline-cloud-samples/tree/mainline/conda_recipes/maya-openjd which is for a maya-openjd adaptor. This would need to be adjusted for Houdini.

Even with a relaxed houdini-openjd conda package, the next issue you would encounter in SMFs is that there is currently no houdini package for 20.0 or 20.5 available in the deadline-cloud conda channel. To overcome this, you could create your own houdini 20.0 / 20.5 conda package or bring your own copy of Houdini and the adaptor package into SMF through other means.

[paulgolter]

Hey @jusiskin @epmog

thanks for your answers!

We already managed to build our own houdini-20.0 package and upload as a conda package to our custom channel.

I also found the maya-openjd example in the deadline-cloud-examples repository.

I was hoping that there would be a way to essentially build the houdini-openjd package from this repository directly without having to go through these extra steps that are defined in this file in the maya example: https://github.com/aws-deadline/deadline-cloud-samples/blob/mainline/conda_recipes/maya-openjd/recipe/meta.yaml

In the maya-openjd example it references the source from pypi: https://pypi.io/packages/source/d/deadline-cloud-for-maya/deadline_cloud_for_maya-{{ version }}.tar.gz

And the issue is of course that my modified state of this deadline-cloud-for-houdini repository is not built and uploaded to pypi. I am not sure what I should use as a base archive / folder or how I can build this respository so I can make a conda package out of it.

I hope you can follow me.

All the best

[mwiebe]

If you push your houdini-openjd customizations to a branch in your github fork, you can reference that from the recipe like the commented-out code here:

https://github.com/aws-deadline/deadline-cloud-samples/blob/mainline/conda_recipes/maya-openjd/recipe/meta.yaml#L14

This is how I built the Maya adaptor with region render support. Is that what you're thinking of?

[jusiskin]

We just released a new PyPI package (0.7.3) one hour ago with the changes from mainline.

From my understanding though, the error message you were seeing:

The current houdini-openjd package raises an exception that it only supports houdini 19.5.

was not because of the unreleased commit in mainline, but rather because the houdini-openjd conda package in deadline-cloud conda channel available to SMF pins the houdini conda package dependency to version 19.5.

Unless you have other customizations/modifications beyond Houdini version support that you need to include in your conda package, you should be fine bringing your own conda package that uses the PyPI package.

[paulgolter]

Hey @mwiebe

you can reference that from the recipe like the commented-out code here:

I totally missed that part in the yaml file. I think that is exactly what I need to build my conda package with the updates from my fork of this repo. Will give that a try.

Hey @jusiskin

We just released a new PyPI package (0.7.3) one hour ago with the changes from mainline.

Perfect, then I don't even need to use my own fork!

[epmog]

You might be interested to know that among other conda packages released yesterday, the latest production builds for Houdini 19.5, Houdini 20, and Houdini 20.5 are available in the deadline-cloud conda channel (with a houdini-openjd package that works with all of them).