search

legend-exp / legend-dataflow

LEGEND data flow management
Other
2 stars 9 forks source link

If `$PRODENV` is not set, then `complete_run.py` fails #45

Closed slwatkins closed 1 day ago

slwatkins commented 2 months ago

If the user does not set PRODENV in their environment, then the following line will fail because there is a comparison between a NoneType object and string (because in this case type(os.getenv("PRODENV")) is NoneType.)

https://github.com/legend-exp/legend-dataflow/blob/37c7bdfe71f261471f6efd1e170d3320eb6b7a1d/scripts/complete_run.py#L177

I recommend changing this if-statement to:

if (os.getenv("PRODENV") is not None) and (os.getenv("PRODENV") in snakemake.params.filedb_path):

Thus, the if-statement first checks if the path is defined, then if so, checks if it is in the filedb_path. If either of these are false, then it goes to the else statement.

gipert commented 4 days ago

The FileDB generation steps need PRODENV to be set (important for the FileDB object being relocatable). @ggmarshall maybe we could make Snakemake fail early is PRODENV is not set?

slwatkins commented 3 days ago

Interesting - I have seen FileDB be generation just fine with this fix, no need to actually define it in, e.g., my bashrc.

gipert commented 3 days ago

Sure it won't fail, but the FileBD won't be relocatable.

slwatkins commented 3 days ago

At which step would the data flow fail due to FileDB not being relocatable? I've been able to build all the way to SKM with this fix and check energies and all that, so I'm a bit surprised I was able to do that without seeing an error due to this. I could totally be misunderstanding what FileDB is used for though.

ggmarshall commented 3 days ago

dataflow won't fail but we wouldn't be able to move the data elsewhere and still use the filedb, its an issue more for users