GoogleCloudPlatform / healthcare-data-harmonization

This is an engine that converts data of one structure to another, based on a configuration file which describes how. There is an accompanying syntax to make writing mappings easier and more robust.
https://cloud.google.com/solutions/healthcare-life-sciences
Apache License 2.0
198 stars 64 forks source link

Missing some key arguments from V1 #61

Open torstees opened 9 months ago

torstees commented 9 months ago

I am not using whistle inside the cloud but as part of a pipeline to convert CSV files into FHIR. There are several arguments in the previous version that don't appear to be present in the shell script, runtime:

-harmonize_code_dir_Spec -lib_dir_spec -verbose

Obviously the format of the arguments is different (now with short names and two dashes for long names) but using any of these causes the help text to be displayed.

When I run without the harmonize and lib_dir args, I get the following stack dump:

$ /home/****/bin/whistle --input_file_spec output/whistle-input/ds-sleep.json --mapping_file_spec _entry.wstl --output_dir output/whistle-output Exception in thread "main" com.google.cloud.verticals.foundations.dataharmonization.exceptions.NoMatchingOverloadsException: Unknown function Transform_Dataset() at com.google.cloud.verticals.foundations.dataharmonization.function.DefaultClosure.suggestFunctionNames(DefaultClosure.java:285) at com.google.cloud.verticals.foundations.dataharmonization.function.DefaultClosure.execute(DefaultClosure.java:254) at com.google.cloud.verticals.foundations.dataharmonization.function.context.impl.DefaultRuntimeContext.evaluate(DefaultRuntimeContext.java:108) at com.google.cloud.verticals.foundations.dataharmonization.function.whistle.WhistleFunction.executeMappings(WhistleFunction.java:102) at com.google.cloud.verticals.foundations.dataharmonization.plugin.Option.withConfig(Option.java:93) at com.google.cloud.verticals.foundations.dataharmonization.function.whistle.WhistleFunction.callInternal(WhistleFunction.java:94) at com.google.cloud.verticals.foundations.dataharmonization.function.context.impl.InitializationContext.wrap(InitializationContext.java:107) at com.google.cloud.verticals.foundations.dataharmonization.function.CallableFunction.call(CallableFunction.java:65) at com.google.cloud.verticals.foundations.dataharmonization.init.Engine.transform(Engine.java:118) at com.google.cloud.verticals.foundations.dataharmonization.Main.main(Main.java:122)

whistle is just a sym link to the runtime script created by installDist

rpolyano commented 8 months ago

Hi,

Whistle 2 is not fully backwards compatible with Whistle 1. So, you will need to update your mappings (usually the changes are relatively few). As the error specifies, there is no function called "Transform_Dataset" in your code.

As for things like code harmonization (harmonize_code_dir_spec) those will be added in the near future with documentation on how to use them.