Incorrect data node resolving

[duncanwp]

Trying to generate a recipe from a valid IID:

await generate_recipe_inputs_from_iids(["CMIP6.CMIP.NCC.NorESM2-LM.1pctCO2.r2i1p1f1.day.tas.gn"])

Results in the error:

Requesting data for Node: https://esgf-node.llnl.gov/esg-search/search and CMIP6.CMIP.NCC.NorESM2-LM.1pctCO2.r2i1p1f1.day.tas.gn...
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
Cell In [6], line 1
----> 1 await generate_recipe_inputs_from_iids(["CMIP6.CMIP.NCC.NorESM2-LM.1pctCO2.r2i1p1f1.day.tas.gn"])

File ~/PycharmProjects/pangeo-forge-esgf/pangeo_forge_esgf/recipe_inputs.py:85, in generate_recipe_inputs_from_iids(iid_list)
     82 for iid in iid_list:
     83     tasks.append(asyncio.ensure_future(iid_request(session, iid, search_node)))
---> 85 raw_input = await asyncio.gather(*tasks)
     86 recipe_inputs = {
     87     iid: {"urls": urls, **kwargs}
     88     for iid, (urls, kwargs) in zip(iid_list, raw_input)
     89     if urls is not None
     90 }
     92 print(
     93     "Failed to create recipe inputs for: \n"
     94     + "\n".join(sorted(list(set(iid_list) - set(recipe_inputs.keys()))))
     95 )

File ~/miniconda3/envs/climatebench-forge/lib/python3.9/asyncio/tasks.py:328, in Task.__wakeup(self, future)
    326 def __wakeup(self, future):
    327     try:
--> 328         future.result()
    329     except BaseException as exc:
    330         # This may also be a cancellation.
    331         self.__step(exc)

File ~/miniconda3/envs/climatebench-forge/lib/python3.9/asyncio/tasks.py:256, in Task.__step(***failed resolving arguments***)
    252 try:
    253     if exc is None:
    254         # We use the `send` method directly, because coroutines
    255         # don't have `__iter__` and `__next__` methods.
--> 256         result = coro.send(None)
    257     else:
    258         result = coro.throw(exc)

File ~/PycharmProjects/pangeo-forge-esgf/pangeo_forge_esgf/recipe_inputs.py:106, in iid_request(session, iid, node, params)
    103 kwargs = None
    105 print(f"Requesting data for Node: {node} and {iid}...")
--> 106 response_data = await _esgf_api_request(session, node, iid, params)
    108 print(f"Filtering response data for {iid}...")
    109 filtered_response_data = await sort_and_filter_response(response_data, session)

File ~/PycharmProjects/pangeo-forge-esgf/pangeo_forge_esgf/recipe_inputs.py:152, in _esgf_api_request(session, node, iid, params)
    150 resp_data = resp_data["response"]["docs"]
    151 if len(resp_data) == 0:
--> 152     raise ValueError(f"No Files were found for {iid}")
    153 return resp_data

ValueError: No Files were found for CMIP6.CMIP.NCC.NorESM2-LM.1pctCO2.r2i1p1f1.day.tas.gn

I think to do with the logic for determining which data node to use. If I switch the noresg.nird.sigma2.no to the top of the data_nodes list in recipe_inputs.py this works successfully.

[duncanwp]

Apologies, I'm talking nonsense CMIP6.CMIP.NCC.NorESM2-LM.1pctCO2.r2i1p1f1.day.tas.gn doesn't exist but ...r1i1p1f1... does!

[jbusecke]

Hi @duncanwp, thanks for using pangeo-forge-esgf. Glad this worked itself out. Do you think that the error message could be edited to be more helpful?

[duncanwp]

No, I think this was just a case of user error :-) Thanks for checking though