issues with the fv3config

[yuenxq]

I realized that this repo hasn't been updated for couple of years, just curious that is it still working? I got the following errors when I use it to setup the initialization for fv3 runs, see below, it looks to me that the cloud data was not existed, Traceback (most recent call last): File "./create_rundir.py", line 6, in fv3config.write_run_directory(config, 'rundir') File "/nix/store/h0j14vrv7lhbwncw688jkn2jdzc3albx-python3.8-fv3config-0.9.0/lib/python3.8/site-packages/fv3config/config/rundir.py", line 22, in write_run_directory write_assets_to_directory(config, target_directory) File "/nix/store/h0j14vrv7lhbwncw688jkn2jdzc3albx-python3.8-fv3config-0.9.0/lib/python3.8/site-packages/fv3config/_asset_list.py", line 257, in write_assets_to_directory asset_list = config_to_asset_list(config) File "/nix/store/h0j14vrv7lhbwncw688jkn2jdzc3albx-python3.8-fv3config-0.9.0/lib/python3.8/site-packages/fv3config/_asset_list.py", line 285, in config_to_asset_list asset_list += get_initial_conditions_asset_list(config) File "/nix/store/h0j14vrv7lhbwncw688jkn2jdzc3albx-python3.8-fv3config-0.9.0/lib/python3.8/site-packages/fv3config/_asset_list.py", line 64, in get_initial_conditions_asset_list source_directory = get_initial_conditions_directory(config) File "/nix/store/h0j14vrv7lhbwncw688jkn2jdzc3albx-python3.8-fv3config-0.9.0/lib/python3.8/site-packages/fv3config/_datastore.py", line 82, in get_initial_conditions_directory ensure_exists(config["initial_conditions"], "initial_conditions") File "/nix/store/h0j14vrv7lhbwncw688jkn2jdzc3albx-python3.8-fv3config-0.9.0/lib/python3.8/site-packages/fv3config/_datastore.py", line 87, in ensure_exists if not filesystem.get_fs(location).exists(location): File "/nix/store/j2gci7gi9rpxnkbamy4nq4xq7c9k27qv-python3.8-fsspec-2021.04.0/lib/python3.8/site-packages/fsspec/asyn.py", line 72, in wrapper return sync(self.loop, func, *args, **kwargs) File "/nix/store/j2gci7gi9rpxnkbamy4nq4xq7c9k27qv-python3.8-fsspec-2021.04.0/lib/python3.8/site-packages/fsspec/asyn.py", line 53, in sync raise result[0] File "/nix/store/j2gci7gi9rpxnkbamy4nq4xq7c9k27qv-python3.8-fsspec-2021.04.0/lib/python3.8/site-packages/fsspec/asyn.py", line 20, in _runner result[0] = await coro File "/nix/store/j2gci7gi9rpxnkbamy4nq4xq7c9k27qv-python3.8-fsspec-2021.04.0/lib/python3.8/site-packages/fsspec/asyn.py", line 256, in _exists await self._info(path) File "/nix/store/8kbbp1g6lil05a4p2pqxbv8wrag58pmx-python3.8-gcsfs-0.7.1/lib/python3.8/site-packages/gcsfs/core.py", line 783, in _info return await self._get_object(path) File "/nix/store/8kbbp1g6lil05a4p2pqxbv8wrag58pmx-python3.8-gcsfs-0.7.1/lib/python3.8/site-packages/gcsfs/core.py", line 577, in _get_object bucket, await self._call("GET", "b/{}/o/{}", bucket, key, json_out=True) File "/nix/store/8kbbp1g6lil05a4p2pqxbv8wrag58pmx-python3.8-gcsfs-0.7.1/lib/python3.8/site-packages/gcsfs/core.py", line 507, in _call self.validate_response(status, contents, json, path, headers) File "/nix/store/8kbbp1g6lil05a4p2pqxbv8wrag58pmx-python3.8-gcsfs-0.7.1/lib/python3.8/site-packages/gcsfs/core.py", line 1224, in validate_response raise IOError("Forbidden: %s\n%s" % (path, msg)) OSError: Forbidden: https://www.googleapis.com/storage/v1/b/vcm-fv3config/o/data%2Finitial_conditions%2Fgfs_c12_example%2Fv1.0 The billing account for the owning project is disabled in state absent

[spencerkclark]

fv3config is a fairly stable project at this point—we still use it in various ways internally. It looks like this could potentially be a Google Cloud Storage configuration issue. The data you are trying to access is in a "Requester Pays" bucket, meaning that you will need to have a billing account set up to access the data. Is that the case for your project?

[yuenxq]

I was following the fv3gfs's guide to setup the access to the data and trying to run some case on CPU for performance study. does the fv3config docment point to different google cloud account?

[yuenxq]

@spencerkclark did you run some case with the repo, https://github.com/ai2cm/fv3gfs-fortran

[spencerkclark]

The tests in the fv3gfs-fortran repository run successfully for me.

Neither repository hard-codes the assumption of a particular Google Cloud project, but your active project must be able to be billed in order to be able to download data from the gs://vcm-fv3config bucket.