search

catalyst-cooperative / pudl-usage-metrics

A dagster ETL for collecting and cleaning PUDL usage metrics.
MIT License
1 stars 0 forks source link

Bump dagster-pandera from 0.14.9 to 0.15.0 #25

Closed dependabot[bot] closed 2 years ago

dependabot[bot] commented 2 years ago

Bumps dagster-pandera from 0.14.9 to 0.15.0.

Release notes

Sourced from dagster-pandera's releases.

0.15.0 - Cool for the Summer (2022-06-14)

Major Changes

  • Software-defined assets are now marked fully stable and are ready for prime time - we recommend using them whenever your goal using Dagster is to build and maintain data assets.
  • You can now organize software defined assets into groups by providing a group_name on your asset definition. These assets will be grouped together in Dagit.
  • Software-defined assets now accept configuration, similar to ops. E.g. 
    from dagster import asset
@asset(config_schema={"iterations": int})
def my_asset(context):
    for i in range(context.op_config["iterations"]):
        ...
  • Asset definitions can now be created from graphs via AssetsDefinition.from_graph: 
    @graph(out={"asset_one": GraphOut(), "asset_two": GraphOut()})
def my_graph(input_asset):
    ...
graph_asset = AssetsDefinition.from_graph(my_graph)
  • execute_in_process and GraphDefinition.to_job now both accept an input_values argument, so you can pass arbitrary Python objects to the root inputs of your graphs and jobs.
  • Ops that return Outputs and DynamicOutputs now work well with Python type annotations. You no longer need to sacrifice static type checking just because you want to include metadata on an output. E.g. 
    from dagster import Output, op
@op
def my_op() -> Output[int]:
    return Output(5, metadata={"a": "b"})
  • You can now automatically re-execute runs from failure. This is analogous to op-level retries, except at the job level.
  • You can now supply arbitrary structured metadata on jobs, which will be displayed in Dagit.
  • The partitions and backfills pages in Dagit have been redesigned to be faster and show the status of all partitions, instead of just the last 30 or so.
  • The left navigation pane in Dagit is now grouped by repository, which makes it easier to work with when you have large numbers of jobs, especially when jobs in different repositories have the same name.
  • The Asset Details page for a software-defined asset now includes a Lineage tab, which makes it easy to see all the assets that are upstream or downstream of an asset.

Breaking Changes and Deprecations

Software-defined assets

This release marks the official transition of software-defined assets from experimental to stable. We made some final changes to incorporate feedback and make the APIs as consistent as possible:

  • Support for adding tags to asset materializations, which was previously marked as experimental, has been removed.
  • Some of the properties of the previously-experimental AssetsDefinition class have been renamed. group_names is now group_names_by_key, asset_keys_by_input_name is now keys_by_input_name, and asset_keys_by_output_name is now keys_by_output_name, asset_key is now key, and asset_keys is now keys.
  • Removes previously experimental IO manager fs_asset_io_manager in favor of merging its functionality with fs_io_manager. fs_io_manager is now the default IO manager for asset jobs, and will store asset outputs in a directory named with the asset key. Similarly, removed adls2_pickle_asset_io_manager, gcs_pickle_asset_io_manager , and s3_pickle_asset_io_manager. Instead, adls2_pickle_io_manager, gcs_pickle_io_manager, and s3_pickle_io_manager now support software-defined assets.
  • (deprecation) The namespace argument on the @asset decorator and AssetIn has been deprecated. Users should use key_prefix instead.
  • (deprecation) AssetGroup has been deprecated. Users should instead place assets directly on repositories, optionally attaching resources using with_resources. Asset jobs should be defined using define_assets_job (replacing AssetGroup.build_job), and arbitrary sets of assets can be materialized using the standalone function materialize (replacing AssetGroup.materialize).
  • (deprecation) The outs property of the previously-experimental @multi_asset decorator now prefers a dictionary whose values are AssetOut objects instead of a dictionary whose values are Out objects. The latter still works, but is deprecated.
  • The previously-experimental property on OpExecutionContext called output_asset_partition_key is now deprecated in favor of asset_partition_key_for_output

Event records

... (truncated)

Changelog

Sourced from dagster-pandera's changelog.

0.15.0 "Cool for the Summer"

Major Changes

  • Software-defined assets are now marked fully stable and are ready for prime time - we recommend using them whenever your goal using Dagster is to build and maintain data assets.
  • You can now organize software defined assets into groups by providing a group_name on your asset definition. These assets will be grouped together in Dagit.
  • Software-defined assets now accept configuration, similar to ops. E.g. 
    from dagster import asset

    @​asset(config_schema={"iterations": int})
def my_asset(context):
for i in range(context.op_config["iterations"]):
...

  • Asset definitions can now be created from graphs via AssetsDefinition.from_graph: 
    @graph(out={"asset_one": GraphOut(), "asset_two": GraphOut()})
def my_graph(input_asset):
    ...

    graph_asset = AssetsDefinition.from_graph(my_graph)

  • execute_in_process and GraphDefinition.to_job now both accept an input_values argument, so you can pass arbitrary Python objects to the root inputs of your graphs and jobs.
  • Ops that return Outputs and DynamicOutputs now work well with Python type annotations. You no longer need to sacrifice static type checking just because you want to include metadata on an output. E.g. 
    from dagster import Output, op

    @​op
def my_op() -> Output[int]:
return Output(5, metadata={"a": "b"})

  • You can now automatically re-execute runs from failure. This is analogous to op-level retries, except at the job level.
  • You can now supply arbitrary structured metadata on jobs, which will be displayed in Dagit.
  • The partitions and backfills pages in Dagit have been redesigned to be faster and show the status of all partitions, instead of just the last 30 or so.
  • The left navigation pane in Dagit is now grouped by repository, which makes it easier to work with when you have large numbers of jobs, especially when jobs in different repositories have the same name.
  • The Asset Details page for a software-defined asset now includes a Lineage tab, which makes it easy to see all the assets that are upstream or downstream of an asset.

Breaking Changes and Deprecations

Software-defined assets

This release marks the official transition of software-defined assets from experimental to stable. We made some final changes to incorporate feedback and make the APIs as consistent as possible:

  • Support for adding tags to asset materializations, which was previously marked as experimental, has been removed.
  • Some of the properties of the previously-experimental AssetsDefinition class have been renamed. group_names is now group_names_by_key, asset_keys_by_input_name is now keys_by_input_name, and asset_keys_by_output_name is now keys_by_output_name, asset_key is now key, and asset_keys is now keys.
  • Removes previously experimental IO manager fs_asset_io_manager in favor of merging its functionality with fs_io_manager. fs_io_manager is now the default IO manager for asset jobs, and will store asset outputs in a directory named with the asset key. Similarly, removed adls2_pickle_asset_io_manager, gcs_pickle_asset_io_manager , and s3_pickle_asset_io_manager. Instead, adls2_pickle_io_manager, gcs_pickle_io_manager, and s3_pickle_io_manager now support software-defined assets.
  • (deprecation) The namespace argument on the @asset decorator and AssetIn has been deprecated. Users should use key_prefix instead.
  • (deprecation) AssetGroup has been deprecated. Users should instead place assets directly on repositories, optionally attaching resources using with_resources. Asset jobs should be defined using define_assets_job (replacing AssetGroup.build_job), and arbitrary sets of assets can be materialized using the standalone function materialize (replacing AssetGroup.materialize).
  • (deprecation) The outs property of the previously-experimental @multi_asset decorator now prefers a dictionary whose values are AssetOut objects instead of a dictionary whose values are Out objects. The latter still works, but is deprecated.

... (truncated)

Commits


Dependabot compatibility score

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
dependabot[bot] commented 2 years ago

Superseded by #26.