Open fjuniorr opened 1 year ago
Another possibly related behaviour is that trying to infer stats gives correct values for fields
and rows
, but not for hash
which gets the value 'sha256:None'
when in theory data should be coming from path
(which is set to 'output.csv'
) and not the in-memory data
.
from frictionless import Resource, steps, Pipeline
pipeline = Pipeline(steps=[
steps.field_update(name='id', descriptor = {'name': 'pkey', 'title': 'Primary Key'}),
steps.row_filter(formula='name != "france"'),
steps.table_write(path='output.csv'),
steps.resource_update(name='data', descriptor={'path': 'output.csv'}),
])
resource = Resource(path='https://raw.githubusercontent.com/frictionlessdata/frictionless-py/d6f2552b4fd950f459130eda9cf80ae0b8b4931e/data/transform.csv')
resource.transform(pipeline)
resource.infer(stats=True)
print(f'{resource=}')
Thanks @fjuniorr for reporting. We will investigate it.
Overview
I want to be able to change both metadata and the data in a transform pipeline and export both data and metadata (I think as in https://github.com/frictionlessdata/frictionless-py/issues/1062). An example might be:
which gives me what I want:
However if I run
resource.to_view()
I getTrying to set
data
toNone
in the pipelinealso don't help because I get
In general is mixing
path
anddata
a bad idea during a pipeline transformation? Is there other alternative to deal with the use case of changing and exporting both data and metadata?