Update the content of a Dataset in AzureML studio

[sjLeonGalan]

In the execution of a job in my MLOps pipeline, I want to obtain a dataset's content, modify it, and upload the new content, overwriting the old content of the file. Is there any option to do it? I have tried different forms, such as using Dataset.Tabular.register_pandas_Dataframe, but it creates a new directory for the new file, and in my case, I need to replace the existing file.

from azureml.core import Dataset
dataset_test = Dataset.Tabular.from_parquet_files(path = [(ds, 'main.parquet')])
file_dataframe = dataset_test.to_pandas_dataframe()
file_path = 'test_dir/main.parquet'

file_dataframe['column'] = 'new_value'

file_dataset = Dataset.Tabular.register_pandas_dataframe(
    dataframe=file_dataframe,
    target=(ds, file_path),
    name='main.parquet',
    description='Test upload new file'
)

--

Detalles del documento

⚠ No edite esta sección. Se requiere para learn.microsoft.com ➟ Vinculación de problema de GitHub.

• ID: ed683fb5-825c-23fa-ad73-02432e9ba89e
• Version Independent ID: a45ba4d2-5789-5b99-3780-85c320baf1ab
• Content: azureml.core.Dataset class - Azure Machine Learning Python
• Content Source: AzureML-Docset/stable/docs-ref-autogen/azureml-core/azureml.core.Dataset(class).yml
• Service: machine-learning
• Sub-service: core
• GitHub Login: @DebFro
• Microsoft Alias: **d

[AjayBathini-MSFT]

@sjLeonGalan It would be great if you could add a link to the documentation you are following for these steps? This would help us redirect the issue to the appropriate team. Thanks!!

[sjLeonGalan]

Im following this doc: https://learn.microsoft.com/es-es/python/api/azureml-core/azureml.core.dataset(class)?view=azure-ml-py #

[AjayBathini-MSFT]

@sjLeonGalan Thanks for your feedback! I've assigned this issue to the author who will investigate and update as appropriate.