Azure / azure-databricks-client

Client library for Azure Databricks
MIT License
72 stars 62 forks source link

PipelinesApiClient.ListPageable() throws when no pipelines are returned #227

Open tommi1hirvonen opened 2 weeks ago

tommi1hirvonen commented 2 weeks ago

When fetching pipelines from a Databricks workspace that doesn't contain any pipelines, PipelinesApiClient.ListPageable() throws ArgumentNullException because the Pipelines property in PipelinesList is null.

// PipelinesApiClient.cs

public global::Azure.AsyncPageable<Pipeline> ListPageable(int pageSize = 25, CancellationToken cancellationToken = default)
{
    return new AsyncPageable<Pipeline>(
        async (string pageToken) =>
        {
            var response = await List(pageSize, pageToken, cancellationToken).ConfigureAwait(false);
            // =====> response.Pipelines is null when no pipelines are returned <=====
            return (response.Pipelines.ToList(), response.HasMore, response.NextPageToken);
        }
    );
}

This can also be a source of surprise when using PipelinesApiClient.List(), since the property IEnumerable<Pipeline> Pipelines in the returned PipelinesList is null rather than an empty sequence.

I think ListPageable() should definitely not throw when there are no pipelines, and I also think most people would expect List() to return an empty sequence in PipelinesList rather than null.

tommi1hirvonen commented 2 weeks ago

I've looked at the source code and compared the combination of PipelinesApiClient and PipelinesList to JobsApiClient and JobList. This issue is not present with jobs, i.e. fetching an empty list of jobs works as expected.

To me, a consistent solution would be to initialize PipelinesList.Pipelines as an empty list in the constructor, same as in JobList.

Would you be open to a community contribution/PR? 🙂