DarkWanderer / ClickHouse.Client

.NET client for ClickHouse
MIT License
315 stars 62 forks source link

WriteToServerAsync always doesn't insert the first row #319

Closed adolfosp closed 1 year ago

adolfosp commented 1 year ago

Hi @DarkWanderer

First of all thanks for creating this ClickHouse client library. I have one problem when I am going to insert rows by bulk. Always the first row doesn't insert in the database. I was debugging this library and the enumerator. MoveNext () always get the second line as the first value to enumerator.Current. Is it a me problem ? Or is it a library problem ?

                try
                {
                    while (hasMore = enumerator.MoveNext())
                    {
                        row = enumerator.Current;
                        for (col = 0; col < row.Length; col++)
                        {
                            columnTypes[col].Write(writer, row[col]);
                        }
                        counter++;

                        if (counter >= BatchSize)
                            break; // We've reached the batch size
                    }
                }
                catch (Exception e)
                {
                    throw new ClickHouseBulkCopySerializationException(row, col, e);
                }

Class: ClickHouseBulkCopy Package Version: 6.6.0

DarkWanderer commented 1 year ago

Hi. I would not expect this to be a bug in the library - BulkCopy insertion is checked as part of test suite. Do you call MoveNext or Skip on the iterator by any chance? Can you try to create a minimum reproducible example?

adolfosp commented 1 year ago

In this example, I used the last version of the library

Example: https://github.com/adolfosp/Issue-319-ClickHouse.Client.git

DarkWanderer commented 1 year ago

Checked your code - your issue is here:

    while (reader.Read())
        await bulkCopyInterface.WriteToServerAsync(reader);

You are reading the first row before passing the reader to ClickHouseBulkCopy. BulkCopy then reads the rest of the reader and your cycle finishes after 1 iteration

BulkCopy class will call the Read method by itself (implicitly via AsEnumerable extension), you don't need to call Read yourself.

adolfosp commented 1 year ago

@DarkWanderer Thanks a lot for your help. It worked for me