Streaming from one card to another

[santolucito]

I'm not sure this is exactly an issue, but I'm hoping to get some help on the best way to stream from one card to another (conveniently configured with sdio thanks to this project) and I'm not sure where to turn. My strategy is to just read from one and write to the other, as below. This is working, but feels a bit slow on large files. I haven't done proper benchmarking yet, but I'm wondering if there is some obviously better way of doing things. Would love any pointers.

// Read from the source file and write to the destination file
    char buf[128];
    UINT br, bw;
    while (f_read(&src_file, buf, sizeof(buf), &br) == FR_OK && br > 0)
    {
        f_write(&dest_file, buf, br, &bw);
        if (bw != br)
        {
            printf("Error writing to destination file\n");
            exit(4);
        }
    }

For context https://github.com/santolucito/SDCopier/blob/099201f4d8a6a58ebd46a4abfbf2753f7aac37d8/no-OS-FatFs/examples/simple/main.c#L118

[carlk3]

Have a look at Performance Tuning Tips. If you want to go fast, a 128 byte buffer is tiny. Remember, the SD card is a block device and the block size is 512 bytes, so anything less than that (or not a multiple of that) will result in quite inefficient read-modify-write operations. Something like 32768 bytes would be more like it. However, watch your stack space. You might want to make the buffer static, or malloc the buffer. (See big_file_test.c or run_cp in command.cpp for examples.)

Also, 4-byte alignment of the buffer is critical to good performance on SDIO. Your stack buffer is unlikely to be aligned. malloc is guaranteed to return an aligned buffer.

[carlk3]

Also, you could use two buffers in a ping-pong arrangement (or you could think of it as a 2-element circular buffer) and two FreeRTOS tasks to pipeline the reads and writes. The parallelism will be limited if the two cards are sharing an SPI bus, but it should be good if the cards are on separate SPIs, or on separate SDIO buses, or one SPI and one SDIO.

[santolucito]

Ah very cool idea. I'll need to have the tasks on separate cores to benefit from this, is that correct? I have my second core handling the UI at the moment, but I can probably rearrange things a bit to make this work.

[carlk3]

FreeRTOS can multi-task on a single core.

[santolucito]

Ah I'll admit, I had to do a quick search to understand that multitasking on a single core is useful to get instruction pipelining to speed things up. Very neat, I'll give this a try and do some benchmarking and update later!

Thank you for the help!