what does available() represents when using queues?

JBSchueler commented 2 years ago

I use 16 queues and each queue has a buffer size of 4 bytes. If I request all 16 queues what will be the value of .available()? Will it be 16 like the number of queues? Or will it be 64, the number of bytes in total captured?

Could you please clarify a bit more what the variables are?

hideakitai commented 2 years ago

Please see the descriptions of APIs added to README

JBSchueler commented 2 years ago

Thanks for the comment update. available() is the sum of the bytes received from all the queues, right?

hideakitai commented 2 years ago

size of the received bytes of the oldest queued transaction result

No, the bytes of the oldest transaction you received that you haven't popped.

JBSchueler commented 2 years ago

That's only valid for size() correct?

available() represent the total amount of bytes received.

size_t available() const; // size of completed (received) transaction results

So, before I copy data I can use size() to check if that queue actually did receive 4 bytes. Just to be sure I did receive all 4 bytes.

JBSchueler commented 2 years ago

Ok, I got a lot further in making it work... function	description
`.size()`	number of bytes in oldest queue
`.available()`	number of queues available
`.remained()`	number of queues not completed

total amount of queues occupied is the sum of .available() and .remained().

There I went wrong in my program... I only checked .available() for a certain threshold before I added new queues. But I forgot that there are .remained() queues as well. The sum of .available() and .remained() should not exceed the queues set by setQueueSize().

Nevertheless I get warnings... [WARNING] slave queue is full with transactions. discard new transaction request

In my program I keep track of how many queues I use. Every queue() I increment a variable p_qoccupied and every .pop() I decrement. But somehow the sum of (.available() and .remained()) and the value of p_qoccupied are different resulting in this log.

rst:0x10 (RTCWDT_RTC_RESET),boot:0x33 (SPI_FAST_FLASH_BOOT)
configsip: 0, SPIWP:0xee
clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00
mode:DIO, clock div:1
load:0x3fff0030,len:1324
ho 0 tail 12 room 4
load:0x40078000,len:13508
load:0x40080400,len:3604
entry 0x400805f0
Queue Buffers : 64
Queue Size     : 4 bytes
Queue Space    : 256 bytes
[WARNING] slave queue is full with transactions. discard new transaction request
[WARNING] slave queue is full with transactions. discard new transaction request
[WARNING] slave queue is full with transactions. discard new transaction request
[WARNING] slave queue is full with transactions. discard new transaction request
[WARNING] slave queue is full with transactions. discard new transaction request
[WARNING] slave queue is full with transactions. discard new transaction request
[WARNING] slave queue is full with transactions. discard new transaction request
[WARNING] slave queue is full with transactions. discard new transaction request
[WARNING] slave queue is full with transactions. discard new transaction request
[WARNING] slave queue is full with transactions. discard new transaction request
[WARNING] slave queue is full with transactions. discard new transaction request
[WARNING] slave queue is full with transactions. discard new transaction request
[WARNING] slave queue is full with transactions. discard new transaction request
[WARNING] slave queue is full with transactions. discard new transaction request
[WARNING] slave queue is full with transactions. discard new transaction request
[WARNING] slave queue is full with transactions. discard new transaction request
[WARNING] slave queue is full with transactions. discard new transaction request
[WARNING] slave queue is full with transactions. discard new transaction request
[WARNING] slave queue is full with transactions. discard new transaction request
[WARNING] slave queue is full with transactions. discard new transaction request
[WARNING] slave queue is full with transactions. discard new transaction request
[WARNING] slave queue is full with transactions. discard new transaction request
[WARNING] slave queue is full with transactions. discard new transaction request
[WARNING] slave queue is full with transactions. discard new transaction request
[WARNING] slave queue is full with transactions. discard new transaction request
[WARNING] slave queue is full with transactions. discard new transaction request
[WARNING] slave queue is full with transactions. discard new transaction request
[WARNING] slave queue is full with transactions. discard new transaction request
[WARNING] slave queue is full with transactions. discard new transaction request
[WARNING] slave queue is full with transactions. discard new transaction request
[WARNING] slave queue is full with transactions. discard new transaction request
[WARNING] slave queue is full with transactions. discard new transaction request
[WARNING] slave queue is full with transactions. discard new transaction request
[WARNING] slave queue is full with transactions. discard new transaction request
[WARNING] slave queue is full with transactions. discard new transaction request
[WARNING] slave queue is full with transactions. discard new transaction request
[WARNING] slave queue is full with transactions. discard new transaction request
[WARNING] slave queue is full with transactions. discard new transaction request
[WARNING] slave queue is full with transactions. discard new transaction request
[WARNING] slave queue is full with transactions. discard new transaction request
[WARNING] slave queue is full with transactions. discard new transaction request
[WARNING] slave queue is full with transactions. discard new transaction request
[WARNING] slave queue is full with transactions. discard new transaction request
[WARNING] slave queue is full with transactions. discard new transaction request
[WARNING] slave queue is full with transactions. discard new transaction request
[WARNING] slave queue is full with transactions. discard new transaction request
[WARNING] slave queue is full with transactions. discard new transaction request

The program I am running is this one

#include <ESP32DMASPISlave.h>

// SPI SOUND
#define SPI_SND_DI_PIN    (21)              // DIN
#define SPI_SND_DO_PIN    (-1)              // NA
#define SPI_SND_CLK_PIN   (26)              // BCLK
#define SPI_SND_CS_PIN    (25)              // LRCLK

#define SAMPLE_FREQ 8000

#define SND_SPI_PORT        (HSPI)
#define SND_SPI_MODE        (SPI_MODE3)
#define SND_SPI_DMA         (SPI_DMA_CH_AUTO) // (2)

#define SND_BUFFER_SIZE     (sizeof(int32_t)) // buffer size/queue in bytes
#define SND_SPI_NQUEUES     (16)              // power of 2
#define SND_SPI_NQBUFS      (4)               // power of 2
#define SND_SPI_NQMAX       (SND_SPI_NQUEUES*SND_SPI_NQBUFS)

ESP32DMASPI::Slave slave;

uint8_t *spi_slave_rx_buf[ SND_SPI_NQMAX ];
int32_t spi_slave_rx_buf32[ SND_SPI_NQUEUES ];

constexpr uint8_t CORE_TASK_SPI_SLAVE {0};
static TaskHandle_t task_handle_spi_dma = NULL;

void set_buffer()
{
    // to use DMA buffer, use these methods to allocate buffer
    for ( unsigned n = 0; n < SND_SPI_NQMAX; n++)
    {
        spi_slave_rx_buf[n] = slave.allocDMABuffer( SND_BUFFER_SIZE );
        memset( spi_slave_rx_buf[n], 0, SND_BUFFER_SIZE );
    }
}

void task_spi_dma( void* pvParameters )
{
    uint32_t  p_qrx_wr = 0;
    uint32_t  p_qrx_rd = 0;
    uint32_t  p_qoccupied = 0;

    while (1)
    {
        // Set new queue
        if ( p_qoccupied < (SND_SPI_NQMAX-SND_SPI_NQUEUES) )
        {
            for ( unsigned i=0; i<SND_SPI_NQUEUES; i++ )
            {
                slave.queue( spi_slave_rx_buf[p_qrx_wr], SND_BUFFER_SIZE );
                p_qoccupied++;
                p_qrx_wr++;
                p_qrx_wr %= SND_SPI_NQMAX;
            }
        }

        // if slave has received transaction data, available() returns size of received transactions
        if ( slave.available() >= (SND_SPI_NQUEUES) ) // if queue is ready
        {
            for ( unsigned i=0; i<SND_SPI_NQUEUES; i++ )
            {
                memcmp( &spi_slave_rx_buf32[i], spi_slave_rx_buf[p_qrx_rd], SND_BUFFER_SIZE );
                p_qrx_rd++;
                p_qrx_rd %= SND_SPI_NQMAX;
                slave.pop();
                p_qoccupied--;
            }
        }
        // if there are other task, give those some time as well...
        vTaskDelay(1);
    }
}

void setup()
{
    pinMode( SPI_SND_CS_PIN, INPUT );
    Serial.begin( 115200 );
    printf( "Queue Buffers : %d\n", SND_SPI_NQMAX );
    printf( "Queue Size     : %d bytes\n", SND_BUFFER_SIZE );
    printf( "Queue Space    : %d bytes\n", SND_BUFFER_SIZE * SND_SPI_NQMAX );

    set_buffer();

    delay( 500 );

    slave.setDMAChannel( SND_SPI_DMA );  // 1 or 2 only
    slave.setDataMode( SND_SPI_MODE );
    slave.setMaxTransferSize( SND_BUFFER_SIZE );
    slave.setQueueSize( SND_SPI_NQMAX );   // transaction queue size

//  bool begin(const uint8_t spi_bus = HSPI, const int8_t sck = -1, const int8_t miso = -1, const int8_t mosi = -1, const int8_t ss = -1);
    slave.begin( SND_SPI_PORT, SPI_SND_CLK_PIN, SPI_SND_DO_PIN, SPI_SND_DI_PIN, SPI_SND_CS_PIN );  // default SPI is HSPI

    xTaskCreatePinnedToCore(
        task_spi_dma,
        "task_spi_dma",
        2048,
        NULL,
        2,
        &task_handle_spi_dma,
        CORE_TASK_SPI_SLAVE );
}

void loop() {
    // Never ending story
}

hideakitai commented 2 years ago

total amount of queues occupied is the sum of .available() and .remained().

No. For example, if you want to transfer 32 bytes from/to master to/from slave,

slave queues transaction and wait for master's transfer
- remained(): 1
- available(): 0
- size(): 0
master transfer transaction 32 bytes and succeeded
slave transaction is executed at the same time
- remained(): 0
- available(): 1
- size(): 32

Reading the codes may tell you more than what I say.

https://github.com/hideakitai/ESP32DMASPI/blob/8a370219245480403b24a5d93f593635ccb69b31/ESP32DMASPISlave.h#L149-L162

For more info about SPI, please refer following.

https://learn.sparkfun.com/tutorials/serial-peripheral-interface-spi/all https://en.wikipedia.org/wiki/Serial_Peripheral_Interface?utm_source=pocket_mylist

JBSchueler commented 2 years ago

Thanks for your reply.

I do understand what you are telling me. I also know the principles of SPI ;) My struggling is the queue part.

You are right about how remained() and available() works. queue() increases remained() and pop() decreases available().

slave queues transaction and wait for master's transfer
- remained(): 1
- available(): 0
- size(): 0
master transfer transaction 32 bytes and succeeded
slave transaction is executed at the same time
- remained(): 0
- available(): 1
- size(): 32
slave pops transaction
- remained(): 0
- available(): 0
- size(): 0

What I trying to figure out the queue. When setting up the DMA parameters you have to set the queue size. https://github.com/hideakitai/ESP32DMASPI/blob/8a370219245480403b24a5d93f593635ccb69b31/ESP32DMASPISlave.h#L186

What I understand now is that I am mixing up queue size ( the maximum amount of queues ) and DMA data space. It has nothing to do with the DMA data space I allocate via allocDMABuffer(). Also remained() and available() are independent, although I have to figure out why they drift compared to p_qoccupied.

It gets clear while I am writing this ;)

Thanks a lot @hideakitai !

JBSchueler commented 2 years ago

I changed my code to keep the queues filled up to the max while the is being read when there are at least SND_SPI_NQUEUES available. So far so good, I don't get errors anymore :)

But... after a very short period of time the behavior is incorrect, no new queues are added anymore.

#include <ESP32DMASPISlave.h>

// SPI SOUND
#define SPI_SND_DI_PIN    (21)              // DIN
#define SPI_SND_DO_PIN    (-1)              // NA
#define SPI_SND_CLK_PIN   (26)              // BCLK
#define SPI_SND_CS_PIN    (25)              // LRCLK

#define T0_PIN            (15)              // Test0 Pin
#define T1_PIN            (04)              // Test1 Pin
#define T2_PIN            (18)              // Test2 Pin
#define T3_PIN            (19)              // Test3 Pin

#define SAMPLE_FREQ 8000

#define SND_SPI_PORT        (HSPI)
#define SND_SPI_MODE        (SPI_MODE3)
#define SND_SPI_DMA         (SPI_DMA_CH_AUTO) // (2)

#define SND_BUFFER_SIZE     (sizeof(int32_t)) // buffer size/queue in bytes
#define SND_SPI_NQUEUES     (64)             // power of 2
#define SND_SPI_NQBUFS      (4)               // power of 2
#define SND_SPI_NQMAX       (SND_SPI_NQUEUES*SND_SPI_NQBUFS)

ESP32DMASPI::Slave slave;

uint8_t *spi_slave_rx_buf[ SND_SPI_NQMAX ];
int32_t spi_slave_rx_buf32[ SND_SPI_NQUEUES ];

constexpr uint8_t CORE_TASK_SPI_SLAVE {0};
static TaskHandle_t task_handle_spi_dma = NULL;

void set_buffer()
{
    // to use DMA buffer, use these methods to allocate buffer
    for ( unsigned n = 0; n < SND_SPI_NQMAX; n++)
    {
        spi_slave_rx_buf[n] = slave.allocDMABuffer( SND_BUFFER_SIZE );
        memset( spi_slave_rx_buf[n], 0, SND_BUFFER_SIZE );
    }
}

void task_spi_dma( void* pvParameters )
{
    uint32_t  p_qrx_wr = 0;
    uint32_t  p_qrx_rd = 0;
    uint32_t  p_qoccupied = 0;

    while (1)
    {
        // Set new queue
        while ( slave.remained() < (SND_SPI_NQMAX-SND_SPI_NQUEUES) )
        {
            digitalWrite( T0_PIN, HIGH );
            slave.queue( spi_slave_rx_buf[p_qrx_wr], SND_BUFFER_SIZE );
            p_qoccupied++;
            p_qrx_wr++;
            p_qrx_wr %= SND_SPI_NQMAX;
            digitalWrite( T0_PIN, LOW );
        }

        // if slave has received transaction data, available() returns size of received transactions
        if ( slave.available() >= (SND_SPI_NQUEUES) ) // if queue is ready
        {
            digitalWrite( T1_PIN, HIGH );
            for ( unsigned i=0; i<SND_SPI_NQUEUES; i++ )
            {
                memcmp( &spi_slave_rx_buf32[i], spi_slave_rx_buf[p_qrx_rd], SND_BUFFER_SIZE );
                p_qrx_rd++;
                p_qrx_rd %= SND_SPI_NQMAX;
                slave.pop();
                p_qoccupied--;
            }
            digitalWrite( T1_PIN, LOW );
        }
        // if there are other task, give those some time as well...
        vTaskDelay(1);
    }
}

void setup()
{
    pinMode( SPI_SND_CS_PIN, INPUT );

    pinMode( T0_PIN, OUTPUT );
    digitalWrite( T0_PIN, LOW );
    pinMode( T1_PIN, OUTPUT );
    digitalWrite( T1_PIN, LOW );
    pinMode( T2_PIN, OUTPUT );
    digitalWrite( T2_PIN, LOW );
    pinMode( T3_PIN, OUTPUT );
    digitalWrite( T3_PIN, LOW );

    Serial.begin( 115200 );
    printf( "Queue Buffers : %d\n", SND_SPI_NQMAX );
    printf( "Queue Size     : %d bytes\n", SND_BUFFER_SIZE );
    printf( "Queue Space    : %d bytes\n", SND_BUFFER_SIZE * SND_SPI_NQMAX );

    set_buffer();

    delay( 500 );

    slave.setDMAChannel( SND_SPI_DMA );  // 1 or 2 only
    slave.setDataMode( SND_SPI_MODE );
    slave.setMaxTransferSize( SND_BUFFER_SIZE );
    slave.setQueueSize( SND_SPI_NQMAX );   // transaction queue size

//  bool begin(const uint8_t spi_bus = HSPI, const int8_t sck = -1, const int8_t miso = -1, const int8_t mosi = -1, const int8_t ss = -1);
    slave.begin( SND_SPI_PORT, SPI_SND_CLK_PIN, SPI_SND_DO_PIN, SPI_SND_DI_PIN, SPI_SND_CS_PIN );  // default SPI is HSPI

    xTaskCreatePinnedToCore(
        task_spi_dma,
        "task_spi_dma",
        2048,
        NULL,
        2,
        &task_handle_spi_dma,
        CORE_TASK_SPI_SLAVE );
}

void loop() {
    // Never ending story
}

The terminal output looks like this

ets Jun  8 2016 00:22:57

rst:0x1 (POWERON_RESET),boot:0x33 (SPI_FAST_FLASH_BOOT)
flash read err, 1000
ets_main.c 371 
ets Jun  8 2016 00:22:57

rst:0x10 (RTCWDT_RTC_RESET),boot:0x33 (SPI_FAST_FLASH_BOOT)
configsip: 0, SPIWP:0xee
clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00
mode:DIO, clock div:1
load:0x3fff0030,len:1324
ho 0 tail 12 room 4
load:0x40078000,len:13508
load:0x40080400,len:3604
entry 0x400805f0
Queue Buffers : 256
Queue Size     : 4 bytes
Queue Space    : 1024 bytes

T0 shows queue pushes T1 shows reading actions

You can see the queue is being filled up at the beginning and when filled, it only adds new queue if needed. The reason why is done every 1ms is because of the vTaskDelay(1)

Every 1ms it has 8 transactions (8kHz sampling rate)

But ater 740ms no new queues are added.... while the reading continues ( the 2 pulses of T1) until there is no data available anymore.

Weird because the program never exceeds the maximum number of allowed queues.

I added a print statement just after I copied the data from SPI DMA memory

                p_qoccupied--;
            }
            printf( "%d\t%d\n", slave.remained(), slave.available() );
            digitalWrite( T1_PIN, LOW );
        }
        // if there are other task, give those some time as well...
        vTaskDelay(1);

Now it runs for more than 4 minutes without crashing but.... I do get warnings that the maximum queue size has been reached. 22 times as can been seen in this logfile. queue_result.txt The remained queue value is 192 which is below the 256 which has been set in the program.

At this point I am lost... no clue why this can happen.

JBSchueler commented 2 years ago

I just captured some kind of bug...

queue_result_bug.txt

The only modification I made is filling the queue to 1/2 instead of 3/4.

    {
        // Set new queue
        while ( slave.remained() < (SND_SPI_NQMAX/2) )
        {

In a very short time the program stopped. The terminal showed a strange value of remained()

128 2
128 2
128 2
128 2
128 2
128 2
128 2
-858993335  2
-858993398  1

JBSchueler commented 2 years ago

Did some more debugging... I modified ESP32DMASPISlave.h by adding some printf.

    size_t remained() const {
        size_t ret = transactions.size();
        if ( ret < 0 ) printf( "%d\n", ret );
        if ( ret > if_cfg.queue_size ) printf( "%d\n", ret );
//        return transactions.size();
        return ret;
    }

Explained: when transactions.size() is negative or is bigger than maximum queue size the value will be printed.

result:

ets Jun  8 2016 00:22:57

rst:0x1 (POWERON_RESET),boot:0x33 (SPI_FAST_FLASH_BOOT)
flash read err, 1000
ets_main.c 371 
ets Jun  8 2016 00:22:57

rst:0x10 (RTCWDT_RTC_RESET),boot:0x33 (SPI_FAST_FLASH_BOOT)
configsip: 0, SPIWP:0xee
clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00
mode:DIO, clock div:1
load:0x3fff0030,len:1324
ho 0 tail 12 room 4
load:0x40078000,len:13508
load:0x40080400,len:3604
entry 0x400805f0
Queue Buffers  : 256
Queue Size     : 4 bytes
Queue Space    : 1024 bytes
-6
-13
-21
-29
-37
-45
-53
-61
-69
-77
-85
-93
-101
-109
-117
-125
-131
-131
-131
-131
-131
-131

To me this looks like there is something going wrong with deque...

hideakitai commented 2 years ago

size_t won't be negative. Sorry, currently, I don't have time to debug your program as volunteer work. Could you summarize the problem and open another issue if you found it in this library?

Closing this issue named "what does available() represents when using queues?".

hideakitai commented 2 years ago

If you found a problem with this library, please feel free to open another issue.

JBSchueler commented 2 years ago

@hideakitai No problem...

Current state: ESP32 as SP Slave. using function queue()

.remained() gives incorrect value over time. This can result in

new transfer queue
crashing the program and reset of ESP32
watchdog timeout

It seems that the library deque is the "bad" guy by giving an incorrect value of .size()

JBSchueler commented 2 years ago

When is dequeue size call not thread safe?

So using deque in RTOS with DMA certainly can cause issues... :(

hideakitai / ESP32DMASPI

what does available() represents when using queues? #15