XMM Cache and Library SD Access - both hog the SPI/I2C bus

[GoogleCodeExporter]

Most of the XMM cache drivers and the library SD card access routines use 
either the SPI or I2C bus.  These agents just grab the bus whenever they need 
it, with no regard for any asynchronous agents that may need access to the bus.

In general, if all access to these buses is via a C thread in XMM mode and in 
hub memory, then there are no issues.  However there are two troublesome 
scenarios:

A) If two competing LMM C threads try to access the SPI bus (directly, or 
indirectly through the SD card routines).

B) If an asynchronous agent (such as a PASM cog) tries to access a SPI or I2C 
device (e.g. such as the SPI ADC on the C3 or the I2C RTC on the PDB), then 
that access will interfere with the XMM cache or the SD routines and very bad 
failures ensue.

Original issue reported on code.google.com by tjstefan...@charter.net on 2 Apr 2012 at 4:09

[GoogleCodeExporter]

Detailed Fix:  Each of the following files now supports the ability to use a 
lock to serialize access to its bus:

lib/drivers/sd_driver.s
loader/spin/c3_cache.spin
loader/spin/sd_cache.spin
loader/spin/eeprom_cache.spin

Note that loader/spin/sd_driver.spin also got these changes (though not 
strictly necessary) so it could stay in near 100% synchrony with 
lib/drivers/sd_driver.s.

Each of these drivers receives its lockId via a new extended command utilizing 
last unused slot in the extended function table, now called the 
"lock_set_handler".  Each driver functions nearly identically to its old 
behavior without the lock being set; the major difference being: the cache 
drivers don't grab the SPI bus via a "mov dira, spidir" during cache hits.  
When the lock is set, the drivers first wait until they receive the lock, then 
do their work on the SPI bus, then get off the bus and clear the lock.

The library now supports two additional calls:

kernel_use_lock(lockId) - instructs the kernel caching to use the specified 
lockId (note: the lock must be allocated by the caller using locknew()).

dfs_use_lock(lockId) - instructs the sd card driver to use the specified lockId 
(again, the lock must be allocated by the caller).

This way, everything functions exactly as today unless the main program 
instructs otherwise.  It is up to the program to determine whether 
kernel_use_lock and/or dfs_use_lock is necessary, and therefore the programmer 
must understand the kernel cache and sd driver structure.  Furthermore, if the 
programmer uses the lock in non-HUB code in an XMM program, then deadlock is 
quite probable.

Original comment by tjstefan...@charter.net on 2 Apr 2012 at 4:11

[GoogleCodeExporter]

In order to QA the cache/driver kernel and sd locks, use the following tests:

==============> Test 1: Kernel Cache Test <==============

In this test, we use a separate asynchronous cog running a program that uses 
both the I2C and SPI busses.  This test program is actually a degenerate test 
case; it is a loop that:

  A) Monitors P0, and if P0 is high, it requests the bus lock, and when it gets it, it ties the I2C and SPI bus lines high preventing their use.
  B) Then monitors P0, and if P0 is low it releases the I2C and SPI buses and releases the lock.

To run the tests, use a C3 board:

1) Compile the attached test_main.c and test.s

2) For each test, start with a jumper the C3's P0 pin on the Port A connector 
to 3.3V.

3) Make sure the program runs until it prints the following line and then stops 
soon after (depending on the test and the state of the cache):

Staring test cog with lock: N

(At this point the "VGA" light will also be lit.)

4) Remove the jumper to P0 or tie P0 to VSS/Ground.

Verify that the program continues running and "VGA" goes off or dims (remember 
that if P0 is unattached it could easily dither between 0 and 1).

5) Jumper P0 back to 3.3V.

Verify that the program stops again.

Catalog of Tests:

propeller-elf-gcc test_main.c test.s -Os -mlmm -o test.elf

propeller-load test.elf -b c3 -r -t       ' Note: This halts via explicit use 
of the lock - proving the test cog does indeed set/clear the lock

propeller-elf-gcc test_main.c test.s -Os -mxmmc -o test.elf

propeller-load test.elf -b eeprom -r -t   ' Tests eeprom_cache.spin
propeller-load test.elf -b c3f -r -t      ' Tests c3_cache.spin in flash-only 
mode
propeller-load test.elf -b c3f -r -t -l   ' Ditto
propeller-load test.elf -b c3 -r -t       ' Tests c3_cache.spin in flash + RAM 
mode (note less cache means less output before the halt)
propeller-load test.elf -b c3 -r -t -l    ' Ditto
propeller-load test.elf -b c3 -r -t -z    ' Tests sd_cache.spin

propeller-elf-gcc test_main.c test.s -Os -mxmm -o test.elf

propeller-load test.elf -b c3 -r -t       ' Tests c3_cache.spin in flash + RAM 
mode
propeller-load test.elf -b c3 -r -t -l    ' Ditto

==============> Test 2: SD Library Test <==============

In this test, we use the same asynchronous cog as in Test 1.  But this time we 
torture the library SD Driver.

Case 1: Compile the attached filetest.c and printf.c (both from demo/c3files, 
but with filetest.c slightly modified):

propeller-elf-gcc filetest.c printf.c test.s -Os -mlmm -o c3test.elf

propeller-load c3test.elf -b c3 -r -t

Make sure that ls, rm, cat, etc. work properly.
Then, issue the "whack" command which kicks off the test cog - the VGA light 
should light.
Depending on the cache state, some commands may work at this point (e.g. help), 
and even the keyboard input can be "out of cache".
However, verify that everything including all SD commands (e.g. ls, rm, etc.) 
will eventually halt until you remove the P0->3.3V jumper; then everything 
works normally.
Again, replacing the jumper should halt the cache and SD access and light the 
VGA light.

propeller-elf-gcc filetest.c printf.c test.s -Os -mxmmc -o c3test.elf

propeller-load c3test.elf -b eeprom -r -t   ' Tests eeprom_cache.spin
propeller-load c3test.elf -b c3f -r -t      ' Tests c3_cache.spin in flash-only 
mode
propeller-load c3test.elf -b c3f -r -t -l   ' Ditto
propeller-load c3test.elf -b c3 -r -t       ' Tests c3_cache.spin in flash + 
RAM mode
propeller-load c3test.elf -b c3 -r -t -l    ' Ditto
propeller-load c3test.elf -b c3f -r -t -z   ' Tests sd_cache.spin

propeller-elf-gcc filetest.c printf.c test.s -Os -mxmm -o c3test.elf

propeller-load c3test.elf -b c3 -r -t       ' Tests c3_cache.spin in flash + 
RAM mode
propeller-load c3test.elf -b c3 -r -t -l    ' Ditto

Case 2: Modify filetest.c - undefine C3_CARD, define 
SD_IS_USING_SD_CACHE_DRIVER, remove the either the dfs_use_lock or the 
kernel_use_lock call.

propeller-elf-gcc filetest.c printf.c test.s -Os -mxmmc -o c3test.elf

propeller-load c3test.elf -b c3 -r -t -z    ' Tests sd_cache.spin

This test proves that the both dfs_use_lock and kernel_use_lock are really the 
same thing in this one (degenerate) case.

==============> Bonus Tests <==============

1) Rerun any of the above tests and remove the call the kernel_use_lock (and/or 
dfs_use_lock) and notice that the program just doesn't work after starting the 
test cog, proving that the test cog does indeed jam the I2C and SPI buses.

2) Modify eeprom.cfg to have the C3 SD card pins, and run both Test 1's and 
Test 2's XMMC -l tests with the eeprom cache driver:

propeller-load test.elf -b eeprom -r -t -l

3) Try all the -l and -z tests with -e (making sure to leave 17K in the eeprom 
for the "eeprom -l -e" tests).

Original comment by tjstefan...@charter.net on 2 Apr 2012 at 4:12

Attachments:

• test_main.c
• test.s
• filetest.c
• printf.c

[GoogleCodeExporter]

Fixed in revision c7d0c7b78c2b

Original comment by tjstefan...@charter.net on 2 Apr 2012 at 4:15

[GoogleCodeExporter]

In thinking about my solution, I realized that there was one scenario that I 
did not test: Calling dfs_use_lock() before calling dfs_mount().  Indeed, after 
testing this, I found that the initialization code in the SD driver and cache 
could end up freeing a lock it didn't hold.

The latest fixes affect sd_driver.s, sd_driver.spin, and sd_cache.spin.  The 
test for these is as follows:

Add a call to "Whack();" to line 537 of filetest.c.

Then, repeat all the Test 2 tests (and Bonus tests if desired) with the "early 
whack".  Note that the XMMC c3 test cache test displays a "funny" corner case: 
The stalled cache miss occurs between the CNT + CLKFREQ and calculation and the 
call to waitcnt - if you wait ~60 seconds the program proceeds normally.

Note that if you don't apply the fixes from the repository and try the "early 
whack" tests, then several of the tests just freeze.

NB: I re-ran the entire suite of tests (Test 1, Test 2, Early Whack, and Bonus) 
with my new fixes.

Original comment by tjstefan...@charter.net on 3 Apr 2012 at 6:15

[GoogleCodeExporter]

The additional fixes are in revision a790439b375b.

Original comment by tjstefan...@charter.net on 3 Apr 2012 at 6:19