GT.M V6.3-003 (which is merged in the YottaDB mainline from r1.20 onwards) introduced a
new feature as part of GTM-8735. Below is the release note for that feature.
MUPIP SET -{FILE|REGION} recognizes the -[NO]READ_ONLY qualifier to indicate whether GT.M
should treat an MM access method segment as read only for all users, including root. This
designation augments UNIX authorizations and prevents any state updates that normally
might require an operational action for a database with no current accessing (attached)
processes. The GT.M help databases have -READ_ONLY set by default. Previously, a database
such as the gtmhelp database in the GT.M distribution typically never received a data update
but nevertheless could require a ROLLBACK, RECOVER or RUNDOWN to ensure a proper at-rest
state. MUPIP emits an error on attempts to set -READ_ONLY on databases with the BG access
method, or to set the access method to BG on databases with -READ_ONLY set. (GTM-8735)
Various issues were found in the above change during internal testing and code review at YottaDB.
1) The help database has the READ_ONLY flag set as well as read-only permissions on the
file to all users (with the default install of YottaDB/GT.M, root owns the files in the
install directory). In this setup, if a process opens the help database (e.g. ZHELP, $ZPEEK,
$$^%PEEKBYNAME etc.) and concurrently an argumentless MUPIP RUNDOWN runs, it would incorrectly
delete the process-private semaphore of the help database created by the live process. Later
when the process that opened the help database halts, it would get a DBFILERR/GVRUNDOWN
error (with error detail "Unable to remove semaphore"). This is because the semaphore
counter was not incremented at "db_init" time by the process that opened the help database
(because it was a file with read-only permissions). The database rundown logic has special
logic based on how many processes with read-write access to the database are active and not
incrementing the semaphore counter for the read-only-permissions case helps with that. The
process with read-only-permission would anyways open database shared memory and based on
the count of # of processes having the shared memory open, the rundown logic can determine
if we are the last writer process vs we are the last reader/writer process. But files with
read-only permissions that also have the READ_ONLY characteristic set in the db file header
are different. There is no shared memory created in this case. And each process that opens
this database has its own process private memory (to simulate database shared memory). In
that case, there is only one process at all times accessing the database (as far as each
process ic concerned). Therefore it is okay to increment the semaphore counter in that case
thereby preventing the argumentless mupip rundown from incorrectly removing the semaphore.
2) If one tries accessing the help database (with read-only permissions) after running an
operational command that requires standalone access to the database file (e.g. MUPIP INTEG
-FILE $gtm_dist/gtmhelp.dat), one gets a CRITSEMFAIL/SYSCALL/ENO22 error. From that point on,
the help database is inaccessible. The only workaround to get it back and running was to do
a MUPIP SET -NOREAD_ONLY -FILE $gtm_dist/gtmhelp.dat followed by a MUPIP SET -READ_ONLY -FILE
$gtm_dist/gtmhelp.dat (both of which require root access). This is because "mu_rndwn_file()"
(the function invoked to get standalone access) could still end up using gtmsecshr to write
the semid/shmid to the db file header. But a READ_ONLY type of database should never be
written to the file header (to avoid REQRUNDOWN/REQROLLBACK errors) and so the fix was to not
use gtmsecshr in mu_rndwn_file() to write to the file header in case of a READ_ONLY database.
3) If a process accesses the help database and halts cleanly, the ftok semaphore (ipcs -a
shows a key of the form 0x2b......) was left around. It turns out this was because the database
rundown code, gds_rundown(), had special code to return right away after removing the private
semaphore in the very early stages of the function. This meant the ftok semaphore release
and remove which happens much later in gds_rundown() did not happen for a READ_ONLY type of
database. That is now fixed by reworking the flow so most of the rundown logic is skipped
for the READ_ONLY case but the return happens at the end just like for the non-READ_ONLY case.
4) While reworking the gds_rundown flow, an additional issue was noticed. And that was
with the STATSDB feature. A database file by default has statistics sharing enabled. If
more than one process does a VIEW "STATSHARE" (or enables the gtm_statshare environment
variable) and opens a file with READ_ONLY characteristic (it does not matter if it has
read-only or read-write permissions) at the same time, the last process to halt issues
a CRITSEMFAIL/SYSCALL/NOTALLDBRNDWN/GVRUNDOWN error. This was because the two processes
were operating without database shared memory but maintaining one statistics database file
(*.gst) which created an out-of-design situation as each would try to delete the file
and its associated semaphore as part of halting. The fix for this was to disallow STATS on
database files that have the READ_ONLY feature turned on. mupip_set_file() has the associated
changes for this fix. A READONLYNOSTATS error is now issued if one tries to enable READ_ONLY
on a database that has STATS turned on or if one tries to enable STATS on a database that
has READ_ONLY turned on or if one tries to enable both at the same time.
5) A MUPIP SET -NOREAD_ONLY -ACCESS_METHOD=BG incorrectly issues a READONLYNOBG error. The
access method specified is BG which is not compatible with READ_ONLY but should be compatible
with the specified NOREAD_ONLY flag. This is now fixed to not issue an error.
6) A MUPIP SET -NOREAD_ONLY turned off the read_only characteristic of the database
file. But it did not ensure that no other process was accessing the database. This meant
that processes which started out when the READ_ONLY characteristic was turned on in the
database file header could be running while the MUPIP SET -NOREAD_ONLY command changed this
characteristic. If one runs further MUPIP commands (e.g. to change access method from MM to
BG etc.), it is possible for the process that still thinks the database file is READ_ONLY
to incorrectly issue errors (e.g. GVGETFAIL, GVDATAFAIL etc.) while trying to read from
a database that is otherwise good but out of date with respect to this process. It is not
clear exactly what other race/timing issues are possible. In any case, this is now fixed
by all processes that open a READ_ONLY database getting a shared lock on the database file
and a MUPIP SET -NOREAD_ONLY command gets an exclusive lock on the same database file. This
way the MUPIP SET command would return a READONLYLKFAIL error if at least one other process
is still accessing the database with the READ_ONLY characteristic set. Additionally, if a
MUPIP command that requires standalone access (e.g. MUPIP SET -NOREAD_ONLY) is currently
running, an attempt by another command to access the database as a READ_ONLY database will
also issue a READONLYLKFAIL error.
7) Additionally, various asserts were added to ensure design constraints are met in dbg builds.
GT.M V6.3-003 (which is merged in the YottaDB mainline from r1.20 onwards) introduced a new feature as part of GTM-8735. Below is the release note for that feature.
MUPIP SET -{FILE|REGION} recognizes the -[NO]READ_ONLY qualifier to indicate whether GT.M should treat an MM access method segment as read only for all users, including root. This designation augments UNIX authorizations and prevents any state updates that normally might require an operational action for a database with no current accessing (attached) processes. The GT.M help databases have -READ_ONLY set by default. Previously, a database such as the gtmhelp database in the GT.M distribution typically never received a data update but nevertheless could require a ROLLBACK, RECOVER or RUNDOWN to ensure a proper at-rest state. MUPIP emits an error on attempts to set -READ_ONLY on databases with the BG access method, or to set the access method to BG on databases with -READ_ONLY set. (GTM-8735)
Various issues were found in the above change during internal testing and code review at YottaDB.
1) The help database has the READ_ONLY flag set as well as read-only permissions on the file to all users (with the default install of YottaDB/GT.M, root owns the files in the install directory). In this setup, if a process opens the help database (e.g. ZHELP, $ZPEEK, $$^%PEEKBYNAME etc.) and concurrently an argumentless MUPIP RUNDOWN runs, it would incorrectly delete the process-private semaphore of the help database created by the live process. Later when the process that opened the help database halts, it would get a DBFILERR/GVRUNDOWN error (with error detail "Unable to remove semaphore"). This is because the semaphore counter was not incremented at "db_init" time by the process that opened the help database (because it was a file with read-only permissions). The database rundown logic has special logic based on how many processes with read-write access to the database are active and not incrementing the semaphore counter for the read-only-permissions case helps with that. The process with read-only-permission would anyways open database shared memory and based on the count of # of processes having the shared memory open, the rundown logic can determine if we are the last writer process vs we are the last reader/writer process. But files with read-only permissions that also have the READ_ONLY characteristic set in the db file header are different. There is no shared memory created in this case. And each process that opens this database has its own process private memory (to simulate database shared memory). In that case, there is only one process at all times accessing the database (as far as each process ic concerned). Therefore it is okay to increment the semaphore counter in that case thereby preventing the argumentless mupip rundown from incorrectly removing the semaphore.
2) If one tries accessing the help database (with read-only permissions) after running an operational command that requires standalone access to the database file (e.g. MUPIP INTEG -FILE $gtm_dist/gtmhelp.dat), one gets a CRITSEMFAIL/SYSCALL/ENO22 error. From that point on, the help database is inaccessible. The only workaround to get it back and running was to do a MUPIP SET -NOREAD_ONLY -FILE $gtm_dist/gtmhelp.dat followed by a MUPIP SET -READ_ONLY -FILE $gtm_dist/gtmhelp.dat (both of which require root access). This is because "mu_rndwn_file()" (the function invoked to get standalone access) could still end up using gtmsecshr to write the semid/shmid to the db file header. But a READ_ONLY type of database should never be written to the file header (to avoid REQRUNDOWN/REQROLLBACK errors) and so the fix was to not use gtmsecshr in mu_rndwn_file() to write to the file header in case of a READ_ONLY database.
3) If a process accesses the help database and halts cleanly, the ftok semaphore (ipcs -a shows a key of the form 0x2b......) was left around. It turns out this was because the database rundown code, gds_rundown(), had special code to return right away after removing the private semaphore in the very early stages of the function. This meant the ftok semaphore release and remove which happens much later in gds_rundown() did not happen for a READ_ONLY type of database. That is now fixed by reworking the flow so most of the rundown logic is skipped for the READ_ONLY case but the return happens at the end just like for the non-READ_ONLY case.
4) While reworking the gds_rundown flow, an additional issue was noticed. And that was with the STATSDB feature. A database file by default has statistics sharing enabled. If more than one process does a VIEW "STATSHARE" (or enables the gtm_statshare environment variable) and opens a file with READ_ONLY characteristic (it does not matter if it has read-only or read-write permissions) at the same time, the last process to halt issues a CRITSEMFAIL/SYSCALL/NOTALLDBRNDWN/GVRUNDOWN error. This was because the two processes were operating without database shared memory but maintaining one statistics database file (*.gst) which created an out-of-design situation as each would try to delete the file and its associated semaphore as part of halting. The fix for this was to disallow STATS on database files that have the READ_ONLY feature turned on. mupip_set_file() has the associated changes for this fix. A READONLYNOSTATS error is now issued if one tries to enable READ_ONLY on a database that has STATS turned on or if one tries to enable STATS on a database that has READ_ONLY turned on or if one tries to enable both at the same time.
5) A MUPIP SET -NOREAD_ONLY -ACCESS_METHOD=BG incorrectly issues a READONLYNOBG error. The access method specified is BG which is not compatible with READ_ONLY but should be compatible with the specified NOREAD_ONLY flag. This is now fixed to not issue an error.
6) A MUPIP SET -NOREAD_ONLY turned off the read_only characteristic of the database file. But it did not ensure that no other process was accessing the database. This meant that processes which started out when the READ_ONLY characteristic was turned on in the database file header could be running while the MUPIP SET -NOREAD_ONLY command changed this characteristic. If one runs further MUPIP commands (e.g. to change access method from MM to BG etc.), it is possible for the process that still thinks the database file is READ_ONLY to incorrectly issue errors (e.g. GVGETFAIL, GVDATAFAIL etc.) while trying to read from a database that is otherwise good but out of date with respect to this process. It is not clear exactly what other race/timing issues are possible. In any case, this is now fixed by all processes that open a READ_ONLY database getting a shared lock on the database file and a MUPIP SET -NOREAD_ONLY command gets an exclusive lock on the same database file. This way the MUPIP SET command would return a READONLYLKFAIL error if at least one other process is still accessing the database with the READ_ONLY characteristic set. Additionally, if a MUPIP command that requires standalone access (e.g. MUPIP SET -NOREAD_ONLY) is currently running, an attempt by another command to access the database as a READ_ONLY database will also issue a READONLYLKFAIL error.
7) Additionally, various asserts were added to ensure design constraints are met in dbg builds.