Document ID: 100750.382
Title: Common Semaphore Problems
Creation Date: 23 August 1991
Last Revision Date: 23 August 1991
Revision Number: 01
Product: RDBMS
Product Version: v6.0 v7.0 v7.1
Platform: UNIX
Information Type: ADVISORY
Impact: MEDIUM
Abstract: This bulletin describes the common problems with the UNIX
Semaphore System that an Oracle DBA might encounter and
explains how they can be avoided.
Keywords: IPC;SEMAPHORE;ORA-7279;ORA-7251;ORA-7252;ORA-9702;SEMNS;
________________________________________________________________________________
COMMON SEMAPHORE PROBLEMS
OVERVIEW
This bulletin describes how Oracle uses the Unix semaphore system,
and the causes of Oracle errors involving Unix semaphores. These
errors often (but not always) indicate a resource problem on your
Unix machine that needs to be fixed before Oracle will be able to
run properly.
EXPLAINING THE UNIX SEMAPHORE SYSTEM
One of the important features Unix provides for inter-process
communication is the semaphore system. Semaphores are
integer-valued objects set aside by the operating system that can
be incremented or decremented atomically. They are designed to
allow processes to synchronize execution, by only allowing one
process to perform an operation on the semaphore at a time. The
other process(es) sleep until the semaphores values are either
incremented or set to 0, depending on the options used.
Semaphores are generally not used one at a time, so Unix uses the
concept of semaphore sets to make it easier to allocate and refer
to semaphores. When your Unix kernel is configured, the maximum
number of semaphores that will be available to the system is set.
Also set are the maximum number of semaphores per set, and the
maximum number of sets that can be allocated. These limits can
only be changed by remaking the Unix kernel and rebooting the
machine.
ORACLE'S NEED FOR SEMAPHORES
Oracle uses semaphores to control concurrency between all the
background processes (pmon, smon, dbwr, lgwr, and oracle shadows).
Semaphores are also used to control two-task communication between
the user process and shadow process if the fast (shared memory)
driver is used. And in the Unix ports based on MIPS RISC
processors, Oracle uses a special semaphore to perform basic
test & set functions that are not provided by the processor.
SEMAPHORES IN USE
Typing "ipcs -sb" will show you what semaphores are allocated to
your system at the moment. This will display all the semaphore
sets allocated, their identifying number, the owner, the number
of semaphores in each set, and more.
Occasionally, unexpected termination of Oracle processes will
leave semaphore resources locked. If your database is not
running, but "ipcs -sb" shows that semaphore sets owned by
oracle are still in use, then you need to de-allocate (free)
them. If you don't do this, then you may not be able to allocate
enough semaphores later to restart your database.
Freeing semaphore sets is done with the "ipcrm" command. For
each set that oracle has allocated, type "ipcrm -s ID" where ID
is the set number you see from the "ipcs" output. Semaphores can
also be freed by rebooting the system.
STARTING UP THE DATABASE
Oracle allocates all the semaphores it needs for the background
processes at the time of database startup. The init.ora parameter
"processes" is used to determine how many semaphores will be
allocated for Oracle's use. If Oracle needs more semaphores than
are allowed in one set, then more sets are grabbed.
A common error received while starting up the database is
ORA-7279: spcre: semget error, unable to get first semaphore set
Your system is trying to allocate the first set of semaphores,
containing either the maximum number of semaphores per set, or the
value specified by the "processes" parameter, whichever is less.
Either your system doesn't have enough semaphores configured, or
too many semaphores or semaphore sets are already allocated. In
this case, the first thing to check is if unused sets are hogging
up all the system's semaphores (see above SEMAPHORES IN USE section).
If that isn't the problem, you'll need to configure more semaphores
on your system. If you don't have any semaphores configured, or
every single one is currently allocated, you may see this error:
ORA-7251: spcre: semget error, could not allocate any semaphores
If the first set full of semaphores was succesfully allocated, but
the second could not be taken, this error will come up:
ORA-7252: spcre: semget error, could not allocate semaphores
Again, the problem can be resolved by making sure that the resources
for your machine are still being held by long-dead Oracle processes,
and otherwise that you have configured enough to begin with.
SHUTDOWN ABORT
When a shutdown abort is done, the Oracle background processes are
killed and the semaphore sets are freed, without waiting for the
user processes to finish what they're doing. The users finally
find out that the database is down when they send a request to the
database by incrementing (or decrementing) a semaphore. The attempt
to modify the semaphore fails, and the user's process (along with
its Oracle shadow process) dies. One or both of the following
errors will be displayed (sometimes only in the trace file):
ORA-7264: spwat: semop error, unable to decrement semaphore
ORA-7265: sppst: semop error, unable to increment semaphore
This is an effective (if ungraceful) way of letting the users know
that the database has been shut down with the abort option.
USING A UNIX MACHINE BASED ON MIPS RISC TECHNOLOGY
On Unix machines with MIPS RISC processors, such as the DEC
RISC Ultrix and MIPS machines, Oracle allocates an extra semaphore
at startup time for latching purposes. Since MIPS RISC chips don't
have test & set built into the hardware, this must be simulated in
sofware with a semaphore.
Oracle uses this semaphore as a latch whenever anyone makes a
connection to the database. But to make sure that no processes
retain a latch even if they are accidentally killed, a semaphore
undo structure is grabbed as well, so that any changes the process
makes to the semaphore can be undone in case of unexpeced process
death. So each process that connects needs to be able to allocate
an undo structure. If there aren't enough undo structures in the
system, the following error will result:
ORA-9702: sem_acquire: cannot acquire latch semaphore
The solution is to either wait until undo structures become
available, or reconfigure the system parameter controlling the
maximum number of undo structures available (SEMMNU).
USING THE FAST DRIVER
Every connection made to the database with the fast driver
requires semaphores to control who can read/write to the shared
memory buffer that is used for communication between the user and
Oracle shadow process. A semaphore set containing 3 semaphores is
allocated for each connection made with the fast driver. If the
system's semaphore resources (sets or total semaphores) are all
currently allocated, then connection will fail with
ORA-2721: osnseminit: cannot create semaphore set
and you will need to wait until semaphores are freed up to make a
connection with the fast driver.
One word of caution: unexpected termination of processes using
the fast driver will leave the semaphore set (and also a shared
memory segment) allocated. Since it's hard to tell which
semaphore set belongs to which process, you may have to wait
until everyone logs off the database (and perhaps for the database
to be shut down) before you can tell which semaphore set you need
to free.
PLANNING YOUR SYSTEM'S SEMAPHORE RESOURCES
Different Unix machines have different names for the semaphore
kernel parameters that you need to adjust, and different methods
to adjust them and rebuild your kernel. You'll need to consult
your System Administrator's Guide to determine exactly what
parameters to modify and how to modify them.
You can determine the total number of semaphores you'll need to
have available with the following equation:
proc (value of the "processes" parameter in your
initSID.ora file, controlling the total number of
connections allowed to the database)
+ test (test = 1 if your system uses a MIPS RISC processor,
= 0 otherwise)
+ 3*f (f = # of fast driver connections that will be made)
+ sys (number of semaphores required for non-Oracle,
i.e., system and/or other application, uses)
-------
tot (minimum number of semaphores you'll need to have
configured)
A good value for the parameter controlling the maximum number of
semaphores (usually called SEMMNS) on your system would be equal
to or larger than the figure you derived above.
Typical values for the maximum number of semaphores per set
(usually called SEMMSL) are 10-25. If you're going to be using the
fast driver for many connections, you'll need to have a large number
of sets available (and fewer semaphores per set to save on memory
usage). Otherwise, refer to Chapter 2 of your Installation and
User's Guide for information regarding how to size the semaphore
sets, and choose how many sets for your system (usually SEMMNI).
-------------------------------------------------------------------------------
Oracle WorldWide Customer Support