Large
SGA On Linux
On Linux the total SGA size is limited to around 2Gig. There are
several ways to allocate more memory than this, but the most robust and
scalable method requires the use
of a shared memory file system (shmfs). The procedure presented in this
article assumes you are using RedHat Advanced Server (AS) 2.1 with the
enterprise kernel
which supports Page Address Extension (PAE).
Create a shared
memory file system (shmfs)
The
shmfs is a memory file system so it can be as large as the maximum
allowable virtual memory supported by Red Hat Linux AS2.1, currently 16
GB, although the enterprise kernel theoretically supports up to 64 GB
of RAM.
The shmfs is created using the following command as the root user:
mount -t shm shmfs -o size=3g /dev/shm
The shared memory file system can be mounted automatically by adding
the following line into /etc/fstab file:
shmfs /dev/shm shm size=3g 0 0
In the above example I've created the shmfs with a size of 3G as that
is the size of the buffer cache I am planning to use. The other
elements of the SGA
are placed in regular memory, not this shared memory file system, so
they should not be included when deciding on the size of the shmfs. It
is advisable to
size this slightly bigger than the actual size needed, but in this
example I've used a 3G shmfs for a 3G buffer cache.
Enabling big
pages
Big pages are enabled by adding the bigpages=xMB
to the
relevant kernel entry in the boot loader file /boot/grub/grub.conf
file, where "x" is
calculated as follows:
(Total SGA size in Gig) x 1024
Then round this value to the nearest hundredth. So for a 4G SGA we
would do the following:
4 x 1024 = 4096 = 4100
So the /boot/grub/grub.conf file entry might look like this:
kernel /vmlinuz-2.4.9-e.40enterprise ro root=/dev/cciss/c0d0p2 bigpages=4100MB
With this entry saved the system should be rebooted. Once the system is
available you must perform the following command as the root user:
echo 2 > /proc/sys/kernel/shm-use-bigpages
Alternatively you can add the following entry into the /etc/sysctl.conf
file so this value persists between reboots:
kernel.shm-use-bigpages = 2
Setting the
SHMMAX value
The shmmax value should be set at half the physical memory up to a
maximum of 4294967295. For a server with 6G of memory we can set this
value to 3G (half physical memory)
using the following command as the root user:
echo 3221225472 > /proc/sys/kernel/shmmax
Alternatively it can be set in the /etc/sysctl.conf file with the
following entry:
kernel.shmmax = 3221225472
The contents of your /etc/sysctl.conf file may look something like this:
kernel.shmmax = 3221225472
kernel.shmmni = 4096
kernel.shmall = 2097152
kernel.sem = 1000 32000 100 150
fs.file-max = 65536
net.ipv4.ip_local_port_range = 1024 65000
kernel.shm-use-bigpages = 2
Alterations to the /etc/sysctl.conf file can be applied without a
reboot by issuing the following command as root:
/sbin/sysctl -p
Instance
Parameters
Some instance parameter changes are necessary to allow the Oracle
instance to use the shared memory file system. The spfile parameters
can be manipulated using the ALTER SYSTEM SET command in a running
instance, or by modifying the spfile contents offline:
-- Change the parameter value in the spfile directly.
ALTER SYSTEM SET parameter = value SCOPE=spfile
-- Create a pfile with the contents of the current spfile.
CREATE PFILE='/tmp/pfile' FROM SPFILE;
-- Manually manipulate the contents of the pfile.
-- Recreate the spfile from the amended pfile.
CREATE SPFILE FROM PFILE='/tmp/pfile';
The following parameter should be added to the spfile or pfile:
use_indirect_data_buffers=true
Also, any references to db_cache_size and db_xK_cache_size parameters
should be removed and replaced with the old style db_block_buffers
parameter entry:
# 3Gig for an 8K db_block_size.
db_block_buffers = 393216
This means that the multiple block size feature is not available when
using this method. Remember that the buffer cache is only one part of
the SGA.
For further information see:
- Metalink Note:211424.1 (second half)