Home > oracle-install > Installation of Oracle RAC 11g R2 (11.2.0.2) –with ACFS file system as Database home

Installation of Oracle RAC 11g R2 (11.2.0.2) –with ACFS file system as Database home

This article is intended as a brief guide to installing Oracle Database 11g (11.2.0.2) Real Application Clusters (RAC) on RedHat Enterprise Linux X86_64.

Environment:

Each node requires at least Two 2 NIC cards one for Public IP and other for Private interconnect.

Node1 : racnode1.ukatru.com
Public IP Address : 192.168.2.52(racnode1.ukatru.com)
Virtual IP Address : 192.168.2.54(racnode1-vip.ukatru.com)

Node2:racnode2.ukatru.com

Public IP Address : 192.168.2.53(racnode2.ukatru.com)
Virtual IP Address : 192.168.2.55(racnode2-vip.ukatru.com)

We need 15GB disk on both nodes to install oracle Cluster software,ASM home and Database home.
 

Set Kernel Parameters

Add the following lines to the /etc/sysctl.conf file:

# Controls the maximum number of shared memory segments, in pages
kernel.shmall = 1073741824
# For 11g, recommended value for file-max is 6815744
fs.file-max = 6815744
# For 10g, uncomment ‘fs.file-max 327679’, comment other entries for this parameter and re-run sysctl -p
# fs.file-max:327679
kernel.msgmni = 2878
kernel.sem = 250 32000 100 142
kernel.shmmni = 4096
net.core.rmem_default = 262144
# For 11g, recommended value for net.core.rmem_max is 4194304
net.core.rmem_max = 4194304
# For 10g, uncomment ‘net.core.rmem_max 2097152’, comment other entries for this parameter and re-run sysctl -p
# net.core.rmem_max=2097152
net.core.wmem_default = 262144
# For 11g, recommended value for wmem_max is 1048576
net.core.wmem_max = 1048576
# For 10g, uncomment ‘net.core.wmem_max 262144’, comment other entries for this parameter and re-run sysctl -p
# net.core.wmem_max:262144
fs.aio-max-nr = 3145728
# For 11g, recommended value for ip_local_port_range is 9000 65500
net.ipv4.ip_local_port_range = 9000 65500
# For 10g, uncomment ‘net.ipv4.ip_local_port_range 1024 65000’, comment other entries for this parameter and re-run sysctl -p
# net.ipv4.ip_local_port_range:1024 65000
# Added min_free_kbytes 50MB to avoid OOM killer on EL4/EL5
vm.min_free_kbytes = 51200

[root@oral01 ~]# sysctl -p

Add the following lines to the /etc/security/limits.conf file:

oracle   soft   nofile    131072
oracle   hard   nofile    131072
oracle   soft   nproc    131072
oracle   hard   nproc    131072
oracle   soft   core    unlimited
oracle   hard   core    unlimited
oracle   soft   memlock    50000000
oracle   hard   memlock    50000000

grid soft nproc 2047
grid hard nproc 16384
grid soft nofile 1024
grid hard nofile 65536
Configure the iSCSI (Initiator) Service:
rpm -Uvh iscsi-initiator-utils-6.2.0.871-0.10.el5.x86_64.rpm
warning: iscsi-initiator-utils-6.2.0.871-0.10.el5.x86_64.rpm: Header V3 DSA signature: NOKEY, key ID 1e5e0159
Preparing…                ########################################### [100%]
   1:iscsi-initiator-utils  ########################################### [100%]

root@racnode1 Server]# service iscsid start
Turning off network shutdown. Starting iSCSI daemon:       [  OK  ]
                                                           [  OK  ]
[root@racnode1 Server]# chkconfig iscsid on
[root@racnode1 Server]# chkconfig iscsi on

[root@racnode2 Server]# service iscsid start
Turning off network shutdown. Starting iSCSI daemon:       [  OK  ]
                                                           [  OK  ]
[root@racnode2 Server]# chkconfig iscsid on
[root@racnode2 Server]# chkconfig iscsi on

Configure oracleasm :
[root@racnode1 tmp]# /etc/init.d/oracleasm configure
Configuring the Oracle ASM library driver.

This will configure the on-boot properties of the Oracle ASM library
driver.  The following questions will determine whether the driver is
loaded on boot and what permissions it will have.  The current values
will be shown in brackets (‘[]’).  Hitting without typing an
answer will keep that current value.  Ctrl-C will abort.

Default user to own the driver interface []: grid
Default group to own the driver interface []: dba
Start Oracle ASM library driver on boot (y/n) [n]: y
Scan for Oracle ASM disks on boot (y/n) [y]:
Writing Oracle ASM library driver configuration: done
Initializing the Oracle ASMLib driver:                     [  OK  ]
Scanning the system for Oracle ASMLib disks:               [  OK  ]

Create ASM Disks(You need to do this on one node only).

[root@racnode1 ~]# oracleasm createdisk ASM1 /dev/sdc1
Writing disk header: done
Instantiating disk: done

Logonto racnode2 and execute the following command to scan asm disks.

[root@racnode2 ~]# /etc/init.d/oracleasm scandisks
Scanning the system for Oracle ASMLib disks:               [  OK  ]
[root@racnode2 ~]# /etc/init.d/oracleasm listdisks
ASM1

$ ./runInstaller
Starting Oracle Universal Installer…

Checking Temp space: must be greater than 120 MB.   Actual 12428 MB    Passed
Checking swap space: must be greater than 150 MB.   Actual 3999 MB    Passed
Checking monitor: must be configured to display at least 256 colors.    Actual 16777216    Passed
Preparing to launch Oracle Universal Installer from /tmp/OraInstall2011-02-13_02-13-26PM. Please wait …$ You can find the log of this install session at:
 /u01/app/oraInventory/logs/installActions2011-02-13_02-13-26PM.log

$ ./runInstaller
Starting Oracle Universal Installer…

Checking Temp space: must be greater than 120 MB.   Actual 12357 MB    Passed
Checking swap space: must be greater than 150 MB.   Actual 3845 MB    Passed
Checking monitor: must be configured to display at least 256 colors.    Actual 16777216    Passed
Preparing to launch Oracle Universal Installer from /tmp/OraInstall2011-02-13_03-07-26PM. Please wait …$

[root@racnode1 ~]#
Running Oracle 11g root script…

The following environment variables are set as:
    ORACLE_OWNER= grid
    ORACLE_HOME=  /u01/app/11.2.0/grid

Enter the full pathname of the local bin directory: [/usr/local/bin]:
   Copying dbhome to /usr/local/bin …
   Copying oraenv to /usr/local/bin …
   Copying coraenv to /usr/local/bin …

Creating /etc/oratab file…
Entries will be added to the /etc/oratab file as needed by
Database Configuration Assistant when a database is created
Finished running generic part of root script.
Now product-specific root actions will be performed.
Using configuration parameter file: /u01/app/11.2.0/grid/crs/install/crsconfig_params
Creating trace directory
LOCAL ADD MODE
Creating OCR keys for user ‘root’, privgrp ‘root’..
Operation successful.
OLR initialization – successful
  root wallet
  root wallet cert
  root cert export
  peer wallet
  profile reader wallet
  pa wallet
  peer wallet keys
  pa wallet keys
  peer cert request
  pa cert request
  peer cert
  pa cert
  peer root cert TP
  profile reader root cert TP
  pa root cert TP
  peer pa cert TP
  pa peer cert TP
  profile reader pa cert TP
  profile reader peer cert TP
  peer user cert
  pa user cert
Adding daemon to inittab
ACFS-9200: Supported
ACFS-9300: ADVM/ACFS distribution files found.
ACFS-9307: Installing requested ADVM/ACFS software.
ACFS-9308: Loading installed ADVM/ACFS drivers.
ACFS-9321: Creating udev for ADVM/ACFS.
ACFS-9323: Creating module dependencies – this may take some time.
ACFS-9327: Verifying ADVM/ACFS devices.
ACFS-9309: ADVM/ACFS installation correctness verified.
CRS-2672: Attempting to start ‘ora.mdnsd’ on ‘racnode1’
CRS-2676: Start of ‘ora.mdnsd’ on ‘racnode1’ succeeded
CRS-2672: Attempting to start ‘ora.gpnpd’ on ‘racnode1’
CRS-2676: Start of ‘ora.gpnpd’ on ‘racnode1’ succeeded
CRS-2672: Attempting to start ‘ora.cssdmonitor’ on ‘racnode1’
CRS-2672: Attempting to start ‘ora.gipcd’ on ‘racnode1’
CRS-2676: Start of ‘ora.cssdmonitor’ on ‘racnode1’ succeeded
CRS-2676: Start of ‘ora.gipcd’ on ‘racnode1’ succeeded
CRS-2672: Attempting to start ‘ora.cssd’ on ‘racnode1’
CRS-2672: Attempting to start ‘ora.diskmon’ on ‘racnode1’
CRS-2676: Start of ‘ora.diskmon’ on ‘racnode1’ succeeded
CRS-2676: Start of ‘ora.cssd’ on ‘racnode1’ succeeded

ASM created and started successfully.

Disk Group CRS_VOTING created successfully.

clscfg: -install mode specified
Successfully accumulated necessary OCR keys.
Creating OCR keys for user ‘root’, privgrp ‘root’..
Operation successful.
CRS-4256: Updating the profile
Successful addition of voting disk a8528c91f2eb4f99bf077f568d5b94b1.
Successfully replaced voting disk group with +CRS_VOTING.
CRS-4256: Updating the profile
CRS-4266: Voting file(s) successfully replaced
##  STATE    File Universal Id                File Name Disk group
—  —–    —————–                ——— ———
 1. ONLINE   a8528c91f2eb4f99bf077f568d5b94b1 (ORCL:ASM1) [CRS_VOTING]
Located 1 voting disk(s).
CRS-2672: Attempting to start ‘ora.asm’ on ‘racnode1’
CRS-2676: Start of ‘ora.asm’ on ‘racnode1’ succeeded
CRS-2672: Attempting to start ‘ora.CRS_VOTING.dg’ on ‘racnode1’
CRS-2676: Start of ‘ora.CRS_VOTING.dg’ on ‘racnode1’ succeeded
ACFS-9200: Supported
ACFS-9200: Supported
CRS-2672: Attempting to start ‘ora.registry.acfs’ on ‘racnode1’
CRS-2676: Start of ‘ora.registry.acfs’ on ‘racnode1’ succeeded
Configure Oracle Grid Infrastructure for a Cluster … succeeded

**********************************************
Running Oracle 11g root script…

The following environment variables are set as:
    ORACLE_OWNER= grid
    ORACLE_HOME=  /u01/app/11.2.0/grid

Enter the full pathname of the local bin directory: [/usr/local/bin]:
   Copying dbhome to /usr/local/bin …
   Copying oraenv to /usr/local/bin …
   Copying coraenv to /usr/local/bin …

Creating /etc/oratab file…
Entries will be added to the /etc/oratab file as needed by
Database Configuration Assistant when a database is created
Finished running generic part of root script.
Now product-specific root actions will be performed.
Using configuration parameter file: /u01/app/11.2.0/grid/crs/install/crsconfig_params
Creating trace directory
LOCAL ADD MODE
Creating OCR keys for user ‘root’, privgrp ‘root’..
Operation successful.
OLR initialization – successful
Adding daemon to inittab
ACFS-9200: Supported
ACFS-9300: ADVM/ACFS distribution files found.
ACFS-9307: Installing requested ADVM/ACFS software.
ACFS-9308: Loading installed ADVM/ACFS drivers.
ACFS-9321: Creating udev for ADVM/ACFS.
ACFS-9323: Creating module dependencies – this may take some time.
ACFS-9327: Verifying ADVM/ACFS devices.
ACFS-9309: ADVM/ACFS installation correctness verified.
CRS-4402: The CSS daemon was started in exclusive mode but found an active CSS daemon on node racnode1, number 1, and is terminating
An active cluster was found during exclusive startup, restarting to join the cluster
Configure Oracle Grid Infrastructure for a Cluster … succeeded

[root@racnode2 bin]# ./crs_stat -t
Name           Type           Target    State     Host
————————————————————
ora….TING.dg ora….up.type ONLINE    ONLINE    racnode1
ora….ER.lsnr ora….er.type ONLINE    ONLINE    racnode1
ora….N1.lsnr ora….er.type ONLINE    ONLINE    racnode2
ora….N2.lsnr ora….er.type ONLINE    ONLINE    racnode1
ora….N3.lsnr ora….er.type ONLINE    ONLINE    racnode1
ora.asm        ora.asm.type   ONLINE    ONLINE    racnode1
ora.cvu        ora.cvu.type   ONLINE    ONLINE    racnode1
ora….network ora….rk.type ONLINE    ONLINE    racnode1
ora.oc4j       ora.oc4j.type  ONLINE    ONLINE    racnode1
ora.ons        ora.ons.type   ONLINE    ONLINE    racnode1
ora….SM1.asm application    ONLINE    ONLINE    racnode1
ora….E1.lsnr application    ONLINE    ONLINE    racnode1
ora….de1.ons application    ONLINE    ONLINE    racnode1
ora….de1.vip ora….t1.type ONLINE    ONLINE    racnode1
ora….SM2.asm application    ONLINE    ONLINE    racnode2
ora….E2.lsnr application    ONLINE    ONLINE    racnode2
ora….de2.ons application    ONLINE    ONLINE    racnode2
ora….de2.vip ora….t1.type ONLINE    ONLINE    racnode2
ora….ry.acfs ora….fs.type ONLINE    ONLINE    racnode1
ora.scan1.vip  ora….ip.type ONLINE    ONLINE    racnode2
ora.scan2.vip  ora….ip.type ONLINE    ONLINE    racnode1
ora.scan3.vip  ora….ip.type ONLINE    ONLINE    racnode1

Create ACFS File system—ora home shared by both nodes:

$ ./asmca &
[1]     10311

[root@racnode1 sysconfig]# /u01/app/grid/cfgtoollogs/asmca/scripts/acfs_script.sh
ACFS file system is running on racnode1,racnode2

[root@racnode1 sysconfig]# df -h
/dev/asm/orahomevg-12
                      9.0G   83M  9.0G   1% /u01/app/oracle
[root@racnode2 disks]# df -h

/dev/asm/orahomevg-12
                      9.0G   83M  9.0G   1% /u01/app/oracle

Database Software Installation:

$ ./runInstaller
Starting Oracle Universal Installer…

Checking Temp space: must be greater than 120 MB.   Actual 12358 MB    Passed
Checking swap space: must be greater than 150 MB.   Actual 3836 MB    Passed

Database Creation:

$ export ORACLE_HOME=/u01/app/oracle/product/11.2.0.2/db_1
$ ./dbca &
[1]     25448

racnode1.ukatru.com:/home/oracle>export ORACLE_SID=oradv11
racnode1.ukatru.com:/home/oracle>sqlplus / as sysdba

SQL*Plus: Release 11.2.0.2.0 Production on Sun Feb 13 19:23:45 2011

Copyright (c) 1982, 2010, Oracle.  All rights reserved.

Connected to:
Oracle Database 11g Enterprise Edition Release 11.2.0.2.0 – 64bit Production
With the Partitioning, Real Application Clusters, Automatic Storage Management, OLAP,
Data Mining and Real Application Testing options

SQL>

Now you have fully functional RAC two node 11gR2 database is available.

Advertisements
Categories: oracle-install
  1. No comments yet.
  1. No trackbacks yet.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: