Finding RAC Wait Events—Sessions Waiting

To identify the sessions experiencing waits on the system, perform the following tasks:

1.   Query GV$SESSION_WAIT to determine whether any sessions are experiencing RAC-related waits (at the current time).

2.   Identify the objects that are causing contention for these sessions.

3.   Try to modify the object or query to reduce contention.

For example, query GV$SESSION_WAIT (shown next) to determine whether any sessions are experiencing RAC cache–related waits. Note that the GV$ views are used much more to show statistics for the entire cluster, whereas the V$ views still show statistics from a single node. If you plan to use RAC, you must extend the V$ views and queries to the GV$ views for multiple nodes. This section is only an initial guide to help you see all of the components. The scope of this book does not cover RAC specifically, but some things that will help you tune RAC.

The output from this query should look something like this:

Run this query to identify objects that are causing contention for these sessions and the object that corresponds to the file and block for each FILE_NUMBER/BLOCK_NUMBER combination returned (this query is a bit slower):

The output is similar to

Modify the object to reduce the chances for application contention by doing the following:

   Reduce the number of rows per block.

   Adjust the block size to a smaller block size.

   Modify INITRANS and FREELISTS.

Oracle Clusterware Main Log Files

Oracle Clusterware Main Log Files

Oracle Clusterware uses a unified log directory structure to consolidate the Oracle Clusterware
component log files. This consolidated structure simplifies diagnostic information collection and
assists during data retrieval and problem analysis.

The main directories used by Oracle Clusterware to store its log files:

• CRS logs are in $ORA_CRS_HOME/log//crsd/. The crsd.log file is archived every 10 MB (crsd.l01, crsd.l02, …).
• CSS logs are in $ORA_CRS_HOME/log//cssd/. The cssd.log file isarchived every 20 MB (cssd.l01, cssd.l02, …).
• EVM logs are in $ORA_CRS_HOME/log//evmd.
• Depending on the resource, specific logs are in $ORA_CRS_HOME/log//racg and in $ORACLE_HOME/log//racg. In the last directory, imon_.log is archived every 10 MB for each service. Each RACG executable has a subdirectory assigned exclusively for that executable. The name of the RACG executable subdirectory is the same as the name of the executable.
• SRVM (srvctl) and OCR (ocrdump, ocrconfig, ocrcheck) logs are in $ORA_CRS_HOME/log//client/ and in $ORACLE_HOME/log//client/.
• Important Oracle Clusterware alerts can be found in alert.log in the $ORA_CRS_HOME/log/ directory.

Diagnose Oracle RAC

Goal

To document the logs that should be uploaded for diagnosing Oracle Clusterware issue.
For more information about diagcollection, check out "diagcollection.sh -help"

This note will be obsolete in future, it's strongly recommended to use TFA to prune and collect files from all nodes:
note 1513912.1 - TFA Collector - Tool for Enhanced Diagnostic Gathering

Solution

Linux/UNIX 11gR2/12cR1

1. Execute the following as root user:

# script /tmp/diag.log
# id
# env
# cd
# $GRID_HOME/bin/diagcollection.sh
# exit The following .gz files will be generated in the current directory and need to be uploaded along with /tmp/diag.log:

crsData_.tar.gz,
ocrData_.tar.gz,
oraData_.tar.gz,
coreData_.tar.gz (only --core option specified)
os_.tar.gz

Please ensure all above information are provided from all the nodes.

Linux/UNIX 10gR2/11gR1

1. Execute the following as root user:

# script /tmp/diag.log
# id
# env
# cd
# export OCH=
# export ORACLE_HOME=
# export HOSTNAME=
# $OCH/bin/diagcollection.pl -crshome=$OCH --collect
# exit

The following .gz files will be generated in the current directory and need to be uploaded along with /tmp/diag.log:
crsData_.tar.gz,
ocrData_.tar.gz,
oraData_.tar.gz,
coreData_.tar.gz (only --core option specified)

2. For 10gR2 and 11gR1, if getting an error while running root.sh, please collect /tmp/crsctl.*
Please ensure all above information are provided from all the nodes.

Windows 11gR2/12cR1:

set GRID_HOME=
%GRID_HOME%\perl\bin\perl %GRID_HOME%\bin\diagcollection.pl --collect
The following .zip files will be generated in the current directory and need to be uploaded:
crsData_.zip,
ocrData_.zip,
oraData_.zip,
coreData_.zip (only --core option specified)

Windows 10gR2/11gR1

set ORACLE_HOME=
set OCH=
set ORACLE_BASE=
$OCH%\perl\bin\perl %OCH%\bin\diagcollection.pl --collect

How to relocate Oracle RAC Service

Starting/Stopping Database instance

The database (all instances) can be shut down / started by running the below command from a command prompt window of the database servers.

srvctl stop database –d opera –o immediate srvctl start database –d opera srvctl start database –d opera –o mount srvctl stop instance –d opera –i opera3 –o immediate or srvctl stop instance –d opera –i "opera1,opera2 –o immediate

Starting/Stopping Services

srvctl start service –s "volors,oxihub" –d opera

How to set auto start resources in 11G RAC

https://oracleracdba1.wordpress.com/2013/01/29/how-to-set-auto-start-resources-in-11g-rac/ 
 

How to relocate Oracle RAC Service

All cluster services related to node 1 will be in OFFLINE state.

Take note on the slhors service. This service will failover to node 3 when either node 1 or 2 is down.

When we check using the srvctl command, you will this:

d:\oracle\1020\crs\bin> srvctl status service -d opera -s slhorsservice slhors is running on instance opera2,opera3

It is now handled by opera2 and opera3 instances because node1 is down.

However, even after node 1 is up, instance opera3 won’t move the service back to opera1, therefore, we have to manually relocate the slhors service handled by opera3 back to opera using the following statement:

d:\oracle\1020\crs\bin> srvctl relocate service -d opera -s slhors -i opera3 -t opera1

Note that the slhors is now handled by opera1, opera2 again as it should be.

Diagnose RAC Problems

How to displays the top-level view of the cluster.
# cd /u01/app/11.2.0/grid/bin
# ./crsctl check cluster -all

How to gives information about the individual resources.
# ./crsctl stat res -t
Once the Oracle software is installed, the cluvfy utility is available to provide useful post-installation information. Use the "-help" flag for usage information.

$ cluvfy stage -help
$ cluvfy stage -post crsinst -n ol6-112-rac1,ol6-112-rac2
Oracle provide the RACcheck tool (MOS [ID 1268927.1]) to audit the configuration of RAC, CRS, ASM, GI etc. It supports database versions from 10.2-11.2, making it a useful starting point for most analysis. The MOS note includes the download and setup details.

$ unzip raccheck.zip
$ cd rachcheck
$ chmod 755 raccheck
$ ./raccheck -a

Format Oracle OCFS Drive

Offline cfs drive. Need to offline the drive before you can format it

1. c:\> ocfsutil /c offlinevol /m X:

To format cfs on O drive

1. create a drive letter O in the windows
2. C:\oracle\1020\crs\cfs>ocfsformat /m o: /c 1024 /v data /f

Check ocr status

1. ocrcheck

To do a manual export ocr

1. ocrconfig -export <backup_file_name>

To import from the backup

1. stop crs service first
2. create an empty file data.ocr
3. ocrconfig -import <OCR export_filename>

Add ocr disk

1. create an empty file first
2. ocrconfig -replace mirror x:\cdata\orange_cluster\data.ocr

Remove ocr disk

1. ocrconfig -replace ocr

Query voting disk

1. C:\>crsctl query css votedisk
   0.     0    J:\cdata\orange_cluster\votedsk

Add voting disk

1. create the directory first
2. c:\> crsctl add css votedisk J:\cdata\orange_cluster\votedsk -force  

Delete voting disk

1. stop the crs service before deleting voting disk
2. crsctl delete css votedisk <VOTEDISK_LOCATION> -force

OCR / Vote disk Maintenance Operations: (ADD/REMOVE/REPLACE/MOVE)

In this Document

Goal

Fix

Prepare the disks

1. Size

2. For raw or block device (pre 11.2)

3. For ASM disks (11.2+)

4. For cluster file system

5. Permissions

ADD/REMOVE/REPLACE/MOVE OCR Device

1. To add an OCRMIRROR device when only OCR device is defined:

2. To remove an OCR device

3. To replace or move the location of an OCR device

ADD/DELETE/MOVE Voting Disk

For 10gR2 release

For 11gR1 release

For 11gR2 release

References

---

Applies to:

Oracle Server - Enterprise Edition - Version 10.2.0.1 to 11.2.0.3 [Release 10.2 to 11.2]
Information in this document applies to any platform.

Goal

The goal of this note is to provide steps to add, remove, replace or move an Oracle Cluster Repository (OCR) or voting disk in Oracle Clusterware 10gR2, 11gR1 and 11gR2 environment. It will also provide steps to move OCR / voting and ASM devices from raw device to block device.

This article is intended for DBA and Support Engineers who need to modify, or move OCR and voting disks files, customers who have an existing clustered environment deployed on a storage array and might want to migrate to a new storage array with minimal downtime.

Typically, one would simply cp or dd the files once the new storage has been presented to the hosts. In this case, it is a little more difficult because:

1. The Oracle Clusterware has the OCR and voting disks open and is actively using them. (Both primary and mirrors)
2. There is an API provided for this function (ocrconfig and crsctl), which is the appropriate interface than typical cp and/or dd commands.

It is highly recommended to take a backup of the voting disk, and OCR device before making any changes.

Oracle Cluster Registry (OCR) and Voting Disk Additional clarifications

For Voting disks (never use even number of voting disks):
External redundancy means 1 voting disk
Normal redundancy means 3 voting disks
High redundancy means 5 voting disks
For OCR: 10.2 and 11.1, maximum 2 OCR devices: OCR and OCRMIRROR
11.2+, upto 5 OCR devices can be added.

Fix

Prepare the disks

For OCR or votind disk addition or replacement, new disks need to be prepared. Please refer to Clusteware/Gird Infrastructure installation guide for different platform for the disk requirement and preparation.

1. Size

For 10.1:
OCR device minimum size (each): 100M
Voting disk minimum size (each): 20M
For 10.2:
OCR device minimum size (each): 256M
Voting disk minimum size (each): 256M
For 11.1:
OCR device minimum size (each): 280M
Voting disk minimum size (each): 280M
For 11.2:
OCR device minimum size (each): 300M
Voting disk minimum size (each): 300M

2. For raw or block device (pre 11.2)

Please refer to Clusterware installation guide on different platform for more details.
On windows platform the new raw device link is created via $CRS_HOME\bin\GUIOracleOBJManager.exe, for example:
\\.\VOTEDSK2
\\.\OCR2

3. For ASM disks (11.2+)

On Windows platform, please refer to Document 331796.1 How to setup ASM on Windows
On Linux platform, please refer to Document 580153.1 How To Setup ASM on Linux Using ASMLIB Disks, Raw Devices or Block Devices?
For other platform, please refer to Clusterware/Gird Infrastructure installation guide.

4. For cluster file system

If OCR is on cluster file system, the new OCR or OCRMIRROR file must be touched before add/replace command can be issued. Otherwise PROT-21: Invalid parameter (10.2/11.) or PROT-30 The Oracle Cluster Registry location to be added is not accessible (for 11.2) will occur.

As root user
# touch
/cluster_fs/ocrdisk.dat
# touch /cluster_fs/ocrmirror.dat
# chown
root:oinstall /cluster_fs/ocrdisk.dat  /cluster_fs/ocrmirror.dat
# chmod 640
/cluster_fs/ocrdisk.dat  /cluster_fs/ocrmirror.dat

It is not required to pre-touch voting disk file on cluster file system.
After delete command is issued, the ocr/voting files on the cluster file system require to be removed manually.

5. Permissions

For OCR device:
chown root:oinstall <OCR device>
chmod 640 <OCR device>
For Voting device:
chown <crs/grid>:oinstall <Voting device>
chmod 644 <Voting device>
For ASM disks used for OCR/Voting disk:
chown griduser:asmadmin <asm disks>
chmod 660 <asm disks>

ADD/REMOVE/REPLACE/MOVE OCR Device

Note: You must be logged in as the root user, because root owns the OCR files. "crsctl -replace" command can only be issued when CRS is running, otherwise "PROT-1: Failed to initialize ocrconfig" will occur.

Please ensure CRS is running on ALL cluster nodes during this operation, otherwise the change will not reflect in the CRS down node, CRS will have problem to startup from this down node. "ocrconfig -repair" option will be required to fix the ocr.loc file on the CRS down node.

For 11.2+ with OCR on ASM diskgroup, due to unpublished Bug 8604794 - FAIL TO CHANGE OCR LOCATION TO DG WITH 'OCRCONFIG -REPAIR -REPLACE', "ocrconfig -repair" to change OCR location to different ASM diskgroup does not work currently. Workaround is to manually edit /etc/oracle/ocr.loc or /var/opt/ocr.loc or Windows registry HYKEY_LOCAL_MACHINE\SOFTWARE\Oracle\ocr, point to desired diskgroup.

Make sure there is a recent copy of the OCR file before making any changes:

ocrconfig
-showbackup

If there is not a recent backup copy of the OCR file, an export can be taken for the current OCR file. Use the following command to generate an export of the online OCR file:

In 10.2

# ocrconfig -export
<OCR export_filename> -s online

In 11.1 and 11.2

# ocrconfig
-manualbackup
node1 2008/08/06 06:11:58
/crs/cdata/crs/backup_20080807_003158.ocr

To recover using this file, the following command can be used:

#
ocrconfig -import <OCR export_filename>

From 11.2+, please also refer How to restore ASM based OCR after complete loss of the CRS diskgroup on Linux/Unix systems Document 1062983.1

To see whether OCR is healthy, run an ocrcheck, which should return with like below.

# ocrcheck
Status of
Oracle Cluster Registry is as follows :
Version : 2
Total space (kbytes) :
497928
Used space (kbytes) : 312
Available space (kbytes) : 497616
ID :
576761409
Device/File Name : /dev/raw/raw1
Device/File integrity check
succeeded
Device/File Name : /dev/raw/raw2
Device/File integrity check
succeeded

Cluster registry integrity check succeeded

For 11.1+,
ocrcheck as root user should also show:
Logical corruption check
succeeded

1. To add an OCRMIRROR device when only OCR device is defined:

To add an OCR mirror device, provide the full path including file name.
10.2 and 11.1:

# ocrconfig -replace
ocrmirror <filename>
eg:
# ocrconfig -replace ocrmirror
/dev/raw/raw2
# ocrconfig -replace ocrmirror /dev/sdc1
# ocrconfig
-replace ocrmirror /cluster_fs/ocrdisk.dat
> ocrconfig -replace ocrmirror
\\.\OCRMIRROR2  - for Windows

11.2+: From 11.2 onwards, upto 4 ocrmirrors can be added

# ocrconfig -add
<filename>
eg:
# ocrconfig -add +OCRVOTE2
# ocrconfig -add
/cluster_fs/ocrdisk.dat

2. To remove an OCR device

To remove an OCR device:
10.2 and 11.1:

# ocrconfig -replace
ocr

11.2+:

# ocrconfig -delete
<filename>
eg:
# ocrconfig -delete +OCRVOTE1

* Once an OCR device is removed, ocrmirror device automatically changes to be OCR device.
* It is not allowed to remove OCR device if only 1 OCR device is defined, the command will return PROT-16.

To remove an OCR mirror device:
10.2 and 11.1:

# ocrconfig -replace
ocrmirror

11.2+:

# ocrconfig -delete
<ocrmirror filename>
eg:
# ocrconfig -delete
+OCRVOTE2

After removal, the old OCR/OCRMIRROR can be deleted if they are on cluster filesystem.

3. To replace or move the location of an OCR device

Note. 1. An ocrmirror must be in place before trying to replace the OCR device. The ocrconfig will fail with PROT-16, if there is no ocrmirror exists.
2. If an OCR device is replaced with a device of a different size, the size of the new device will not be reflected until the clusterware is restarted.

10.2 and 11.1:
To replace the OCR device with <filename>, provide the full path including file name.

# ocrconfig -replace
ocr <filename>
eg:
# ocrconfig -replace ocr /dev/sdd1
$ ocrconfig
-replace ocr \\.\OCR2 - for Windows

To replace the OCR mirror device with <filename>, provide the full path including file name.

# ocrconfig -replace
ocrmirror <filename>
eg:
# ocrconfig -replace ocrmirrow
/dev/raw/raw4
# ocrconfig -replace ocrmirrow \\.\OCRMIRROR2  - for
Windows

11.2:
The command is same for replace either OCR or OCRMIRRORs (at least 2 OCR exist for replace command to work):

# ocrconfig -replace
<current filename> -replacement <new filename>
eg:
# ocrconfig
-replace /cluster_file/ocr.dat -replacement +OCRVOTE
# ocrconfig -replace
+CRS -replacement +OCRVOTE

ADD/DELETE/MOVE Voting Disk

Note: 1. crsctl votedisk commands must be run as root for 10.2 and 11.1, but can be run as grid user for 11.2+
2. For 11.2, when using ASM disks for OCR and voting, the command is same for Windows and Unix platform.

To take a backup of voting disk:

$ dd if=voting_disk_name of=backup_file_name

For Windows:

ocopy \\.\votedsk1 o:\backup\votedsk1.bak

For 10gR2 release

Shutdown the Oracle Clusterware (crsctl stop crs as root) on all nodes before making any modification to the voting disk. Determine the current voting disk location using:
crsctl query css votedisk

1. To add a Voting Disk, provide the full path including file name:

# crsctl add css
votedisk <VOTEDISK_LOCATION> -force
eg:
# crsctl add css votedisk
/dev/raw/raw1 -force
# crsctl add css votedisk /cluster_fs/votedisk.dat
-force
> crsctl add css votedisk \\.\VOTEDSK2 -force   - for
windows

2. To delete a Voting Disk, provide the full path including file name:

# crsctl delete css
votedisk <VOTEDISK_LOCATION> -force
eg:
# crsctl delete css votedisk
/dev/raw/raw1 -force
# crsctl delete css votedisk /cluster_fs/votedisk.dat
-force
> crsctl delete css votedisk \\.\VOTEDSK1 -force   - for
windows

3. To move a Voting Disk, provide the full path including file name, add a device first before deleting the old one:

# crsctl add css
votedisk <NEW_LOCATION> -force
# crsctl delete css votedisk
<OLD_LOCATION> -force
eg:
# crsctl add css votedisk /dev/raw/raw4
-force
# crsctl delete css votedisk /dev/raw/raw1
-force

After modifying the voting disk, start the Oracle Clusterware stack on all nodes

# crsctl start
crs

Verify the voting disk location using

# crsctl query css
votedisk

For 11gR1 release

Starting with 11.1.0.6, the below commands can be performed online (CRS is up and running).

1. To add a Voting Disk, provide the full path including file name:

# crsctl add css
votedisk <VOTEDISK_LOCATION>
eg:
# crsctl add css votedisk
/dev/raw/raw1
# crsctl add css votedisk /cluster_fs/votedisk.dat
>
crsctl add css votedisk \\.\VOTEDSK2        - for windows

2. To delete a Voting Disk, provide the full path including file name:

# crsctl delete css
votedisk <VOTEDISK_LOCATION>
eg:
# crsctl delete css votedisk
/dev/raw/raw1 -force
# crsctl delete css votedisk
/cluster_fs/votedisk.dat
> crsctl delete css votedisk \\.\VOTEDSK1     -
for windows

3. To move a Voting Disk, provide the full path including file name:

# crsctl add css
votedisk <NEW_LOCATION>
# crsctl delete css votedisk
<OLD_LOCATION>
eg:
# crsctl add css votedisk /dev/raw/raw4
#
crsctl delete css votedisk /dev/raw/raw1

Verify the voting disk location:

# crsctl query css
votedisk

For 11gR2 release

From 11.2, votedisk can be stored on either ASM diskgroup or cluster file systems. The following commands can only be executed when Grid Infrastructure is running. As grid user:

1. To add a Voting Disk
a. When votedisk is on cluster file system:

$ crsctl add css
votedisk <cluster_fs/filename>

b. When votedisk is on ASM diskgroup, no add option available.
The number of votedisk is determined by the diskgroup redundancy. If more copy of votedisk is desired, one can move votedisk to a diskgroup with higher redundancy. See step 4.
If a votedisk is removed from a normal or high redundancy diskgroup for abnormal reason, it can be added back using:

alter diskgroup <vote diskgroup
name> add disk '</path/name>' force;

2. To delete a Voting Disk
a. When votedisk is on cluster file system:

$ crsctl delete css
votedisk <cluster_fs/filename>

b. When votedisk is on ASM, no delete option available, one can only replace the existing votedisk group with another ASM diskgroup

3. To move a Voting Disk on cluster file system

$ crsctl add css votedisk
<new_cluster_fs/filename>
$ crsctl delete css votedisk
<old_cluster_fs/filename>

4. To move voting disk on ASM from one diskgroup to another diskgroup due to redundancy change or disk location change

$ crsctl replace
votedisk <+diskgroup>|<vdisk>

Example here is moving from external redundancy +OCRVOTE diskgroup to normal redundancy +CRS diskgroup

1. create new diskgroup +CRS as
desired

2. $ crsctl query css votedisk
## 
STATE    File Universal Id                File Name Disk group
--  -----   
-----------------                --------- ---------
1. ONLINE  
5e391d339a594fc7bf11f726f9375095 (ORCL:ASMDG02) [+OCRVOTE]
Located 1 voting
disk(s).

3. $ crsctl replace votedisk +CRS
Successful addition of
voting disk 941236c324454fc0bfe182bd6ebbcbff.
Successful addition of voting
disk 07d2464674ac4fabbf27f3132d8448b0.
Successful addition of voting disk
9761ccf221524f66bff0766ad5721239.
Successful deletion of voting disk
5e391d339a594fc7bf11f726f9375095.
Successfully replaced voting disk group
with +CRS.
CRS-4266: Voting file(s) successfully replaced

4. $ crsctl query css votedisk
##  STATE    File Universal
Id                File Name Disk group
--  -----   
-----------------                --------- ---------
1. ONLINE  
941236c324454fc0bfe182bd6ebbcbff (ORCL:CRSD1) [CRS]
2. ONLINE  
07d2464674ac4fabbf27f3132d8448b0 (ORCL:CRSD2) [CRS]
3. ONLINE  
9761ccf221524f66bff0766ad5721239 (ORCL:CRSD3) [CRS]
Located 3 voting
disk(s).

5. To move voting disk between ASM diskgroup and cluster file system
a. Move from ASM diskgroup to cluster file system:

$ crsctl query css votedisk
## 
STATE    File Universal Id                File Name Disk group
--  -----   
-----------------                --------- ---------
1. ONLINE  
6e5850d12c7a4f62bf6e693084460fd9 (ORCL:CRSD1) [CRS]
2. ONLINE  
56ab5c385ce34f37bf59580232ea815f (ORCL:CRSD2) [CRS]
3. ONLINE  
4f4446a59eeb4f75bfdfc4be2e3d5f90 (ORCL:CRSD3) [CRS]
Located 3 voting
disk(s).

$ crsctl replace votedisk /rac_shared/oradata/vote.test3
Now
formatting voting disk: /rac_shared/oradata/vote.test3.
CRS-4256: Updating
the profile
Successful addition of voting disk
61c4347805b64fd5bf98bf32ca046d6c.
Successful deletion of voting disk
6e5850d12c7a4f62bf6e693084460fd9.
Successful deletion of voting disk
56ab5c385ce34f37bf59580232ea815f.
Successful deletion of voting disk
4f4446a59eeb4f75bfdfc4be2e3d5f90.
CRS-4256: Updating the profile
CRS-4266:
Voting file(s) successfully replaced

$ crsctl query css votedisk
## 
STATE    File Universal Id                File Name Disk group
--  -----   
-----------------                --------- ---------
1. ONLINE  
61c4347805b64fd5bf98bf32ca046d6c (/rac_shared/oradata/vote.disk) []
Located 1
voting disk(s).

b. Move from cluster file system to ASM diskgroup

$ crsctl query css votedisk
## 
STATE    File Universal Id                File Name Disk group
--  -----   
-----------------                --------- ---------
1. ONLINE  
61c4347805b64fd5bf98bf32ca046d6c (/rac_shared/oradata/vote.disk) []
Located 1
voting disk(s).

$ crsctl replace votedisk +CRS
CRS-4256: Updating the
profile
Successful addition of voting disk
41806377ff804fc1bf1d3f0ec9751ceb.
Successful addition of voting disk
94896394e50d4f8abf753752baaa5d27.
Successful addition of voting disk
8e933621e2264f06bfbb2d23559ba635.
Successful deletion of voting disk
61c4347805b64fd5bf98bf32ca046d6c.
Successfully replaced voting disk group
with +CRS.
CRS-4256: Updating the profile
CRS-4266: Voting file(s)
successfully replaced

[oragrid@auw2k4 crsconfig]$ crsctl query css
votedisk
##  STATE    File Universal Id                File Name Disk
group
--  -----    -----------------                ---------
---------
1. ONLINE   41806377ff804fc1bf1d3f0ec9751ceb (ORCL:CRSD1)
[CRS]
2. ONLINE   94896394e50d4f8abf753752baaa5d27 (ORCL:CRSD2) [CRS]
3.
ONLINE   8e933621e2264f06bfbb2d23559ba635 (ORCL:CRSD3) [CRS]
Located 3 voting
disk(s).

6. To verify:

$ crsctl query css
votedisk

CRSCTL Commands CheatSheet

You can find below various commands which can be used to administer Oracle Clusterware using crsctl. This is for purpose of easy reference.

 

Relocate services

#srvctl status service -s opera -s slors

#srvctl relocate service -d opera -s slors -i opera3 -t opera1

 

Start Oracle Clusterware

#crsctl start crs

 

Stop Oracle Clusterware

#crsctl stop crs

 

Enable Oracle Clusterware

#crsctl enable crs

 

It enables automatic startup of Clusterware daemons

 

Disable Oracle Clusterware

#crsctl disable crs

It disables automatic startup of Clusterware daemons. This is useful when you are performing some
operations like OS patching and does not want clusterware to start the daemons automatically.

 

Checking Voting disk Location

$crsctl query css votedisk

0. 0 /dev/sda3
1. 0 /dev/sda5
2. 0 /dev/sda6
Located 3 voting disk(s).

Note: -Any command which just needs to query information can be run using oracle user. But anything which alters Oracle Clusterware requires root privileges.

Add Voting disk

#crsctl add css votedisk path

 

Remove Voting disk

#crsctl delete css votedisk path

 

Check CRS Status

$crsctl check crs

Cluster Synchronization Services appears healthy

Cluster Ready Services appears healthy

Event Manager appears healthy

You can also see particular daemon status

$crsctl check cssd

Cluster Synchronization Services appears healthy

$crsctl check crsd

Cluster Ready Services appears healthy

$crsctl check evmd

Event Manager appears healthy

You can also check Clusterware status on both the nodes using

$crsctl check cluster

prod01 ONLINE

prod02 ONLINE

 

Checking Oracle Clusterware Version

To determine software version (binary version of the software on a particular cluster node) use

$crsctl query crs softwareversion

Oracle Clusterware version on node [prod01] is [11.1.0.6.0]

For checking active version on cluster, use

$ crsctl query crs activeversion

Oracle Clusterware active version on the cluster is [11.1.0.6.0]

As per documentation, multiple versions are used while upgrading.

There are other options for CRSCTL too which can be seen using

$crsctl

Or

$crsctl help