Home » Server Options » RAC & Failsafe » OEL 5.4, Clusterware 11g R1 (11.1.0.6) (RAC After Install 11gR1: No resources are registered.)
OEL 5.4, Clusterware 11g R1 (11.1.0.6) [message #463796] Sat, 03 July 2010 14:02 Go to next message
DBA_SangramKeshari
Messages: 44
Registered: October 2009
Location: Mumbai
Member
Hello ,

I was installing Clusterware 11gR1(11.1.0.6) on Oracle Enterprise linux 5.4. The install didn't completed successfully.
was having no issues for user uqivalenece checking,rpm installation both cluvfy checks were successfull prior to installation.

Something went wrong at the time of Configuration Assistant was running and failed while ons config was running.


--crs_stat utility permissions

oracle@oel54ora11gr1rac1[/email] bin]$ pwd
/u01/crs/oracle/product/11.1.0/crs/bin

-r-xr-x--x 1 root oinstall 2388 Jul 3 21:33 crs_unregister
-r-xr-x--x 1 root oinstall 2388 Jul 3 21:33 crs_stop
-r-xr-x--x 1 root oinstall 2388 Jul 3 21:33 crs_stat
-r-xr-x--x 1 root oinstall 2388 Jul 3 21:33 crs_start
-r-xr-x--x 1 root oinstall 2388 Jul 3 21:33 crs_relocate
-r-xr-x--x 1 root oinstall 2388 Jul 3 21:33 crs_register
-r-xr-x--x 1 root oinstall 2388 Jul 3 21:33 crs_profile

[oracle@oel54ora11gr1rac1[/email] bin]$ ./crs_stat -t
CRS-0202: No resources are registered.

[oracle@oel54ora11gr1rac1[/email] bin]$ ./srvctl start nodeapps -n oel54ora11gr1rac1
PRKO-2006 : Invalid node name: oel54ora11gr1rac1

[oracle@oel54ora11gr1rac1[/email] ~]$ ./crs_stat -t
CRS-0184: Cannot communicate with the CRS daemon.


[oracle@oel54ora11gr1rac2[/email] bin]$ ./crs_stat -t
CRS-0184: Cannot communicate with the CRS daemon.



I am unable to debug and no clue how to fix it. Please feel free to share your idea and experiece to correct.

My concern id how to start and stop my crs resources.


Regards,
Sangram

[Updated on: Sat, 03 July 2010 14:06]

Report message to a moderator

Re: OEL 5.4, Clusterware 11g R1 (11.1.0.6) [message #463826 is a reply to message #463796] Sun, 04 July 2010 10:01 Go to previous messageGo to next message
mkounalis
Messages: 147
Registered: October 2009
Location: Dallas, TX
Senior Member
De-install, clean up lingering files, verify base os config using cluvfy, re-install.
Re: OEL 5.4, Clusterware 11g R1 (11.1.0.6) [message #463828 is a reply to message #463796] Sun, 04 July 2010 10:03 Go to previous messageGo to next message
mkounalis
Messages: 147
Registered: October 2009
Location: Dallas, TX
Senior Member
Also, did you try 'crs_start -all?
Re: OEL 5.4, Clusterware 11g R1 (11.1.0.6) [message #464106 is a reply to message #463828] Tue, 06 July 2010 05:12 Go to previous messageGo to next message
DBA_SangramKeshari
Messages: 44
Registered: October 2009
Location: Mumbai
Member
Even I tried with #crsctl stop crs then, cleaning up four crs related file in /etc/init.d then ocrInfo.loc , formatted the ocr disk using dd command, the rebuild ocr using root.sh

I am frustrated now, OCR is again corrupt. This is know bug. But how to fix as the version is 11.1.0.6
Re: OEL 5.4, Clusterware 11g R1 (11.1.0.6) [message #464129 is a reply to message #463796] Tue, 06 July 2010 07:13 Go to previous messageGo to next message
mkounalis
Messages: 147
Registered: October 2009
Location: Dallas, TX
Senior Member
Are you sure you have lun equivalency across your nodes? How do you know that your OCR is corrupt? Did you uninstall the entire 'crs stack and reinstall? Simply cleaning and reformatting the OCR and voting disks and the re-running root.sh isn't the best idea. Sounds to me like you have luns that are mapped differently on each of the nodes. Re-check you shared lun configuration. What makes you think this is a known bug? Do you have a sr open with oracle?

[Updated on: Tue, 06 July 2010 07:14]

Report message to a moderator

Re: OEL 5.4, Clusterware 11g R1 (11.1.0.6) [message #464130 is a reply to message #464129] Tue, 06 July 2010 07:31 Go to previous messageGo to next message
DBA_SangramKeshari
Messages: 44
Registered: October 2009
Location: Mumbai
Member
To rebuild ocr i follow the below steps.


-------------------------------------------------
rm -rf /etc/oracle /etc/oratab /etc/oraInst.loc

cd /etc/init.d
rm init.evmd init.cssd init.crsd init.crs

cd /etc/init.d
ls -ltr init.evmd init.cssd init.crsd init.crs

cd /u01/crs/oracle/product/11.1.0/crs/
ls -ltr root.sh
sh root.sh
-------------------------------------------------
[root@oel54ora11gr1rac1 bin]# ./ocrcheck
Status of Oracle Cluster Registry is as follows :
Version : 2
Total space (kbytes) : 2096108
Used space (kbytes) : 1968
Available space (kbytes) : 2094140
ID : 620591252
Device/File Name : /u01/DISKS/OCR1
Device/File integrity check succeeded
Device/File Name : /u01/DISKS/OCR2
Device/File integrity check succeeded

Cluster registry integrity check succeeded

[root@oel54ora11gr1rac1 bin]# ./crsctl query css votedisk
0. 0 /u01/DISKS/VD1
1. 0 /u01/DISKS/VD2
Located 2 voting disk(s).
[root@oel54ora11gr1rac1 bin]#
[root@oel54ora11gr1rac1 bin]# su - oracle
[oracle@oel54ora11gr1rac1 ~]$ crsctl query css votedisk
0. 0 /u01/DISKS/VD1
1. 0 /u01/DISKS/VD2
Located 2 voting disk(s).
[oracle@oel54ora11gr1rac1 ~]$ ocrcheck
Status of Oracle Cluster Registry is as follows :
Version : 2
Total space (kbytes) : 2096108
Used space (kbytes) : 1968
Available space (kbytes) : 2094140
ID : 620591252
Device/File Name : /u01/DISKS/OCR1
Device/File integrity check succeeded
Device/File Name : /u01/DISKS/OCR2
Device/File integrity check succeeded

Cluster registry integrity check succeeded




[oracle@oel54ora11gr1rac2 ~]$ crsctl query css votedisk
Oracle Cluster Registry initialization failed accessing Oracle Cluster Registry device: PROC-26: Error while accessing the physical storage
[oracle@oel54ora11gr1rac2 ~]$ ocrcheck
PROT-602: Failed to retrieve data from the cluster registry


Regards,
SangyD
Re: OEL 5.4, Clusterware 11g R1 (11.1.0.6) [message #464134 is a reply to message #464130] Tue, 06 July 2010 07:38 Go to previous messageGo to next message
mkounalis
Messages: 147
Registered: October 2009
Location: Dallas, TX
Senior Member
Ok - did you re-install the 'crs stack after a full cleanup? And again, did you verify lun equivalency? How did you originally prepare the luns across your nodes?
Re: OEL 5.4, Clusterware 11g R1 (11.1.0.6) [message #464135 is a reply to message #464134] Tue, 06 July 2010 07:40 Go to previous messageGo to next message
mkounalis
Messages: 147
Registered: October 2009
Location: Dallas, TX
Senior Member
And do you have more than one node running? Would be a good troubleshooting step to just start one node and see if it works. Then start another node and see if the corruption then appears. Are you sure that permissions on your devices are set properly?

[Updated on: Tue, 06 July 2010 07:40]

Report message to a moderator

Re: OEL 5.4, Clusterware 11g R1 (11.1.0.6) [message #464136 is a reply to message #464134] Tue, 06 July 2010 07:40 Go to previous messageGo to next message
DBA_SangramKeshari
Messages: 44
Registered: October 2009
Location: Mumbai
Member
What do you mean by lun equivalency?
I am a newbee to RAC
Sofar I heard of oracle user equvalence.
Please provide the command.


Regards,
SangyD
Re: OEL 5.4, Clusterware 11g R1 (11.1.0.6) [message #464138 is a reply to message #464136] Tue, 06 July 2010 07:44 Go to previous messageGo to next message
mkounalis
Messages: 147
Registered: October 2009
Location: Dallas, TX
Senior Member
Did you follow the oracle installation guide? Lun equivalency is important - your OCR on node one might be a voting disk on node two ... it's not a single command it's a big part of preparing the nodes before install. Try cleaning up and then starting only one node. If it works, start another node. If it gets corrupted then probably luns are not configured correctly. Full install guides are available at docs.oracle.com.
Re: OEL 5.4, Clusterware 11g R1 (11.1.0.6) [message #464139 is a reply to message #464138] Tue, 06 July 2010 07:47 Go to previous messageGo to next message
mkounalis
Messages: 147
Registered: October 2009
Location: Dallas, TX
Senior Member
By the way, your error message doesn't say that the structures are corrupt, it says it can't access them. Recheck your device permissions. Again - the install guides describe all this in great detail. With rac installation, there are no shortcuts Wink
Re: OEL 5.4, Clusterware 11g R1 (11.1.0.6) [message #464144 is a reply to message #463796] Tue, 06 July 2010 08:11 Go to previous messageGo to next message
DBA_SangramKeshari
Messages: 44
Registered: October 2009
Location: Mumbai
Member
I have already installed 10 RACR2 manytime on RHEL4 but this the first time i am installing 11g R1 RAC on OEL 5.4.

please see below if could find any discrepencies, I am unable to find any...

[oracle@oel54ora11gr1rac1 ~]$ cd /u01/DISKS
[oracle@oel54ora11gr1rac1 DISKS]$ ls -ltr
total 0
lrwxrwxrwx 1 oracle oinstall 9 Jul 1 00:24 VD2 -> /dev/sdj1
lrwxrwxrwx 1 oracle oinstall 9 Jul 1 00:24 VD1 -> /dev/sdi1
lrwxrwxrwx 1 oracle oinstall 9 Jul 1 00:24 OCRBKP2 -> /dev/sde1
lrwxrwxrwx 1 oracle oinstall 9 Jul 1 00:24 OCRBKP1 -> /dev/sdd1
lrwxrwxrwx 1 oracle oinstall 9 Jul 1 00:24 OCR2 -> /dev/sdc1
lrwxrwxrwx 1 oracle oinstall 9 Jul 1 00:24 OCR1 -> /dev/sdb1
lrwxrwxrwx 1 oracle oinstall 9 Jul 1 00:24 VDBKP2 -> /dev/sdl1
lrwxrwxrwx 1 oracle oinstall 9 Jul 1 00:24 VDBKP1 -> /dev/sdk1
lrwxrwxrwx 1 oracle oinstall 9 Jul 1 00:24 BLANK2 -> /dev/sdn1
lrwxrwxrwx 1 oracle oinstall 9 Jul 1 00:24 BLANK1 -> /dev/sdm1
lrwxrwxrwx 1 oracle oinstall 9 Jul 1 00:24 ASMDISK3 -> /dev/sdh1
lrwxrwxrwx 1 oracle oinstall 9 Jul 1 00:24 ASMDISK2 -> /dev/sdg1
lrwxrwxrwx 1 oracle oinstall 9 Jul 1 00:24 ASMDISK1 -> /dev/sdf1
[oracle@oel54ora11gr1rac1 DISKS]$ ls -ltr /dev/sd* |grep oracle
brw-rw---- 1 oracle oinstall 8, 65 Jul 6 16:16 /dev/sde1
brw-rw---- 1 oracle oinstall 8, 81 Jul 6 16:16 /dev/sdf1
brw-rw---- 1 oracle oinstall 8, 49 Jul 6 16:16 /dev/sdd1
brw-rw---- 1 oracle oinstall 8, 97 Jul 6 16:16 /dev/sdg1
brw-rw---- 1 oracle oinstall 8, 113 Jul 6 16:16 /dev/sdh1
brw-rw---- 1 oracle oinstall 8, 193 Jul 6 16:16 /dev/sdm1
brw-rw---- 1 oracle oinstall 8, 161 Jul 6 16:16 /dev/sdk1
brw-rw---- 1 oracle oinstall 8, 177 Jul 6 16:16 /dev/sdl1
brw-rw---- 1 oracle oinstall 8, 209 Jul 6 16:16 /dev/sdn1
brw-rw---- 1 oracle oinstall 8, 33 Jul 6 17:58 /dev/sdc1
brw-r----- 1 oracle oinstall 8, 129 Jul 6 18:36 /dev/sdi1
brw-r----- 1 oracle oinstall 8, 145 Jul 6 18:36 /dev/sdj1



[oracle@oel54ora11gr1rac2 ~]$ cd /u01/DISKS
[oracle@oel54ora11gr1rac2 DISKS]$ ls -ltr
total 0
lrwxrwxrwx 1 oracle oinstall 9 Jul 1 00:24 OCRBKP2 -> /dev/sde1
lrwxrwxrwx 1 oracle oinstall 9 Jul 1 00:24 OCRBKP1 -> /dev/sdd1
lrwxrwxrwx 1 oracle oinstall 9 Jul 1 00:24 OCR2 -> /dev/sdc1
lrwxrwxrwx 1 oracle oinstall 9 Jul 1 00:24 OCR1 -> /dev/sdb1
lrwxrwxrwx 1 oracle oinstall 9 Jul 1 00:24 VDBKP2 -> /dev/sdl1
lrwxrwxrwx 1 oracle oinstall 9 Jul 1 00:24 VDBKP1 -> /dev/sdk1
lrwxrwxrwx 1 oracle oinstall 9 Jul 1 00:24 VD2 -> /dev/sdj1
lrwxrwxrwx 1 oracle oinstall 9 Jul 1 00:24 VD1 -> /dev/sdi1
lrwxrwxrwx 1 oracle oinstall 9 Jul 1 00:24 BLANK2 -> /dev/sdn1
lrwxrwxrwx 1 oracle oinstall 9 Jul 1 00:24 BLANK1 -> /dev/sdm1
lrwxrwxrwx 1 oracle oinstall 9 Jul 1 00:24 ASMDISK2 -> /dev/sdg1
lrwxrwxrwx 1 oracle oinstall 9 Jul 1 00:24 ASMDISK1 -> /dev/sdf1
lrwxrwxrwx 1 oracle oinstall 9 Jul 1 00:24 ASMDISK3 -> /dev/sdh1
[oracle@oel54ora11gr1rac2 DISKS]$ ls -ltr /dev/sd* |grep oracle
brw-rw---- 1 oracle oinstall 8, 33 Jul 6 16:16 /dev/sdc1
brw-rw---- 1 oracle oinstall 8, 49 Jul 6 16:16 /dev/sdd1
brw-rw---- 1 oracle oinstall 8, 81 Jul 6 16:16 /dev/sdf1
brw-rw---- 1 oracle oinstall 8, 97 Jul 6 16:16 /dev/sdg1
brw-rw---- 1 oracle oinstall 8, 65 Jul 6 16:16 /dev/sde1
brw-rw---- 1 oracle oinstall 8, 113 Jul 6 16:16 /dev/sdh1
brw-rw---- 1 oracle oinstall 8, 161 Jul 6 16:16 /dev/sdk1
brw-rw---- 1 oracle oinstall 8, 177 Jul 6 16:16 /dev/sdl1
brw-rw---- 1 oracle oinstall 8, 193 Jul 6 16:16 /dev/sdm1
brw-rw---- 1 oracle oinstall 8, 209 Jul 6 16:16 /dev/sdn1
brw-rw---- 1 oracle oinstall 8, 145 Jul 6 17:37 /dev/sdj1
brw-r----- 1 oracle oinstall 8, 129 Jul 6 18:37 /dev/sdi1



Regards,
SangyD
Re: OEL 5.4, Clusterware 11g R1 (11.1.0.6) [message #464151 is a reply to message #464144] Tue, 06 July 2010 08:31 Go to previous messageGo to next message
mkounalis
Messages: 147
Registered: October 2009
Location: Dallas, TX
Senior Member
Again - you need to follow the install docs. The output above doesn't prove or disprove lun equivalency. Please get with your sysadmin or storage administrator to confirm luns are the same on both nodes. This forum isn't a replacement for oracle support - you should probably open up a ticket with oracle as you are running oracle enterprise Linux as well. And for what it's worth, your device permissions are wrong - I don't know how many times I need to tell you to follow the docs.
Re: OEL 5.4, Clusterware 11g R1 (11.1.0.6) [message #464152 is a reply to message #464144] Tue, 06 July 2010 08:33 Go to previous messageGo to next message
DBA_SangramKeshari
Messages: 44
Registered: October 2009
Location: Mumbai
Member
This one i am doing in Vmware at home. I am Myself is the Sys Admin Smile Sad

Regards,
SangyD
Re: OEL 5.4, Clusterware 11g R1 (11.1.0.6) [message #464178 is a reply to message #464152] Tue, 06 July 2010 10:04 Go to previous messageGo to next message
mkounalis
Messages: 147
Registered: October 2009
Location: Dallas, TX
Senior Member
How did you configure shared storage for your installation? Did you fix your permissions? That may be your issue . . . . .
Re: OEL 5.4, Clusterware 11g R1 (11.1.0.6) [message #464181 is a reply to message #464178] Tue, 06 July 2010 10:08 Go to previous messageGo to next message
mkounalis
Messages: 147
Registered: October 2009
Location: Dallas, TX
Senior Member
Please read the docs - you need to set the permissions according to what is required . . . .

http://download.oracle.com/docs/cd/B28359_01/install.111/b28263/storage.htm#CDECEBGH
Re: OEL 5.4, Clusterware 11g R1 (11.1.0.6) [message #464272 is a reply to message #464178] Wed, 07 July 2010 03:44 Go to previous message
DBA_SangramKeshari
Messages: 44
Registered: October 2009
Location: Mumbai
Member
It's successfully done now. There was a file coruuption in virtual disk which was being used by VMware. just restores thatone from my last backup. Now All is Well Wink

cluster is in synch

Regards,
SangyD
Previous Topic: RAC with ASM or GPFS on IBM AIX machine ?
Next Topic: SRVCTL shows instance down while instance is up
Goto Forum:
  


Current Time: Fri Mar 29 06:20:37 CDT 2024