We can use the RMAN RESTORE VALIDATE command to check and verify the integrity of the backups which are stored on tapes or disk as well.

A point to keep in mind is that the RESTORE DATABASE VALIDATE command will only check for the datafile backups and will not check for either the archivelog of controlfile backups. We need to issue additional RESTORE ARCHIVELOG VALIDATE as well as RESTORE CONTROLFILE VALIDATE commands.

We can also issue the RESTORE SPFILE VALIDATE command to check the backup of the server parameter file. By default, RMAN only checks for physical corruption while validating. We can also instruct RMAN to check for logical corruption via the VALIDATE CHECK LOGICAL command.

The RESTORE DATABASE VALIDATE command will check for the last level 0 or FULL tape or disk based backup, but the RESTORE ARCHIVELOG ALL command will check for all the archivelog files catalogued based on the retention policy.

So when we are validating the restore of archivelog files, it may be better to check for the existence or accessibility in the tape library or disk of all the archivelog files which have been generated since the last full or level 0 backup.

For example if we are taking daily backups, we would like to check for all archivelog file backups for the past day using the command as shown below:

RMAN> restore archivelog from time ’sysdate-1′ validate;

Some other examples of the VALIDATE command are:

RMAN> restore database validate;
RMAN> restore controlfile validate;
RMAN> restore spfile validate;
RMAN> restore tablespace users validate;
RMAN> validate backupset 922;
RMAN> validate recovery area;

An example is shown below where we are checking the validity of a database backup for a database which has a weekly level 0 backup. Note that the RESTORE DATABASE VALIDATE command will cause RMAN to check for the last level 0 backup which has been performed in this case on the 15th of November.

RMAN> restore database validate;

Starting restore at 20-NOV-09
using channel ORA_SBT_TAPE_1
using channel ORA_DISK_1

channel ORA_SBT_TAPE_1: starting validation of datafile backup set
channel ORA_SBT_TAPE_1: reading from backup piece vnkud0k5_1_1
channel ORA_SBT_TAPE_1: piece handle=vnkud0k5_1_1 tag=TAG20091115T060908
channel ORA_SBT_TAPE_1: restored backup piece 1
channel ORA_SBT_TAPE_1: validation complete, elapsed time: 00:14:15
Finished restore at 20-NOV-09

RMAN> list backup of database tag TAG20091115T060908;

List of Backup Sets
===================

BS Key Type LV Size Device Type Elapsed Time Completion Time
——- —- — ———- ———– ———— —————
3054 Incr 0 15.59G SBT_TAPE 00:39:00 15-NOV-09
BP Key: 3054 Status: AVAILABLE Compressed: NO Tag: TAG20091115T060908
Handle: vnkud0k5_1_1 Media:
List of Datafiles in backup set 3054
File LV Type Ckp SCN Ckp Time Name
—- — —- ———- ——— —-
1 0 Incr 23674579905 15-NOV-09 /u04/oradata/emrep/system01.dbf
2 0 Incr 23674579905 15-NOV-09 /u04/oradata/emrep/undotbs01.dbf
3 0 Incr 23674579905 15-NOV-09 /u04/oradata/emrep/sysaux01.dbf
4 0 Incr 23674579905 15-NOV-09 /u04/oradata/emrep/users01.dbf
5 0 Incr 23674579905 15-NOV-09 /u04/oradata/emrep/mgmt.dbf
6 0 Incr 23674579905 15-NOV-09 /u04/oradata/emrep/mgmt_ecm_depot1.dbf
7 0 Incr 23674579905 15-NOV-09 /u04/oradata/emrep/patrol01.dbf

Let us simulate the case where an archivelog is not present on disk and hence the validate will fail as shown below.

[PROD] emrep:/u05/oradata/emrep/arch > mv arch18053.1.669202302.log arch18053.1.669202302.log.old
[PROD] emrep:/u05/oradata/emrep/arch > rman target /

Recovery Manager: Release 11.1.0.6.0 – Production on Fri Nov 20 12:23:31 2009

Copyright (c) 1982, 2007, Oracle. All rights reserved.

connected to target database: EMREP (DBID=3892233981)

RMAN> restore archivelog from time ’sysdate-1′ validate;

Starting restore at 20-NOV-09
using target database control file instead of recovery catalog
allocated channel: ORA_SBT_TAPE_1
channel ORA_SBT_TAPE_1: SID=261 device type=SBT_TAPE
channel ORA_SBT_TAPE_1: Data Protection for Oracle: version 5.4.1.0
allocated channel: ORA_DISK_1
channel ORA_DISK_1: SID=221 device type=DISK

RMAN-00571: ===========================================================
RMAN-00569: =============== ERROR MESSAGE STACK FOLLOWS ===============
RMAN-00571: ===========================================================
RMAN-03002: failure of restore command at 11/20/2009 12:23:37
RMAN-06026: some targets not found – aborting restore
RMAN-06025: no backup of archived log for thread 1 with sequence 18053 and starting SCN of 23679034351 found to restore

Businesses are increasingly realizing the immense value that historical data can provide to help them understand market trends and customer behaviour as well. Further, many regulatory and compliance policies
like SOX and BASEL-2 mandate retention of historical data.

Until now, this unfortunately involved application rewrites,administration overheads or costly third-party software solutions. Total Recall or the new Oracle 11g Flashback Data Archive feature has greatly enhanced the flashback technology which was introduced in Oracle 9i to make it far more than just a tool to recover from logical corruptions and human error. It is now an out-of-the box Information Life Cycle Management tool and provides ease of management of historical information and long term secure data tracking with minimal performance overheads.

11g Flashback Data Archive provides the automated ability to track and store all transactional changes to a table over its lifetime without having to build this intelligence into your application. Prior to 11g, Flashback technology to a large part was based on the availability of undo data or flashback logs and both the undo data as well as flashback logs are subject to recycling when out of space pressure exists. The UNDO tablespace in Oracle was primarily meant for transaction consistency and not data archival.

Flashback Data Archive or Total Recall Features

Flashback Data Archive uses a background process fbda to capture data asynchronously. It runs every 5 minutes which is the default as well as at more frequent intervals depending on system activity. The Primary source for historical data is the Undo data, but this undo data is not reused until the historical data is written first.

Getting Started

Creating a Flashback Data Archive (lets call it FBDA)

SQL> CREATE TABLESPACE his_data_1
  2  DATAFILE '+data' SIZE 500M;

Tablespace created.

SQL> CREATE FLASHBACK ARCHIVE DEFAULT fba1
  2  TABLESPACE his_data_1
  3  RETENTION 2 DAY;

Flashback archive created.

SQL> SELECT FLASHBACK_ARCHIVE_NAME, to_char(CREATE_TIME,'dd-mon-yyyy') Created,
  2  RETENTION_IN_DAYS,STATUS FROM DBA_FLASHBACK_ARCHIVE;

FLASHBACK_ARCHIVE_NA CREATED     RETENTION_IN_DAYS STATUS
-------------------- ----------- ----------------- -------
FBA1                 02-nov-2009                 2 DEFAULT

SQL> GRANT FLASHBACK ARCHIVE ON fba1 TO scott;

Grant succeeded.

SQL> ALTER TABLE mysales FLASHBACK ARCHIVE fba1;

Table altered.

SQL> CREATE TABLE
  2  EMPSAL_HIS
  3  (EMPNO number,
  4  ENAME VARCHAR2(10),
  5  SAL NUMBER,
  6  FLASHBACK ARCHIVE;

Table created.

SQL> select TABLE_NAME,FLASHBACK_ARCHIVE_NAME,ARCHIVE_TABLE_NAME,STATUS
  2  from USER_FLASHBACK_ARCHIVE_tables;

TABLE_NAME            FLASHBACK_ARCHIVE_NA  ARCHIVE_TABLE_NAME               STATUS
-----------	  --------------------  --------------------	      ------------
MYSALES               FBA1                 SYS_FBA_HIST_77429               ENABLED
EMPSAL_HIS            FBA1                 SYS_FBA_HIST_77419               ENABLED

Note: for every table where Flashback Archive is enabled, corresponding internal history tables are created in the flashback archive tablespace as shown below. These tables are automatically partitioned and compressed as well

SQL> select object_id from dba_objects where object_name=‘DEPT';

 OBJECT_ID
----------
     73201

SQL> select table_name,tablespace_name from user_tables;

TABLE_NAME                     TABLESPACE_NAME
------------------------------ ------------------------------
DEPT                           USERS
SYS_FBA_DDL_COLMAP_73201       HIS_DATA_1
SYS_FBA_TCRV_73201             HIS_DATA_1
SYS_FBA_HIST_73201

SQL> desc SYS_FBA_HIST_73201
 Name                                      Null?    Type
 ----------------------------------------- -------- ----------------------------
 RID                                                VARCHAR2(4000)
 STARTSCN                                           NUMBER
 ENDSCN                                             NUMBER
 XID                                                RAW(8)
 OPERATION                                          VARCHAR2(1)
EMPNO                                              NUMBER(4)
 ENAME                                              VARCHAR2(10)
 MGR                                                NUMBER(10)
DEPTNO                                             NUMBER(2)
 JOB                                                VARCHAR2(20)

SQL> SELECT TABLE_NAME,PARTITION_NAME,COMPRESSION from USER_TAB_PARTITIONS;

TABLE_NAME                     PARTITION_NAME                 COMPRESS
------------------------------ ------------------------------ --------
SYS_FBA_HIST_77429            HIGH_PART                      ENABLED
SYS_FBA_HIST_77419            HIGH_PART                      ENABLED

Similar to the clusterware processes in a RAC environment, in 11g R2 even for a standalone instance, functionality is now available to automatically restart components like the database, listener, ASM diskgroup, service etc in the event of their failure.

This feature is called Oracle Restart and it runs out of the Grid Infrastructure home which is separate from the database home.

Oracle Restart is managed by CRSCTL utility and it uses Oracle High Availability Services to start and stop the components managed by Oracle Restart.

Using Oracle Restart, we can stop or start all the components running out of a single Oracle Home with a single srvctl command as well as Oracle Restart will automatically start all components on machine reboot in the proper order taking into account dependencies like ASM instance and the managed database instances. So no more do we have to use the dbstart and dbstop scripts to manage this.

Oracle Restart can be extended to Data Guard configurations as well by integrating with Data Guard Broker to manage the start and stop of database services following a data guard role transition.

Let us examine some of the srvctl commands which can be executed in a single instance environment as well as see the use of the crsctl command – normally previously only used for RAC environments.

-bash-3.2$ crs_stat -t
Name           Type           Target    State     Host
------------------------------------------------------------
ora.DATA1.dg   ora....up.type ONLINE    ONLINE    redhat64
ora....ER.lsnr ora....er.type ONLINE    ONLINE    redhat64
ora....WARE.dg ora....up.type ONLINE    ONLINE    redhat64
ora.asm        ora.asm.type   ONLINE    ONLINE    redhat64
ora.cssd       ora.cssd.type  ONLINE    ONLINE    redhat64
ora.diskmon    ora....on.type ONLINE    ONLINE    redhat64
ora.eons       ora.eons.type  ONLINE    ONLINE    redhat64
ora.ons        ora.ons.type   ONLINE    ONLINE    redhat64

bash-3.2$ ./crsctl check has
CRS-4638: Oracle High Availability Services is online

-bash-3.2$ ./crsctl check css
CRS-4529: Cluster Synchronization Services is online

-bash-3.2$ ./srvctl start listener

-bash-3.2$ ps -ef |grep tns
oracle   14899     1  0 14:08 ?        00:00:00 /u02/app/oracle/product/11.2.0/grid/bin/tnslsnr LISTENER -inherit

-bash-3.2$ ./srvctl status listener
Listener LISTENER is enabled
Listener LISTENER is running on node(s): redhat64

-bash-3.2$ ./srvctl status asm
ASM is running on redhat64

-bash-3.2$ ./srvctl status diskgroup -g DATA1
Disk Group DATA1 is running on redhat64

-bash-3.2$ ./srvctl config asm
ASM home: /u02/app/oracle/product/11.2.0/grid
ASM listener: LISTENER
Spfile: +DATA1/asm/asmparameterfile/registry.253.700932479
ASM diskgroup discovery string: /dev/raw/raw*

In case a process dies unexpectedly, Oracle Restart will automatically start the process. In the example below, we kill the PMON process of the ora11gr2 instance and find that in less than 5 seconds the instance has been restarted by the Oracle High Availablity Service running in the background constantly monitoring the health of the managed components.

-bash-3.2$ ps -ef |grep pmon
oracle   12710     1  0 Oct22 ?        00:00:01 asm_pmon_+ASM
oracle   15752     1  0 14:14 ?        00:00:00 ora_pmon_ora11gr2
oracle   15975 18271  0 14:15 pts/2    00:00:00 grep pmon

-bash-3.2$ kill -9  15752

-bash-3.2$ ps -ef |grep pmon
oracle   12710     1  0 Oct22 ?        00:00:01 asm_pmon_+ASM
oracle   16679     1  0 14:22 ?        00:00:00 ora_pmon_ora11gr2
oracle   16775 18271  0 14:22 pts/2    00:00:00 grep pmon

In this example we stop all the components running out of a particular home by executing the srvctl stop home command. After stopping the database home, we find that only the ASM instance is running as it is managed by a seperate home which is the Grid Infrastructure home. After the database home is started, the database instance ora11gr2 also is automatically started.

[oracle@redhat346 ~]$ srvctl stop home -o /u01/app/oracle/product/11.2.0/dbhome_1 -s /tmp/state

[oracle@redhat346 ~]$ ps -ef |grep pmon
oracle   19841     1  0 Sep18 ?        00:01:24 asm_pmon_+ASM
oracle   31843 27855  0 15:19 pts/1    00:00:00 grep pmon

[oracle@redhat346 ~]$ srvctl start home -o /u01/app/oracle/product/11.2.0/dbhome_1 -s /tmp/state

[oracle@redhat346 ~]$ ps -ef |grep pmon
oracle   19841     1  0 Sep18 ?        00:01:24 asm_pmon_+ASM
oracle   26690     1  0 Oct08 ?        00:00:41 ora_pmon_emrep
oracle   32054     1  0 15:20 ?        00:00:00 ora_pmon_testdb
oracle   32344     1  0 15:20 ?        00:00:00 ora_pmon_ora11gr2
oracle   32620 27855  0 15:22 pts/1    00:00:00 grep pmon

Let us have a look at the installation screenshots of 11g Release 2 on a Red Hat Linux platform. Currently the 11g R2 software has only been released for the Linux platform with release dates for other platforms like Solaris and AIX set for sometime end October to mid November.

There are quite a few new installation options available as can be seen from the screenshots below. What we are installing below is “Grid Infrastructure for a stand alone server”.

In 11g R2, ASM is now part of what is called the Grid Infrastructure (nothing to do with Grid Control). It is no longer an option available in dbca and there is a command line option asmca which is launched from the Grid Infrastructure home. The ASM instance will be running from the Grid Home and not the database Oracle Home.

Further, the Grid Infrastructure can be installed for a stand alone server or for a cluster deployment. The ASM and Clusterware are instaled in the same home directory and it should be noted that now in 11g R2, the clusterware files like the OCR and the Voting Disk can be located on ASM storage and raw devices is no longer supported.

Also, now there is separation between SYSASM and SYSDBA and if you want to connect to the ASM instance you need to do it as SYSASM. It is recommended to also create a separate group in addition to the DBA group specific for ASM administration and make SYSASM grantees members of this group. In one of the screenshots below we will see an alert being displayed when we have selected the dba group for the ASM installation.

The following are some of the new ASM related features introduced in 11g R2:

[oracle@redhat346 stage]$ sqlplus sys/xxx as sysdba

SQL*Plus: Release 11.2.0.1.0 Production on Fri Oct 16 09:10:55 2009

Copyright (c) 1982, 2009, Oracle.  All rights reserved.

Connected to:
Oracle Database 11g Enterprise Edition Release 11.2.0.1.0 - 64bit Production
With the Automatic Storage Management option

SQL> shutdown immediate;
ORA-01031: insufficient privileges

The example below will show how we can configure and mount a ACFS on Red HatLinux with 11g R2.

Note: ASM operations like creating and adding disk groups etc which were earlier performed via DBCA will now be performed via the GUI ASM Configuration Assistant which needs to be launched via the ‘asmca‘ command from the Grid Infrastructure Home.

We first create a disk group called ASMCFS and then create a volume called ASMCFS_VOL1 of 25 GB using the disk group that we just created.

We will then use a mount point /u03 to mount this file system. A configuration script is created which includes the commands to be run as root to mount this file system. After the script is run we will see the ACFS file system mounted on /u03.

/bin/mount -t acfs /dev/asm/asmcfs_vol1-44 /u03

chown oracle:dba /u03

chmod 775 /u03

[oracle@redhat346 stage]$ df -k

Filesystem           1K-blocks      Used Available Use% Mounted on
.....
......
/dev/asm/asmcfs_vol1-44
                      26214400     89112  26125288   1% /u03

In one of my earlier posts How To Transport a Tablespace from 10g ASM to 11g ASM on AIX we discussed how to transport a tablespace stored on ASM storage from Windows to AIX using the DBMS_FILE_TRANSFER package.

We will now look at how the same thing can be done using FTP with Oracle XML DB Repository and its virtual folders.

All we need to do is to run the following script and provide the port numbers which will be used for the FTP and HTTP protocols for accessing the repository.

@?/rdbms/admin/catxdbdbca 8080 7787

In this case we have assigned the port 8080 for the FTP protocol and the port 7787 is for HTTP access.

The ASM related directories on the Windows Oracle database are accessed via the sys/asm virtual folder and then we use FTP to copy the ASM files over the network and then use RMAN to convert it to AIX 64 bit format

The example below shows us how we are connecting to the Windows server via port 8080 using the FTP protocol. Note we are then connecting with a database user – SYSTEM in this case.

testdb:/u01/oracle > ftp cbdorca261 8080
Connected to cbdorca261.bankwest.com.
220- cbdorca261
Unauthorised use of this FTP server is prohibited and may be subject to civil and criminal prosecution.
220 cbdorca261 FTP Server (Oracle XML DB/Oracle Database) ready.
Name (cbdorca261:oracle): system
331 pass required for SYSTEM
Password:
230 SYSTEM logged in
ftp> i
200  Type set to I.
ftp> cd sys/asm
250 CWD Command successful
ftp> cd DATA/genua1/datafile
250 CWD Command successful
ftp> get test_tts.376.699110247
200 PORT Command successful
150 BIN Data Connection
226 BIN Transfer Complete
104865792 bytes received in 2.25 seconds (4.551e+04 Kbytes/s)
local: test_tts.376.699110247 remote: test_tts.376.699110247
ftp> quit
221 QUIT Goodbye.

 testdb:/u01/oracle > rman target /

Recovery Manager: Release 11.1.0.7.0 - Production on Thu Oct 1 13:50:02 2009

Copyright (c) 1982, 2007, Oracle.  All rights reserved.

connected to target database: TESTDB (DBID=2482257163)

RMAN>  convert datafile2>
 '/u01/oracle/test_tts.376.699110247'
3> FORMAT '+DATA'
4> from platform 'Microsoft Windows IA (32-bit)';

Starting conversion at target at 01-OCT-09
using target database control file instead of recovery catalog
allocated channel: ORA_DISK_1
channel ORA_DISK_1: SID=140 device type=DISK
channel ORA_DISK_1: starting datafile conversion
input file name=/u01/oracle/test_tts.376.699110247
converted datafile=+DATA/testdb/datafile/test_tts.267.699112231
channel ORA_DISK_1: datafile conversion complete, elapsed time: 00:00:03
Finished conversion at target at 01-OCT-09

imp file=tts.dmp transport_tablespace=y tablespaces=TEST_TTS datafiles=+DATA/testdb/datafile/test_tts.267.699112231

Import: Release 11.1.0.7.0 - Production on Thu Oct 1 13:57:33 2009

Copyright (c) 1982, 2007, Oracle.  All rights reserved.

Username: sys as sysdba
Password:

Connected to: Oracle Database 11g Enterprise Edition Release 11.1.0.7.0 - 64bit Production
With the Partitioning and Real Application Testing options

Export file created by EXPORT:V10.02.01 via conventional path
About to import transportable tablespace(s) metadata...
import done in US7ASCII character set and AL16UTF16 NCHAR character set
import server uses WE8MSWIN1252 character set (possible charset conversion)
export client uses WE8MSWIN1252 character set (possible charset conversion)
. importing SYS's objects into SYS
. importing SYS's objects into SYS
. importing SYSTEM's objects into SYSTEM
. . importing table                    "MYOBJECTS"
. importing SYS's objects into SYS
Import terminated successfully without warnings.

SQL> conn system/oracle
Connected.

SQL> select file_name from dba_data_files
  2  where tablespace_name='TEST_TTS';

FILE_NAME
--------------------------------------------------------------------------------
+DATA/testdb/datafile/test_tts.267.699112231

SQL> select tablespace_name from user_tables
  2  where table_name='MYOBJECTS';

TABLESPACE_NAME
------------------------------
TEST_TTS

SQL> select count (*) from myobjects;

  COUNT(*)
----------
     56454

asmcmd is a command line tool that we can use to manage Oracle ASM instances, ASM disk groups, files and dierctories, templates very much as in the same way we would do while working with say a UNIX file system.

ASMCMD can be launched in interactive or non-interactive modes and we need to first ensure that our environment points to the “Grid Infrastructure Home” as in 11g R2, ASM is not part of the standard database software installation. Also chec that the ORACLE_SID points to the ASM instance which should be running.

Let us have a quick look at some of the useful command options which are now available with 11g Release 2.

lsct: Lists information about current Oracle ASM clients from the V$ASM_CLIENT view.

ASMCMD> lsct
DB_Name  Status     Software_Version  Compatible_version  Instance_Name  Disk_Group
+ASM     CONNECTED        11.2.0.1.0          11.2.0.1.0  +ASM           DATA
testdb   CONNECTED        11.2.0.1.0          11.2.0.0.0  testdb         DATA

cp: We can copy files from ASM to OS as wel as from OS to ASM disk groups

ASMCMD> cp EXAMPLE.265.697907183 /tmp/example01.bak
copying +DATA/TESTDB/DATAFILE/EXAMPLE.265.697907183 -> /tmp/example01.bak

ASMCMD> cp /tmp/example01.bak +DATA/TESTDB/DATAFILE/example01.bak
copying /tmp/example01.bak -> +DATA/TESTDB/DATAFILE/example01.bak

ASMCMD> ls
EXAMPLE.265.697907183
SYSAUX.257.697907095
SYSTEM.256.697907095
UNDOTBS1.258.697907095
USERS.259.697907095
example01.bak

du: Total space in MB used by files – particular directory can also be specified

ASMCMD> pwd
+DATA/TESTDB/DATAFILE

ASMCMD> du
Used_MB      Mirror_used_MB
   1574                1574

ASMCMD> du +DATA/TESTDB/ONLINELOG
Used_MB      Mirror_used_MB
    153                 153

find: we can use the wildcard or can specify a particular file type by using the ‘–type’ clause

ASMCMD> find --type ONLINELOG +DATA *
+DATA/TESTDB/ONLINELOG/group_1.261.697907177
+DATA/TESTDB/ONLINELOG/group_2.262.697907179
+DATA/TESTDB/ONLINELOG/group_3.263.697907179

ASMCMD> find +DATA example*
+DATA/ASM/DATAFILE/example01.bak.267.698929915
+DATA/TESTDB/DATAFILE/EXAMPLE.265.697907183
+DATA/TESTDB/DATAFILE/example01.bak

ls: list the contents of an ASM Disk Group directory as well as attributes of files located in the directory

ASMCMD> ls -s
Block_Size  Blocks      Bytes      Space  Name
      8192   12801  104865792  106954752  EXAMPLE.265.697907183
      8192   89601  734011392  736100352  SYSAUX.257.697907095
      8192   89601  734011392  736100352  SYSTEM.256.697907095
      8192    7681   62922752   65011712  UNDOTBS1.258.697907095
      8192     641    5251072    6291456  USERS.259.697907095
                                          example01.bak => +DATA/ASM/DATAFILE/example01.bak.267.698929915

ASMCMD> ls -l
Type      Redund  Striped  Time             Sys  Name
DATAFILE  UNPROT  COARSE   SEP 30 11:00:00  Y    EXAMPLE.265.697907183
DATAFILE  UNPROT  COARSE   SEP 28 23:00:00  Y    SYSAUX.257.697907095
DATAFILE  UNPROT  COARSE   SEP 29 22:00:00  Y    SYSTEM.256.697907095
DATAFILE  UNPROT  COARSE   SEP 18 22:00:00  Y    UNDOTBS1.258.697907095
DATAFILE  UNPROT  COARSE   SEP 18 22:00:00  Y    USERS.259.697907095
                                            N    example01.bak => +DATA/ASM/DATAFILE/example01.bak.267.698929915

iostat: Uses the V$ASM_DISK_IOSTAT view to display I/O statistics of disks in mounted ASM disk groups

ASMCMD> iostat -G DATA
Group_Name  Dsk_Name   Reads        Writes
DATA        DATA_0000  25448671744  19818926592

ASMCMD> iostat -t
Group_Name  Dsk_Name   Reads        Writes       Read_Time    Write_Time
DATA        DATA_0000  25450195456  19819686912  6491.434444  8042.604156

lsdg: Uses V$ASM_DISKGROUP_STAT view to list information about a particular disk group

ASMCMD> lsdg DATA
State    Type    Rebal  Sector  Block       AU  Total_MB  Free_MB  Req_mir_free_MB  Usable_file_MB    Name
MOUNTED  EXTERN  N         512   4096  1048576     51200    49375                0           49375    DATA/

ASMCMD> lsdsk -t -G DATA
Create_Date  Mount_Date  Repair_Timer  Path
18-SEP-09    18-SEP-09   0             /dev/raw/raw1

lsattr: List attributes of a disk group

ASMCMD> lsattr -l -G DATA
Name                     Value
access_control.enabled   FALSE
access_control.umask     066
au_size                  1048576
cell.smart_scan_capable  FALSE
compatible.asm           11.2.0.0.0
compatible.rdbms         10.1.0.0.0
disk_repair_time         3.6h
sector_size              512

Use the setattr command to change an attribute

ASMCMD> setattr -G data compatible.rdbms 11.2.0.0.0
ASMCMD> lsattr -l -G DATA
Name                     Value
access_control.enabled   FALSE
access_control.umask     066
au_size                  1048576
cell.smart_scan_capable  FALSE
compatible.asm           11.2.0.0.0
compatible.rdbms         11.2.0.0.0
disk_repair_time         3.6h
sector_size              512

In 10g Enterprise Manager Grid Control, we can create corrective action or ‘fixit’ jobs which can automatically run if say a metric threshold is crossed or a particular event occurs. For example, we may have situations where we monitor the disk space used by the archive log files and as soon as the disk space free or available comes below a threshold, we would like another action to automatically occur which in this case could be to backup the archive log files to tape and then delete them to free up the disk space.

Read more about this feature in this worked example …….

We can now use SQL*PLUS type substitution variables in RMAN like &1, &2 and so on and the same can be linked to shell scripts as shown below with the new clause USING where we are passing the Incremental Level and the backup tag to the actual RMAN command.

Also note that in 11g we can run RMAN scripts using the ‘@’ like a SQL script

testbkp.rcv

connect target sys/oracle
run{
backup incremental level &1
database
tag &2;
}
exit;

testbkp.sh

#! /bin/ksh
export level=$1
export tag=$2_`date +%d%b%y`
rman @testbkp.rcv using $level $tag

Note the actual RMAN command that is run …….. it adds a date along with the tag name (parameter 2) and runs a level 2 backup as the first parameter was ‘2′

apex:/u01/oracle> ./testbkp.sh 2 dly_inc_bkp

Recovery Manager: Release 11.1.0.6.0 – Production on Tue Sep 22 11:28:05 2009

Copyright (c) 1982, 2007, Oracle. All rights reserved.

RMAN> connect target *
2> run{
3> backup incremental level 2
4> database
5> tag dly_inc_bkp_22Sep09;
6> }
7> exit;
connected to target database: APEX (DBID=1312143933)

Starting backup at 22-SEP-09
using target database control file instead of recovery catalog
allocated channel: ORA_SBT_TAPE_1
channel ORA_SBT_TAPE_1: SID=349 device type=SBT_TAPE
channel ORA_SBT_TAPE_1: Data Protection for Oracle: version 5.5.1.0
channel ORA_SBT_TAPE_1: starting incremental level 2 datafile backup set
channel ORA_SBT_TAPE_1: specifying datafile(s) in backup set
input datafile file number=00003 name=/u02/oradata/apex/undotbs01.dbf
input datafile file number=00002 name=/u02/oradata/apex/sysaux01.dbf
input datafile file number=00001 name=/u02/oradata/apex/system01.dbf
input datafile file number=00005 name=/u02/oradata/apex/monitor_data01.dbf
input datafile file number=00007 name=/u02/oradata/apex/encrypt_data01.dbf
input datafile file number=00008 name=/u02/oradata/apex/disc10_data01.dbf
channel ORA_SBT_TAPE_1: starting piece 1 at 22-SEP-09
channel ORA_SBT_TAPE_1: finished piece 1 at 22-SEP-09
piece handle=a5kpsim9_1_1 tag=DLY_INC_BKP_22SEP09 comment=API Version 2.0,MMS Version 5.5.1.0
channel ORA_SBT_TAPE_1: backup set complete, elapsed time: 00:00:25
Finished backup at 22-SEP-09

Starting Control File and SPFILE Autobackup at 22-SEP-09
piece handle=c-1312143933-20090922-05 comment=API Version 2.0,MMS Version 5.5.1.0
Finished Control File and SPFILE Autobackup at 22-SEP-09

Recovery Manager complete.

11g Release 2 Edition-based redefinition enables you to upgrade or change the database component of an application while it is in use, thereby minimising or eliminating down time which was earlier required when an object like a procedure required to be modified, but we could not do it without any outage as the application was accessing the procedure or package which needed to be modified.

An edition is like a workspace or private environment where database objects are redefined. When we are satisfied with the change that we have made, those changes in the edition can be then rolled out to all the application users.

Let us see an example of this where we create a procedure, make some changes to the procedure in a new edition and then make those changes visible to other database users.

SQL>  GRANT CREATE ANY EDITION, DROP ANY EDITION to sh;

Grant succeeded.

SQL> conn sh/sh
Connected.

SQL> CREATE OR REPLACE PROCEDURE hello IS
  BEGIN
    DBMS_OUTPUT.PUT_LINE('Hello, edition 1.');
  END hello;
/
  2    3    4    5
Procedure created.

SQL> set serverout on

SQL> BEGIN hello(); END;
/
  2  Hello, edition 1.

PL/SQL procedure successfully completed.

SQL> CREATE EDITION e2;

Edition created.

SQL> ALTER SESSION SET EDITION = e2;

Session altered.

SQL> CREATE OR REPLACE PROCEDURE hello IS
  BEGIN
    DBMS_OUTPUT.PUT_LINE('Hello, edition 2.');
  END hello;
/
  2    3    4    5
Procedure created.

SQL> BEGIN hello(); END;
  2  /
Hello, edition 2.

PL/SQL procedure successfully completed.

Note, now if we change the edition to the default edition ‘ORA$BASE’, we will now see the original procedure and not the updated one

SQL> ALTER SESSION SET EDITION = ora$base;

Session altered.

SQL>  BEGIN hello(); END;
  2  /
Hello, edition 1.

Let us now see how this can be rolled out to database user HR

SQL> grant execute on hello to hr;

Grant succeeded.

SQL> GRANT USE ON EDITION e2 to public;

Grant succeeded.

We now make the default edition of the database, E2 so that all changes in the new edition E2 will be visible to all database users

SQL> conn sys as sysdba
Enter password:
Connected.

SQL> ALTER DATABASE DEFAULT EDITION=e2;

Database altered.

SQL> conn hr/hr
Connected.

SQL> set serverout on

SQL>  exec sh.hello;
Hello, edition 2.

Prior to Oracle 11g, the default behaviour was to make statistics available for the optimizer to use as soon as they were gathered.

While statistics are required to enable the optimizer to generate optimal execution plans, sometimes just by gathering fresh statistics tried and trusted execution plans can abruptly change and thereby adversely affect application performance.

In Oracle 11g we can now ‘defer’ the publication of statistics until they have been tried and tested and once we have confirmed that the execution plans are correct and optimal. Statistics remain in the pending state until they are published and the parameter optimizer_use_pending_statistics (default value FALSE) which when set at the session level will enables us to test the pending statistics independently of other database sessions.

Let us look at a test case using the SALES table in the SH schema.

If we see the query below, it appears that the data for the column CHANNEL_ID is skewed where majority of the rows have the value 3 and a very small majority have the value 9.

SQL> select channel_id,count(*) from sales
2 group by channel_id order by 2;

CHANNEL_ID COUNT(*)
———- ———-
9 2074
4 118416
2 258025
3 540328

But both the queries below are performing a full table scan of the SALES table when ideally it should be performing an index scan when the value 9 is used in the WHERE clause considering it accounts for a very small proportion of the rows in the SALES table.

SELECT
S.cust_id
,S.prod_id
,SUM(S.amount_sold)
FROM sh.sales S
WHERE
channel_id=9
GROUP BY S.cust_id, S.prod_id
ORDER BY S.cust_id, S.prod_id
/

SELECT
S.cust_id
,S.prod_id
,SUM(S.amount_sold)
FROM sh.sales S
WHERE
channel_id=3
GROUP BY S.cust_id, S.prod_id
ORDER BY S.cust_id, S.prod_id
/

As earlier mentioned, statistics for a table are published immediately by default, so we use the DBMS_STATS package to change this default behaviour.

SQL> exec dbms_stats.set_table_prefs('SH','SALES','PUBLISH','FALSE');

PL/SQL procedure successfully completed.

SQL> select dbms_stats.get_prefs('PUBLISH', 'SH', 'SALES' ) FROM DUAL;

DBMS_STATS.GET_PREFS('PUBLISH','SH','SALES')
----------------------------------------------------------------------------------------------
FALSE

Since we observed that the data in the table is skewed, we will gather histograms which may help the optimizer make more informed decisions when generating an execution plan.

Note that since we have disabled the publishing of statistics for the table SALES, even though we have analyzed the table, the LAST_ANALYZED column shows that the table has not been recently analyzed.

SQL> EXEC DBMS_STATS.GATHER_TABLE_STATS('SH','SALES',METHOD_OPT => 'FOR ALL COLUMNS SIZE AUTO');

PL/SQL procedure successfully completed.

SQL> SELECT LAST_ANALYZED FROM DBA_TABLES WHERE TABLE_NAME='SALES';

LAST_ANAL
---------
18-AUG-2009

However, we we query the DBA_TAB_PENDING_STATS view, it shows that the SALES table which is partitioned has been analyzed today.

SQL> SELECT TABLE_NAME,PARTITION_NAME ,LAST_ANALYZED  FROM DBA_TAB_PENDING_STATS;

TABLE_NAME                     PARTITION_NAME                 LAST_ANAL
------------------------------ ------------------------------ ---------
SALES                                                         17-SEP-09
SALES                          SALES_Q3_1998                  17-SEP-09
SALES                          SALES_Q3_1999                  17-SEP-09
SALES                          SALES_Q3_2000                  17-SEP-09
SALES                          SALES_Q3_2001                  17-SEP-09
SALES                          SALES_Q3_2002                  17-SEP-09
SALES                          SALES_Q3_2003                  17-SEP-09
SALES                          SALES_Q4_1998                  17-SEP-09
SALES                          SALES_Q4_1999                  17-SEP-09
SALES                          SALES_Q4_2000                  17-SEP-09
SALES                          SALES_Q4_2001                  17-SEP-09
SALES                          SALES_Q4_2002                  17-SEP-09
SALES                          SALES_Q4_2003                  17-SEP-09
SALES                          SALES_1995                     17-SEP-09
SALES                          SALES_1996                     17-SEP-09
SALES                          SALES_H1_1997                  17-SEP-09
SALES                          SALES_H2_1997                  17-SEP-09
SALES                          SALES_Q1_1998                  17-SEP-09
SALES                          SALES_Q1_1999                  17-SEP-09
SALES                          SALES_Q1_2000                  17-SEP-09
SALES                          SALES_Q1_2001                  17-SEP-09
SALES                          SALES_Q1_2002                  17-SEP-09
SALES                          SALES_Q1_2003                  17-SEP-09
SALES                          SALES_Q2_1998                  17-SEP-09
SALES                          SALES_Q2_1999                  17-SEP-09
SALES                          SALES_Q2_2000                  17-SEP-09
SALES                          SALES_Q2_2001                  17-SEP-09
SALES                          SALES_Q2_2002                  17-SEP-09
SALES                          SALES_Q2_2003                  17-SEP-09

We would like to see if the query execution plan has changed or improved after the recent gathering of statistics. We can use the parameter optimizer_use_pending_statistics for this by setting the value to TRUE (default is FALSE)

SQL> alter session set optimizer_use_pending_statistics=TRUE;

Session altered.

We now find that the optimizer is performing an index scan when the value 9 is used as a predicate value for CHANNEL_ID while it performs a full table scan when the value 3 is used which is a much more optimal plan than the original one before gathering fresh statistics.

explain plan
for
SELECT
     S.cust_id
    ,S.prod_id
    ,SUM(S.amount_sold)
  FROM sh.sales S
 WHERE
channel_id=3
 GROUP BY S.cust_id, S.prod_id
 ORDER BY S.cust_id, S.prod_id
/  2    3    4    5    6    7    8    9   10   11   12

Explained.

SQL> select * from table(dbms_xplan.display);

PLAN_TABLE_OUTPUT
----------------------------------------------------------------------------------------------
Plan hash value: 3701591983

----------------------------------------------------------------------------------------------
| Id  | Operation            | Name  | Rows  | Bytes | Cost (%CPU)| Time     | Pstart| Pstop |
----------------------------------------------------------------------------------------------
|   0 | SELECT STATEMENT     |       |  4992 | 79872 |  2649   (4)| 00:00:32 |       |       |
|   1 |  SORT GROUP BY       |       |  4992 | 79872 |  2649   (4)| 00:00:32 |       |       |
|   2 |   PARTITION RANGE ALL|       |   541K|  8461K|  2601   (2)| 00:00:32 |     1 |    28 |
|*  3 |    TABLE ACCESS FULL | SALES |   541K|  8461K|  2601   (2)| 00:00:32 |     1 |    28 |
----------------------------------------------------------------------------------------------

SQL> explain plan
for
SELECT
     S.cust_id
    ,S.prod_id
    ,SUM(S.amount_sold)
  FROM sh.sales S
 WHERE
channel_id=9
 GROUP BY S.cust_id, S.prod_id
 ORDER BY S.cust_id, S.prod_id
/
  2    3    4    5    6    7    8    9   10   11   12
Explained.

SQL> select * from table(dbms_xplan.display);

PLAN_TABLE_OUTPUT
--------------------------------------------------------------------------------------------------------
Plan hash value: 3360497850

--------------------------------------------------------------------------------------------------------
-

| Id  | Operation                           | Name              | Rows  | Bytes | Cost (%CPU)| Time
|

-------------------------------------------------------------------------------------------------------
-

|   0 | SELECT STATEMENT                    |                   |   913 | 14608 |   315   (1)| 00:00:04
|

|   1 |  SORT GROUP BY                      |                   |   913 | 14608 |   315   (1)| 00:00:04
|

|   2 |   PARTITION RANGE ALL               |                   |  1423 | 22768 |   314   (0)| 00:00:04
|

|   3 |    TABLE ACCESS BY LOCAL INDEX ROWID| SALES             |  1423 | 22768 |   314   (0)| 00:00:04
|

|   4 |     BITMAP CONVERSION TO ROWIDS     |                   |       |       |            |
|

|*  5 |      BITMAP INDEX SINGLE VALUE      | SALES_CHANNEL_BIX |       |       |            |
|

--------------------------------------------------------------------------------------------------------

Since we have now confirmed that the plan is acceptable and the new statistics can be used by the optimizer, we PUBLISH the statistics now for the SALES table. Note that the LAST_ANALYZED column is also updated and the DBA_TAB_PENDING_STATS shows that there are no more pending statistics which need to be published.

SQL> EXEC DBMS_STATS.PUBLISH_PENDING_STATS ('SH','SALES');

PL/SQL procedure successfully completed.

SQL> SELECT LAST_ANALYZED FROM DBA_TABLES WHERE TABLE_NAME='SALES';

LAST_ANAL
---------
17-SEP-09

SELECT COUNT(*) FROM DBA_TAB_PENDING_STATS;

  COUNT(*)
----------
         0

SQL>  alter session set optimizer_use_pending_statistics=FALSE;

Session altered.

In versions prior to Oracle 11g, we were very likely to have faced the error shown below especially if we used range partitioning and the partition column was a date field.

We needed to ensure that we precreated all the partitions before hand based on the expected values of data that would be inserted (or updated) in a table.

ERROR at line 1:
ORA-14400: inserted partition key does not map to any partition

While we could use the MAXVALUE clause to create a ‘catch-all’ partition, this would not help us when we would like to perform any partition maintenance operations based on a date range or if wanted to use the partitioning feature to perform some kind of data archiving at the partition level.

11g Interval Partitioning

In Oracle 11g, the creation of partitions (for range) is automated and partitions are created as and when needed and takes the task of managing the creation of new partitions from the DBA. All that is required is to define the interval criteria and create the first partition. Subsequent partitions are created automatically based on the interval criteria.

create table mypart
   (ename varchar2(20), doj date)
   partition by range (doj)
   INTERVAL (NUMTOYMINTERVAL(1,'YEAR'))
   STORE IN (tbs1,tbs2)
  (partition p_2009 values less than (to_date('01-JAN-2010','DD-MON-YYYY'))
  )
;

Note the NUMTOYMINTERVAL is an SQL Funtion used to convert a number to an INTERVAL YEAR TO MONTH literal. The accepted values are ‘YEAR’ and ‘MONTH’.

The STORE IN clause will create in the partitions in a round robin manner in tablespaces tbs1 and tbs2 as we will see below.

Let us now insert some values into the table.

SQL> insert into mypart
  2   values
  3   ('Tom','21-SEP-2009');

1 row created.

SQL> insert into mypart
  2  values
  3  ('Joe','02-JAN-2010');

1 row created.

What has happened after the second insert? – a new partition ‘SYS_P42′ has been created for the year 2010 with a high value of ‘01-JAN-2011′.

The first partition that we had precreated has been created in the default tablespace USERS since no tablespace name has been prescribed and the new partition has been created in the tablespace TBS2. The next partition that comes along will be created in tablespace TBS1 and so on.

SQL> select partition_name,high_value from user_tab_partitions
  2  where table_name='MYPART';

PARTITION_NAME                 HIGH_VALUE
------------------------------ --------------------------------------------------------------------------------
P_2009                         TO_DATE(' 2010-01-01 00:00:00', 'SYYYY-MM-DD HH24:MI:SS', 'NLS_CALENDAR=GREGORIA
SYS_P42                        TO_DATE(' 2011-01-01 00:00:00', 'SYYYY-MM-DD HH24:MI:SS', 'NLS_CALENDAR=GREGORIA

SQL> select partition_name,tablespace_name from user_tab_partitions
  2  where table_name='MYPART';

PARTITION_NAME                 TABLESPACE_NAME
------------------------------ ------------------------------
P_2009                         USERS
SYS_P43                        TBS2

We can use the ALTER TABLE SET INTERVAL command to convert a range partitioned table to an interval partitioned table as shown below.

SQL> create table mypart2
   (ename varchar2(20), doj date)
   partition by range (doj)
  (partition p_2009 values less than (to_date('01-JAN-2010','DD-MON-YYYY'))
  )
;    2    3    4    5    6

Table created.

SQL> alter table mypart2
  2  SET INTERVAL (NUMTOYMINTERVAL(1,'YEAR');

Table altered.

We need to keep the following points in mind when using Interval Partitioning:

Analyze the impact of change on SQL statements using the SQL Performance Analyzer

In the earlier post, we had discussed the Database Capture and Replay component of 11g Real Application Testing – (http://gavinsoorma.wordpress.com/2009/09/11/11g-real-application-testing-making-changes-with-confidence/)

We can also analyse the impact of upgrading from 10.2.0.4 to 11.1.0.6 at the SQL statement level by using the other component of 11g Real Application Testing which is called SQL Performance Analyzer or SPA.

We will use the following two SQL statements to create a SQL Tuning Set (STS) in the 10.2.0.4 environment. This STS is then transported to the 11g target environment and then we will use the GUI Database Control to replay the statements in the STS and generate a Performance Analysis report which will help us identify if any statements have improved or digressed in the changed environment.

SELECT /*+TEST_SPA*/ p.prod_name, s.time_id, t.week_ending_day, SUM(s.amount_sold)
FROM sales s, products p, times t
WHERE s.time_id=t.time_id
AND s.prod_id = p.prod_id
GROUP BY p.prod_name, s.time_id, t.week_ending_day;

SELECT /*+TEST_SPA*/ p.prod_category, t.week_ending_day, s.cust_id, SUM(s.amount_sold)
FROM sales s, products p, times t
WHERE s.time_id = t.time_id
AND s.prod_id = p.prod_id
AND p.prod_category = ‘Peripherals and Accessories’
GROUP BY p.prod_category, t.week_ending_day, s.cust_id;

We are using a ‘tag’ TEST_SPA to help us easily identify these SQL statements in the shared pool.

Source 10.2.0.4 environment – Create and Export SQL Tuning Set

SQL> exec dbms_sqltune.drop_sqlset(‘SH_10204′);

PL/SQL procedure successfully completed.

SQL> exec dbms_sqltune.create_sqlset(‘SH_10204′,’Demo Workload to test SPA’);

PL/SQL procedure successfully completed.

SQL> DECLARE
2 stscur dbms_sqltune.sqlset_cursor;
3 BEGIN
4 OPEN stscur FOR
5 SELECT VALUE(P)
6 FROM TABLE(dbms_sqltune.select_cursor_cache(
7 ’sql_text like ”SELECT /*+TEST_SPA%”’,
8 null, null, null, null, null, null, ‘ALL’)) P;
9 — populate the sqlset
10 dbms_sqltune.load_sqlset(:sts_name, stscur);
11 end;
12 /

PL/SQL procedure successfully completed.

SQL> select sql_id,plan_hash_value, substr(sql_text,1,40) text, executions
2 from dba_sqlset_statements
3 where sqlset_name =’SH_10204′
4 order by sql_id, plan_hash_value;

SQL_ID PLAN_HASH_VALUE TEXT EXECUTIONS
————- ————— ————————————————– ———-
5930m31ryy4b1 3901800458 SELECT /*+TEST_SPA*/ p.prod_category, t. 1
5tbr81fmrnyc0 1679814994 SELECT /*+TEST_SPA*/ p.prod_name, s.time 1

In order to transport the SQL tuning set that we have just created, we need to create a staging table to hold the contents of the STS and then we will use the export utility to generate a dump file which will then be copied to the target 11g environment.

SQL> execute DBMS_SQLTUNE.CREATE_STGTAB_SQLSET(table_name =>’SH_STS’);

PL/SQL procedure successfully completed.

SQL> execute DBMS_SQLTUNE.PACK_STGTAB_SQLSET(sqlset_name => ‘SH_10204′,staging_table_name => ‘SH_STS’);

PL/SQL procedure successfully completed.

SQL> select count (*) from sh_sts;

COUNT(*)
———-
2

Read More …..

CASE STUDY:

Using 11g Real Application Testing to analyse the impact of making the following changes:

1) Upgrade the database from 10g to 11g
2) Convert from single instance database to a two node RAC database
3) Convert from file system based storage to 11g ASM

Overview of Real Application Testing

One of the best new features in Oracle 11g is the Real Application Testing option which enables us to test the result of a change without actually implementing the same in a production environment. Changes such as database version upgrades, optimizer changes, hardware and storage changes all carry with them an element of risk and uncertainty. Using this feature much of that uncertainty and guesswork can be avoided.

Real Application Testing comprises of two components:

a) Database Replay
b) SQL Performance Analyzer (SPA)

SPA enables us to test the impact of a change at the level of a particular SQL statement to see if any improvements or regressions occurred in the execution of the SQL statement because of the change.

Database Replay enables us to capture workload real time in a production environment and then replay the same in a clone or copy of the changed production environment with the exact timings, concurrency and transaction characteristics found in the original workload. The AWR comparison and Database Replay reports can easily help us pinpoint any potential problems and errors as well any divergence in performance.

This feature will afford businesses increased confidence and surety when faced with a system change as well as significantly reducing the costs as well as resources associated with the testing phase.

Database Capture is now supported in versions 9.2.0.8 as well as 10.2.0.3 and 10.2.0.4 enabling us to test the impact of upgrades from both 9i as well as 10g to 11g. Note, however, that replay can only be performed in a Oracle 11g database.

At a very high level, the database capture and replay will involve the following steps:

1) Prepare the source database for capture
2) Capture the workload
3) Copy the workload capture files to the target machine
4) Prepare the target database for replay and process captured workload files
5) Start the workload replay clients
6) Once the workload replay has finished, examine the generated reports and make the analysis

Read More ….

Although table compression was introduced in Oracle 9i and 10g, it was aimed more at bulk load operations for data warehouse environments. The overhead associated with compression and uncompression made it unsuitable for OLTP type shops.

In Oracle 11g, Advanced Compression (which is an additional licensed feature) includes OLTP compression capabilities as well as compression of unstructured data like images and text with Secure Files and also direct compression of the export dump file generated by Data Pump which is also directly read without any uncompress while doing an import.

The compression feature has been greatly enhanced so as to remove any overhead associated with dealing with compressed data while performing any DML activities.The COMPRESS FOR ALL OPERATIONS keyword needs to be used for enabling OLTP data compression.

While Oracle does claim compression ratios of 3:1, we were able to see clearly 2:1 ratios in terms of storage reductions and no real performance overhead while performing deletes and inserts on compressed data as shown in the example below.

So this feature could provide us both Storage as well as Performance gains due to the reduced number of blocks which will need to be read while performing I/O.

UNCOMPRESSED DATA

SQL> create table myobjects
2 as select * from all_objects;

Table created.

SQL> select sum(bytes)/1048576 from user_segments where segment_name=’MYOBJECTS’;

SUM(BYTES)/1048576
——————
21

SQL> insert into myobjects
2 select * from all_objects;

56261 rows created.

Elapsed: 00:00:09.39

SQL> select count(*) from myobjects where object_type=’TABLE’;

COUNT(*)
———-
444

Elapsed: 00:00:00.66

SQL> delete myobjects;

168780 rows deleted.

Elapsed: 00:00:12.22

COMPRESSED DATA

SQL> create table myobjects_compress
2 compress for all operations
3 as select * from all_objects;

SQL> select sum(bytes)/1048576 from user_segments where segment_name=’MYOBJECTS_COMPRESS’;

SUM(BYTES)/1048576
——————
9

SQL> insert into myobjects_compress
2 select * from all_objects;

56261 rows created.

Elapsed: 00:00:09.08

SQL> select count(*) from myobjects_compress where object_type=’TABLE’;

COUNT(*)
———-
444

Elapsed: 00:00:00.21

SQL> delete myobjects_compress;

168783 rows deleted.

Elapsed: 00:00:10.96

Compression of Data Pump Exports

While doing an export, we can use the parameter COMPRESSION=ALL (other options are DATA_ONLY or METADATA) and we see that the size of the dumpfile has reduced very significantly while using this compression feature.

The uncompressed dumpfile was 113 MB while the compressed dumpfile was just 13 MB. This feature can be very useful when we need to take a large export on disk but are limited by disk space. The earlier methods required us to use a pipe while performing an export, but we had to then uncompress the file while performing the import as well which is a very costly operation in terms of time and resources.

$ expdp directory=data_file_dir dumpfile=uncompress.dmp schemas=SH compression=ALL

middba1:/u01/oracle/product/11.1.0/db/demo/schema/sales_history> ls -l *.dmp
-rw-rw—- 1 oracle dba 113610752 Sep 3 12:25 uncompress.dmp
-rw-rw—- 1 oracle dba 13426688 Sep 3 12:21 compress.dmp

I have created a simple but very useful Unix shell script driven menu to enable easy monitoring of any number of Oracle databases from a single central location. This covers most of the normal DBA day to day standard monitoring requirements as well as a good interface for aiding in performance tuning.

Set up is pretty simple requiring a single user called MONITOR in each database with SELECT privilege on the data dictionary tables.

Drop me an email at gavin.soorma@bankwest.com.au so that I can send you the zip file along with the installation instructions.

A few screen shots of the various menu options are shown below

       #######################################################
                        [DBA Menu - PRDU010]

                        Connected to Database- cpsprd
        #######################################################

        (1) Database Overview
        (2) Database Storage
        (3) Database Objects
        (4) Database Security
        (5) Database Monitor
        (6) Database Performance

        Enter your choice (q to quit):

        ########################################################

        #######################################################
                        Database Monitoring Menu - cpsprd
        #######################################################

        (1)  User Sessions
        (2)  Redo Log File Switches
        (3)  Current SQL (by Unix PID)
        (4)  Current SQL (by SID)
        (5)  Current SQL (by Oracle Username)
        (6)  Rollback Segment Usage
        (7)  Locked Sessions
        (8)  Locked Objects
        (9)  Datafile I/O
        (10) DBMS Jobs Status
        (11) View Long Operations
        (12) Active Transactions
        (13) Idle Sessions
        (14) Flashback Logging
        (15) Sessions With High I/O
        (16) Data Guard Log Shipping
        (17) Redo Generation By Day

        Enter your choice (q to quit):

        #######################################################

             #######################################################
                        Database Performance Menu - cpsprd
        	  #######################################################

        (1)  Waits - Instance Level (by time waited)
        (2)  Waits - By Class
        (3)  Waits - Waiting Session's SQL
        (4)  Waits - What are users currently waiting on?
        (5)  Waits - Datafile Level
        (6)  Waits - Block & Object Level
        (7)  Locked Sessions
        (8)  Large Objects In Shared Pool
        (9)  Hit Ratios - Quick Check
        (10) Top 5 Problem Queries (Disk Reads)
        (11) Top 5 Problem Queries (Buffer Gets)
        (12) Top 5 Problem Queries (Disk Reads/Executions)
        (13) Top 5 Queries (Executions)
        (14) Top 5 Latch Waits (by sleeps)
        (15) Top 5 Latch Waits (by misses)
        (16) Current Waits On Latches
        (17) Enqueue Waits - By Session
        (18) Enqueue Waits - By Database
        (19) Enqueue Waits - By Object
        (20) Key CBO init.ora Parameters
        (21) Tables ANALYZE Status
        (22) Indexes ANALYZE Status
        (23) Top Segments with most I/O

        Enter your choice (q to quit):

        #######################################################

One of the very good new features in Oracle 11g is the Snapshot Standby database where we can basically convert the physical standby database into a read-write real time clone of the production database and we can then temporarily use this environment for carrying out any kind of development testing, QA type work or to test the impact of a proposed production change on an application.

Basically, the Snapshot Standby database in turn uses the Flashback Database technology to create a guaranteed restore point to which the database can be later flashed back to.

So we need to keep in mind that whatever limitations are present in the Flashback database technology, the same will also be inherent in the Snapshot database fetaure.

Let us illustrate this feature by using an example where we would like to test the impact of the change of some important optimizer related parameters on a key application query using the actual production data.

In production, the SQL query below is shown to be peforming a full table/partition scan and using hash joins. We would like to see if a nested loop join and index usage will prove beneficial to this SQL query that is used frequently by the application.

explain plan for
SELECT c.cust_last_name, sum(s.amount_sold) AS dollars,sum(s.quantity_sold) as quantity
FROM sales s , customers c, products p
WHERE c.cust_id = s.cust_id
AND s.prod_id = p.prod_id
AND c.cust_state_province IN ('Dublin','Galway')GROUP BY c.cust_last_name
/

SQL> select * from table(dbms_xplan.display);

PLAN_TABLE_OUTPUT
------------------------------------------------------------------------------------------------------
Plan hash value: 1248656060

------------------------------------------------------------------------------------------------------
| Id  | Operation              | Name        | Rows  | Bytes | Cost (%CPU)| Time     | Pstart| Pstop |
------------------------------------------------------------------------------------------------------
|   0 | SELECT STATEMENT       |             |   519 | 23355 |   935   (6)| 00:00:12 |       |       |
|   1 |  HASH GROUP BY         |             |   519 | 23355 |   935   (6)| 00:00:12 |       |       |
|*  2 |   HASH JOIN            |             | 99644 |  4378K|   927   (5)| 00:00:12 |       |       |
|   3 |    INDEX FULL SCAN     | PRODUCTS_PK |    72 |   288 |     1   (0)| 00:00:01 |       |       |
|*  4 |    HASH JOIN           |             | 99644 |  3989K|   924   (5)| 00:00:12 |       |       |
|*  5 |     TABLE ACCESS FULL  | CUSTOMERS   |   766 | 18384 |   408   (1)| 00:00:05 |       |       |

PLAN_TABLE_OUTPUT
------------------------------------------------------------------------------------------------------
|   6 |     PARTITION RANGE ALL|             |   918K|    14M|   507   (6)| 00:00:07 |     1 |    28 |
|   7 |      TABLE ACCESS FULL | SALES       |   918K|    14M|   507   (6)| 00:00:07 |     1 |    28 |
------------------------------------------------------------------------------------------------------

We will now convert the physical standby database to a Snapshot Standby database and note that after the conversion takes place, internally a guaranteed restore point has been created and the database has been opened in read-write mode. Also note the DATABASE_ROLE column.

SQL>  ALTER DATABASE CONVERT TO SNAPSHOT STANDBY;

Database altered.

SQL> shutdown immediate;
ORA-01507: database not mounted

ORACLE instance shut down.
SQL> startup;
ORACLE instance started.

Total System Global Area  521936896 bytes
Fixed Size                  2139784 bytes
Variable Size             415238520 bytes
Database Buffers           96468992 bytes
Redo Buffers                8089600 bytes
Database mounted.
Database opened.

SQL> select NAME,GUARANTEE_FLASHBACK_DATABASE from  v$restore_point;

NAME                                                         GUA
------------------------------------------------------------ ---
SNAPSHOT_STANDBY_REQUIRED_09/02/2009 10:12:21                YES

SQL> select open_mode,database_role from v$database;

OPEN_MODE  DATABASE_ROLE
---------- ----------------
READ WRITE SNAPSHOT STANDBY

We will now make some changes to some optimizer related parameters:

SQL> alter system set optimizer_index_cost_adj=20;

System altered.

SQL> alter system set optimizer_index_caching=10;

System altered.

SQL> alter system set optimizer_mode=FIRST_ROWS;

System altered.

We run the same query again and we can see that the plan has changed and the optimizer is now using the index scan with nested loop joins.

--------------------------------------------------------------------------------------------------------------
| Id  | Operation                             | Name                 | Rows  | Bytes | Cost (%CPU)| Time     |
--------------------------------------------------------------------------------------------------------------
|   0 | SELECT STATEMENT                      |                      |   519 | 23355 |   588  (11)| 00:00:08 |
|   1 |  HASH GROUP BY                        |                      |   519 | 23355 |   588  (11)| 00:00:08 |
|   2 |   NESTED LOOPS                        |                      | 99644 |  4378K|   580  (10)| 00:00:07 |
|*  3 |    HASH JOIN                          |                      | 99644 |  3989K|   577  (10)| 00:00:07 |
|*  4 |     TABLE ACCESS BY INDEX ROWID       | CUSTOMERS            |   766 | 18384 |   241   (2)| 00:00:03 |
|   5 |      BITMAP CONVERSION TO ROWIDS      |                      |       |       |            |          |
|   6 |       BITMAP INDEX FULL SCAN          | CUSTOMERS_GENDER_BIX |       |       |            |          |
|   7 |     PARTITION RANGE ALL               |                      |   918K|    14M|   326  (12)| 00:00:04 |
|   8 |      TABLE ACCESS BY LOCAL INDEX ROWID| SALES                |   918K|    14M|   326  (12)| 00:00:04 |
|   9 |       BITMAP CONVERSION TO ROWIDS     |                      |       |       |            |          |
|  10 |        BITMAP INDEX FULL SCAN         | SALES_PROMO_BIX      |       |       |            |          |
|* 11 |    INDEX UNIQUE SCAN                  | PRODUCTS_PK          |     1 |     4 |     1   (0)| 00:00:01 |
--------------------------------------------------------------------------------------------------------------

While the original physical standby database has been now converted to snapshot database, some changes are happening on the Primary database and those changes are shipped to the standby site but not yet applied. They will accumulate on the standby site and will be applied after the snapshot standby database is converted back to a physical standby database.

SQL> create table mytest as select * from sales;

Table created.

SQL> update mytest set prod_id=1;

918843 rows updated.

On the standby site, we will now convert the shapshot standby database to the original mode of physical standby database in Active Data Guard mode.

Note that after the physical standby database is opened, changes that were made on the Primary database while it was open in shapshot standby mode are now applied as well.

SQL> shutdown immediate;
Database closed.
Database dismounted.
ORACLE instance shut down.
SQL> startup mount;
ORACLE instance started.

Total System Global Area  521936896 bytes
Fixed Size                  2139784 bytes
Variable Size             415238520 bytes
Database Buffers           96468992 bytes
Redo Buffers                8089600 bytes
Database mounted.
SQL> ALTER DATABASE CONVERT TO PHYSICAL STANDBY;

Database altered.

SQL> shutdown immediate;
ORA-01507: database not mounted

ORACLE instance shut down.

SQL> startup mount;
ORACLE instance started.

Total System Global Area  521936896 bytes
Fixed Size                  2139784 bytes
Variable Size             415238520 bytes
Database Buffers           96468992 bytes
Redo Buffers                8089600 bytes
Database mounted.
SQL>  recover managed standby database disconnect;
Media recovery complete.
SQL> recover managed standby database cancel;
Media recovery complete.
SQL> alter database open;

Database altered.

Elapsed: 00:01:06.29
SQL> recover managed standby database using current logfile disconnect;
Media recovery complete.

SQL> select count(*) from sh.mytest;

  COUNT(*)
----------
    918843

SQL> select distinct prod_id from sh.mytest;

   PROD_ID
----------
         1

This example demonstrates how to transport a tablespace from a 10g ASM on Windows environment to 11g ASM on AIX environment using the DBMS_FILE_TRANSFER package.

The source database on Windows is GENUA1 and the target database is a 11g RAC database RACDB1.

Create the tablespace and the test object in the source database

SQL> create tablespace test_asm
  2  datafile size 100m;

Tablespace created.

SQL> select file_name from dba_data_files
  2  where tablespace_name='TEST_ASM';

FILE_NAME
--------------------------------------------------------------------------------

+DATA/genua1/datafile/test_asm.376.696333127

SQL> conn system/xxx
Connected.

SQL> create table myobjects
  2  tablespace test_asm
  3  as select * from dba_objects;

Table created.

SQL> select count(*) from system.myobjects;

  COUNT(*)
----------
     56504

Create a database link from the source database to the target database

SQL> CREATE DATABASE LINK  DBS2 CONNECT TO
  2  system identified by oracle
  3  using 'racdb1';

Database link created.

SQL> select * from dual@dbs2;

D
-
X

Create a directory to hold the converted datafile on the source

SQL> create directory source_dir as '+DATA/genua1/xtransport/';

Directory created.

Create a directory to hold the tablespace meta data dump on the source

SQL> create directory source_dir_2 as '+DATA';

Directory created.

Create an O/S level directory for the data pump export logfile

SQL> create directory test_asm_log as 'd:\oracle\';

Directory created.

Create a directory on the target to hold the transported datafile

SQL> CREATE OR REPLACE DIRECTORY target_dir  AS '+DATA';

Directory created.

Create an O/S level directory for the data pump import logfile

 create directory test_asm_log as '/u01/oracle/';

Directory created.

Make the tablespace which is going to be transported read only

SQL> alter tablespace test_asm read only;

Tablespace altered.

Export the tablespace meta data using Data Pump

C:\Documents and Settings\bwdba_cbduat>expdp directory=source_dir_2 dumpfile=test_asm.dmp transport_tablespaces=TEST_ASM transport_full_check=Y logfile=test_asm_log:exp.log

Export: Release 10.2.0.4.0 - Production on Monday, 31 August, 2009 10:41:34

Copyright (c) 2003, 2007, Oracle.  All rights reserved.

Username: system
Password:

Connected to: Oracle Database 10g Enterprise Edition Release 10.2.0.4.0 - Produc
tion
With the Partitioning, Real Application Clusters, OLAP, Data Mining
and Real Application Testing options
Starting "SYSTEM"."SYS_EXPORT_TRANSPORTABLE_01":  system/******** directory=sour
ce_dir_2 dumpfile=test_asm.dmp transport_tablespaces=TEST_ASM transport_full_che
ck=Y logfile=test_asm_log:exp.log
Processing object type TRANSPORTABLE_EXPORT/PLUGTS_BLK
Processing object type TRANSPORTABLE_EXPORT/TABLE
Processing object type TRANSPORTABLE_EXPORT/INDEX
Processing object type TRANSPORTABLE_EXPORT/INDEX_STATISTICS
Processing object type TRANSPORTABLE_EXPORT/POST_INSTANCE/PLUGTS_BLK
Master table "SYSTEM"."SYS_EXPORT_TRANSPORTABLE_01" successfully loaded/unloaded

******************************************************************************
Dump file set for SYSTEM.SYS_EXPORT_TRANSPORTABLE_01 is:
  +DATA/test_asm.dmp
Job "SYSTEM"."SYS_EXPORT_TRANSPORTABLE_01" successfully completed at 10:42:16

Use DBMS_FILE_TRANSFER to transfer the export dump file to the target

SQL> begin
  2   dbms_file_transfer.put_file
  3   (source_directory_object => 'source_dir_2',
  4   source_file_name => 'test_asm.dmp',
  5   destination_directory_object => 'target_dir',
  6  destination_file_name => 'test_asm.dmp',
  7   destination_database => 'dbs2');
  8   end;
  9  /

PL/SQL procedure successfully completed.

Use RMAN to convert the tablespace to the target AIX 64 bit format

Note that the converted datafile will reside in the directory we created in an earlier step - +DATA/genua1/xtransport

RMAN> convert tablespace test_asm to
2> platform 'AIX-Based Systems (64-bit)'
3> format '+DATA';

Starting backup at 31/AUG/09
configuration for DISK channel 2 is ignored
configuration for DISK channel 3 is ignored
configuration for DISK channel 4 is ignored
allocated channel: ORA_DISK_1
channel ORA_DISK_1: sid=470 instance=genua11 devtype=DISK
channel ORA_DISK_1: starting datafile conversion
input datafile fno=00224 name=+DATA/genua1/datafile/test_asm.376.696333127
converted datafile=+DATA/genua1/xtransport/test_asm.266.696338369
channel ORA_DISK_1: datafile conversion complete, elapsed time: 00:00:07
Finished backup at 31/AUG/09

Use DBMS_FILE_TRANSFER to copy the converted datafile to the target

SQL> begin
  2    dbms_file_transfer.put_file
  3    (source_directory_object => 'source_dir',
  4     source_file_name => 'test_asm.266.696338369',
  5    destination_directory_object => ' target_dir',
  6    destination_file_name => 'tts1_db1.dbf',
  7     destination_database => 'dbs2');
  8    end;
  9     /

PL/SQL procedure successfully completed.

On the target import the datafile meta data using Data Pump

The imp.par import parameter file has the following contents 

directory=target_dir
dumpfile=test_asm.dmp
logfile=test_asm_log:imp.log
TRANSPORT_DATAFILES='+DATA1/tts1_db1.dbf'
keep_master=y

middba1:/u01/oracle> impdp parfile=imp.par

Import: Release 11.1.0.6.0 - 64bit Production on Monday, 31 August, 2009 11:39:32

Copyright (c) 2003, 2007, Oracle.  All rights reserved.

Username: system
Password:

Connected to: Oracle Database 11g Enterprise Edition Release 11.1.0.6.0 - 64bit Production
With the Partitioning, Real Application Clusters and Real Application Testing options
Master table "SYSTEM"."SYS_IMPORT_TRANSPORTABLE_02" successfully loaded/unloaded
Starting "SYSTEM"."SYS_IMPORT_TRANSPORTABLE_02":  system/******** parfile=imp.par
Processing object type TRANSPORTABLE_EXPORT/PLUGTS_BLK
Processing object type TRANSPORTABLE_EXPORT/TABLE
Processing object type TRANSPORTABLE_EXPORT/INDEX
Processing object type TRANSPORTABLE_EXPORT/INDEX_STATISTICS
Processing object type TRANSPORTABLE_EXPORT/POST_INSTANCE/PLUGTS_BLK
Job "SYSTEM"."SYS_IMPORT_TRANSPORTABLE_02" successfully completed at 11:39:48

On the target database RACDB1, check that the TEST_ASM tablespace has been plugged in and the MYOBJECTS table is present

middba1:/u01/oracle> sql

SQL*Plus: Release 11.1.0.6.0 - Production on Mon Aug 31 11:41:29 2009

Copyright (c) 1982, 2007, Oracle.  All rights reserved.

Connected to:
Oracle Database 11g Enterprise Edition Release 11.1.0.6.0 - 64bit Production
With the Partitioning, Real Application Clusters and Real Application Testing options

SQL> select file_name from dba_data_files where tablespace_name='TEST_ASM';

FILE_NAME
--------------------------------------------------------------------------------
+DATA/tts1_db1.dbf

SQL> select count(*) from system.myobjects;

  COUNT(*)
----------
     56504

By default, when we are using ASM with a Flash Recovery Area configured, backups to disk will go to the “backupset” subdirectory – unless we use the FORMAT clause while taking the backup to point the backups to another location on disk.

When we try to duplicate a database which is hosted on an ASM storage to a conventional O/S file system, the restore will fail as it will look for the ASM diskgroups where the backup was taken on the target node as well.

We will see an error like …..

channel ORA_AUX_DISK_1: reading from backup piece +DBATEAM/bwdba/backupset/2009_08_25/nnndf0_tag20090825t131155_0.388.695826727
channel ORA_AUX_DISK_1: ORA-19870: error while restoring backup piece +DBATEAM/bwdba/backupset/2009_08_25/nnndf0_tag20090825t131155_0.388.695826727
ORA-19505: failed to identify file “+DBATEAM/bwdba/backupset/2009_08_25/nnndf0_tag20090825t131155_0.388.695826727″
ORA-17503: ksfdopn:2 Failed to open file +DBATEAM/bwdba/backupset/2009_08_25/nnndf0_tag20090825t131155_0.388.695826727
ORA-15001: diskgroup “DBATEAM” does not exist or is not mounte
failover to previous backup

In this case we will need to use the BACKUPSET clause of the BACKUP command to relocate that backup from the ASM storage to a O/S file system. We then ftp this backupset to the target node ensuring that the same directory file structure is available on the target node as compared to the source node where the ASM disk backup was restored. We can then proceed as normal with the RMAN Duplicate database procedure.

RMAN> list backup of database;

List of Backup Sets
===================

BS Key  Type LV Size       Device Type Elapsed Time Completion Time
------- ---- -- ---------- ----------- ------------ ---------------
14      Full    1.40G      DISK        00:00:47     26-AUG-09
        BP Key: 14   Status: AVAILABLE  Compressed: NO  Tag: TAG20090826T102022
        Piece Name: +DBATEAM/bwdba/backupset/2009_08_26/nnndf0_tag20090826t102022_0.397.695902837
  List of Datafiles in backup set 14
  File LV Type Ckp SCN    Ckp Time  Name
  ---- -- ---- ---------- --------- ----
  1       Full 2765425    26-AUG-09 +DBATEAM/bwdba/datafile/system.256.695305997
  2       Full 2765425    26-AUG-09 +DBATEAM/bwdba/datafile/sysaux.257.695305999
  3       Full 2765425    26-AUG-09 +DBATEAM/bwdba/datafile/undotbs1.258.695305999
  4       Full 2765425    26-AUG-09 +DBATEAM/bwdba/datafile/users.259.695305999
  5       Full 2765425    26-AUG-09 +DBATEAM/bwdba/datafile/example.267.695306155
  6       Full 2765425    26-AUG-09 +DBATEAM/bwdba/datafile/undotbs2.268.695306391

RMAN> backup device type disk format '/u01/oracle/backup/bwdba_bkp.%u'
2> backupset 14
3> ;

Starting backup at 26-AUG-09
allocated channel: ORA_DISK_1
channel ORA_DISK_1: SID=108 instance=bwdba1 device type=DISK
input backup set count=15 STAMP=695902823 creation_time=26-AUG-09
channel ORA_DISK_1: starting piece 1 at 26-AUG-09
channel ORA_DISK_1: backup piece +DBATEAM/bwdba/backupset/2009_08_26/nnndf0_tag20090826t102022_0.397.695902837
piece handle=/u01/oracle/backup/bwdba_bkp.0fknl8j7 comment=NONE
channel ORA_DISK_1: finished piece 1 at 26-AUG-09
channel ORA_DISK_1: backup set complete, elapsed time: 00:00:25
Finished backup at 26-AUG-09

We will see now that the backup location has changed

RMAN> list backup of database;

List of Backup Sets
===================

BS Key  Type LV Size       Device Type Elapsed Time Completion Time
------- ---- -- ---------- ----------- ------------ ---------------
14      Full    1.40G      DISK        00:00:47     26-AUG-09
        BP Key: 18   Status: AVAILABLE  Compressed: NO  Tag: TAG20090826T102022
        Piece Name: /u01/oracle/backup/bwdba_bkp.0fknl8j7
  List of Datafiles in backup set 14
  File LV Type Ckp SCN    Ckp Time  Name
  ---- -- ---- ---------- --------- ----
  1       Full 2765425    26-AUG-09 +DBATEAM/bwdba/datafile/system.256.695305997
  2       Full 2765425    26-AUG-09 +DBATEAM/bwdba/datafile/sysaux.257.695305999
  3       Full 2765425    26-AUG-09 +DBATEAM/bwdba/datafile/undotbs1.258.695305999
  4       Full 2765425    26-AUG-09 +DBATEAM/bwdba/datafile/users.259.695305999
  5       Full 2765425    26-AUG-09 +DBATEAM/bwdba/datafile/example.267.695306155
  6       Full 2765425    26-AUG-09 +DBATEAM/bwdba/datafile/undotbs2.268.695306391

While one of the performance best practices that is always recommended is the use of bind variables over the literals, Optimizer plan unstability was one of the problems faced by many Oracle 10g shops. When queries that normally perform well, suddenly stopped performing, very often it is found that this bad performance is being observed only for a few or specific parameters or values. And this is particularly so on tables with highly skewed data especially the columns which are being referenced via the WHERE clause.

In Oracle 11g however, with Adaptive Cursor Sharing, if the optimizer observes a sub-optimal plan for a particular bind variable value, it will enable particular bind variables or range of bind variable values to use a different execution plan for the same SQL statement.

To illustrate the 11g Adaptive Cursor Sharing, we will create a table called MYOBJECTS which is based on the ALL_OBJECTS view. The data in the OBJECT_TYPE column is skewed – for example there are more than 50,000 rows with the value ‘SYNONYM’ and only 1 row with the value ‘RULE’. The table has about 100,000 rows.

SQL> conn sh/sh
Connected.

SQL> create table myobjects
  2  as select * from all_objects;

Table created.

SQL> create index myobject_ind on myobjects (object_type);

Index created.

We then gather statistics on the table and see that histograms have been created on the table which has the skewed data.

SQL> exec dbms_stats.gather_table_stats(null,'MYOBJECTS', METHOD_OPT => 'FOR ALL COLUMNS SIZE 10');

PL/SQL procedure successfully completed.

SQL> SELECT column_name, histogram FROM user_tab_cols WHERE  table_name ='MYOBJECTS';

COLUMN_NAME                    HISTOGRAM
------------------------------ ---------------
OWNER                          HEIGHT BALANCED
OBJECT_NAME                    HEIGHT BALANCED
SUBOBJECT_NAME                 HEIGHT BALANCED
OBJECT_ID                      HEIGHT BALANCED
DATA_OBJECT_ID                 HEIGHT BALANCED
OBJECT_TYPE                    HEIGHT BALANCED
CREATED                        HEIGHT BALANCED
LAST_DDL_TIME                  HEIGHT BALANCED
TIMESTAMP                      HEIGHT BALANCED
STATUS                         FREQUENCY
TEMPORARY                      FREQUENCY
GENERATED                      FREQUENCY
SECONDARY                      FREQUENCY

If we query the MYOBJECTS Table using the predicate ‘RULE’, we find that an INDEX RANGE SCAN using the MYOBJECTS_IND index is being carried out by the optimizer.

If we change the predicate to ‘SYNONYM’ we find that a FULL TABLE SCAN is being performed instead which is the expected behaviour given the distribution of rows in the table.

SQL> explain plan for
  2  select * from myobjects where object_type='RULE';

Explained.

SQL> set linesize 120
SQL> select * from table(dbms_xplan.display);

PLAN_TABLE_OUTPUT
---------------------------------------------------------------------------------------------
Plan hash value: 2533318700

--------------------------------------------------------------------------------------------
| Id  | Operation                   | Name         | Rows  | Bytes | Cost (%CPU)| Time     |
--------------------------------------------------------------------------------------------
|   0 | SELECT STATEMENT            |              |  1612 |   147K|    56   (0)| 00:00:01 |
|   1 |  TABLE ACCESS BY INDEX ROWID| MYOBJECTS    |  1612 |   147K|    56   (0)| 00:00:01 |
|*  2 |   INDEX RANGE SCAN          | MYOBJECT_IND |  1612 |       |     5   (0)| 00:00:01 |
--------------------------------------------------------------------------------------------

SQL>  explain plan for
  2  select * from myobjects where object_type='SYNONYM';

Explained.

SQL> select * from table(dbms_xplan.display);

PLAN_TABLE_OUTPUT
------------------------------------------------------------------------------------------------
Plan hash value: 2581838392

-------------------------------------------------------------------------------
| Id  | Operation         | Name      | Rows  | Bytes | Cost (%CPU)| Time     |
-------------------------------------------------------------------------------
|   0 | SELECT STATEMENT  |           | 34398 |  3157K|   284   (2)| 00:00:04 |
|*  1 |  TABLE ACCESS FULL| MYOBJECTS | 34398 |  3157K|   284   (2)| 00:00:04 |
-------------------------------------------------------------------------------

Let us now see how the optimizer handles bind variables and how it performs the task of bind variable peeking. In the first case, the variable v_obj_type is assigned the value ‘SYNONYM’ and in the second case the same variable is assigned the value ‘RULE’.

Remember, when literals were being used, using the value ‘RULE’ in the WHERE clause resulted in an index range scan, while usage of the value ‘SYNONYM’ resulted in a full table scan.

SQL> VARIABLE v_obj_type VARCHAR2(10)
SQL> EXEC :v_obj_type := 'SYNONYM';

PL/SQL procedure successfully completed.

SQL> select * from myobjects where object_type=:v_obj_type;

40156 rows selected.

SQL> select * from table(dbms_xplan.display_cursor);

PLAN_TABLE_OUTPUT
-----------------------------------------------------------------------------------------------
SQL_ID  5utj7m6zxjdsc, child number 0
-------------------------------------
select * from myobjects where object_type=:v_obj_type

Plan hash value: 2581838392

-------------------------------------------------------------------------------
| Id  | Operation         | Name      | Rows  | Bytes | Cost (%CPU)| Time     |
-------------------------------------------------------------------------------
|   0 | SELECT STATEMENT  |           |       |       |   284 (100)|          |
|*  1 |  TABLE ACCESS FULL| MYOBJECTS | 34398 |  3157K|   284   (2)| 00:00:04 |
-------------------------------------------------------------------------------

Although the predicate is now the value ‘RULE’ based on the bind variable that is being used here, we find that the optimizer is still performing a FULL TABLE SCAN, while in fact it should be actually performing an index scan.

But since bind peeking is being performed here, the optimizer will use the same plan even though the value of the variable is different. If the data is skewed as in this case, it can cause a lot of performance problems and optimizer unstability depending on which bind variable value is assigned at hard parse.

SQL> VARIABLE v_obj_type VARCHAR2(10)
SQL> EXEC :v_obj_type := 'RULE';

PL/SQL procedure successfully completed.

SQL> select * from myobjects where object_type=:v_obj_type;

no rows selected

SQL> /

no rows selected

SQL> /

no rows selected

SQL>  select * from table(dbms_xplan.display_cursor);

PLAN_TABLE_OUTPUT
----------------------------------------------------------------------------------------------
SQL_ID  5utj7m6zxjdsc, child number 0
-------------------------------------
select * from myobjects where object_type=:v_obj_type

Plan hash value: 2581838392

-------------------------------------------------------------------------------
| Id  | Operation         | Name      | Rows  | Bytes | Cost (%CPU)| Time     |
-------------------------------------------------------------------------------
|   0 | SELECT STATEMENT  |           |       |       |   284 (100)|          |
|*  1 |  TABLE ACCESS FULL| MYOBJECTS | 34398 |  3157K|   284   (2)| 00:00:04 |
-------------------------------------------------------------------------------

But if we query the V$SQL view, we find that for this particular SQL_ID, the column IS_BIND_SENSITIVE shows ‘Y’ but the column ‘IS_BIND_AWARE’ still shows ‘N’.

This means that Oracle is aware that this query can require different execution plans based on the bind variable values but has not yet acted on this at this stage.

SQL> SELECT sql_id, child_number, is_bind_sensitive, is_bind_aware
 from v$sql
 WHERE  sql_text ='select * from myobjects where object_type=:v_obj_type';  2    3

SQL_ID        CHILD_NUMBER I I
------------- ------------ - -
5utj7m6zxjdsc            0 Y N

We then execute the same SQL statement once again using the same bind variable value, and we find that this time a more optimal execution plan has been chosen which in this case is an INDEX RANGE SCAN

SQL>  VARIABLE v_obj_type VARCHAR2(10)
SQL> EXEC :v_obj_type := 'RULE';

PL/SQL procedure successfully completed.

SQL> select * from myobjects where object_type=:v_obj_type;

no rows selected

SQL> select * from table(dbms_xplan.display_cursor);

PLAN_TABLE_OUTPUT
-----------------------------------------------------------------------------------------------
SQL_ID  5utj7m6zxjdsc, child number 1
-------------------------------------
select * from myobjects where object_type=:v_obj_type

Plan hash value: 2745750972

---------------------------------------------------------------------------------------------
| Id  | Operation                   | Name          | Rows  | Bytes | Cost (%CPU)| Time     |
---------------------------------------------------------------------------------------------
|   0 | SELECT STATEMENT            |               |       |       |    30 (100)|          |
|   1 |  TABLE ACCESS BY INDEX ROWID| MYOBJECTS     |   823 | 83946 |    30   (0)| 00:00:01 |

PLAN_TABLE_OUTPUT
-------------------------------------------------------------------------------------------------
|*  2 |   INDEX RANGE SCAN          | MYOBJECTS_IND |   823 |       |     3   (0)| 00:00:01 |
---------------------------------------------------------------------------------------------

If we now query the V$SQL view, we find that this optimizer change is also reflected in the column IS_BIND_SENSITIVE showing the value ‘Y’ instead of ‘N’ which was the earlier value.

SQL> SELECT sql_id, child_number, is_bind_sensitive, is_bind_aware
 from v$sql
 WHERE  sql_text ='select * from myobjects where object_type=:v_obj_type';  2    3

SQL_ID        CHILD_NUMBER I I
------------- ------------ - -
5utj7m6zxjdsc            0 Y N
5utj7m6zxjdsc            1 Y Y

Script to check integrity of a 9i or 10g database before upgrade to 11g.

Run this script connected as sysdba in the 9i or 10g database.

NAME: DBUPGDIAG.SQL

This script can be downloaded from Oracle Metalink note 556610.1

– PURPOSE:
– This script is intended to provide a user friendly output to diagonise
– the status of the database before (or) after upgrade. The script will
– create a file called db_upg_diag__.log in your local
– working directory. This does not make any DDL / DML modifications.
–
– This script will work in both Windows and Unix platforms from database
– version 9.2 or higher.

col TODAY NEW_VALUE _DATE
col VERSION NEW_VALUE _VERSION
set termout off
select to_char(SYSDATE,’fmMonth DD, YYYY’) TODAY from DUAL;
select version from v$instance;
set termout on
set echo off
set feedback off
set head off
set verify off
Prompt
PROMPT Enter location for Spooled output:
Prompt
DEFINE log_path = &1
column timecol new_value timestamp
column spool_extension new_value suffix
SELECT to_char(sysdate,’dd-Mon-yyyy_hhmi’) timecol,’.log’ spool_extension FROM
sys.dual;
column output new_value dbname
SELECT value || ‘_’ output FROM v$parameter WHERE name = ‘db_name’;
spool &log_path/db_upg_diag_&&dbname&&timestamp&&suffix
set linesize 150
set pages 100
set trim on
set trims on
col Compatible for a35
col comp_id for a12
col comp_name for a40
col org_version for a11
col prv_version for a11
col owner for a12
col object_name for a40
col object_type for a40
col Wordsize for a25
col Metadata for a8
col ‘Initial DB Creation Info’ for a35
col ‘Total Invalid JAVA objects’ for a45
col ‘Role’ for a30
col ‘User Existence’ for a27
col “JAVAVM TESTING” for a15
Prompt
Prompt
set feedback off head off
select LPAD(‘*** Start of LogFile ***’,50) from dual;
select LPAD(‘Oracle Database Upgrade Diagnostic Utility’,44)||
LPAD(TO_CHAR(SYSDATE, ‘MM-DD-YYYY HH24:MI:SS’),26) from dual;
Prompt
Prompt ===============
Prompt Database Uptime
Prompt ===============
SELECT to_char(startup_time, ‘HH24:MI DD-MON-YY’) “Startup Time”
FROM v$instance;
Prompt
Prompt =================
Prompt Database Wordsize
Prompt =================
SELECT distinct(‘This is a ‘ || (length(addr)*4) || ‘-bit database’) “WordSize”
FROM v$process;
Prompt
Prompt ================
Prompt Software Verison
Prompt ================
SELECT * FROM v$version;
Prompt
Prompt =============
Prompt Compatibility
Prompt =============
SELECT ‘Compatibility is set as ‘||value Compatible
FROM v$parameter WHERE name =’compatible’;
Prompt
Prompt ================
Prompt Component Status
Prompt ================
Prompt
SET SERVEROUTPUT ON;
DECLARE

ORG_VERSION varchar2(12);
PRV_VERSION varchar2(12);
P_VERSION VARCHAR2(10);

BEGIN

SELECT version INTO p_version
FROM registry$ WHERE cid=’CATPROC’ ;

IF SUBSTR(p_version,1,5) = ‘9.2.0′ THEN

DBMS_OUTPUT.PUT_LINE(RPAD(‘Comp ID’, ||RPAD(‘Component’,35)||
RPAD(‘Status’,10) ||RPAD(‘Version’, 15));

DBMS_OUTPUT.PUT_LINE(RPAD(‘ ‘,8,’-') ||RPAD(‘ ‘,35,’-')||
RPAD(‘ ‘,10,’-') ||RPAD(‘ ‘,15,’-'));

FOR x in (SELECT SUBSTR(dr.comp_id,1,8) comp_id,
SUBSTR(dr.comp_name,1,35) comp_name,
dr.status Status,SUBSTR(dr.version,1,15) version
FROM dba_registry dr,registry$ r
WHERE dr.comp_id=r.cid and dr.comp_name=r.cname
ORDER BY 1)

LOOP

DBMS_OUTPUT.PUT_LINE(RPAD(SUBSTR(x.comp_id,1,8),8) ||
RPAD(SUBSTR(x.comp_name,1,35),35)||
RPAD(x.status,10) || RPAD(x.version, 15));
END LOOP;

ELSIF SUBSTR(p_version,1,5) != ‘9.2.0′ THEN

DBMS_OUTPUT.PUT_LINE(RPAD(‘Comp ID’, ||RPAD(‘Component’,35)||
RPAD(‘Status’,10) ||RPAD(‘Version’, 15)||
RPAD(‘Org_Version’,15)||RPAD(‘Prv_Version’,15));

DBMS_OUTPUT.PUT_LINE(RPAD(‘ ‘,8,’-') ||RPAD(‘ ‘,35,’-')||
RPAD(‘ ‘,10,’-')||RPAD(‘ ‘,15,’-')||RPAD(‘ ‘,15,’-')||
RPAD(‘ ‘,15,’-'));

FOR y in (SELECT SUBSTR(dr.comp_id,1,8) comp_id,
SUBSTR(dr.comp_name,1,35) comp_name, dr.status Status,
SUBSTR(dr.version,1,11) version,org_version,prv_version
FROM dba_registry dr,registry$ r
WHERE dr.comp_id=r.cid and dr.comp_name=r.cname
ORDER BY 1)

LOOP

DBMS_OUTPUT.PUT_LINE(RPAD(substr(y.comp_id,1,8), ||
RPAD(substr(y.comp_name,1,35),35)||RPAD(y.status,10) ||
RPAD(y.version, 15)||RPAD(y.org_version,15)||RPAD(y.prv_version,15));

END LOOP;

END IF;
END;
/
SET SERVEROUTPUT OFF
Prompt
Prompt
Prompt ======================================================
Prompt List of Invalid Database Objects Owned by SYS / SYSTEM
Prompt ======================================================
Prompt
set head on
SELECT case count(object_name)
WHEN 0 THEN ‘There are no Invalid Objects’
ELSE ‘There are ‘||count(object_name)||’ Invalid objects’
END “Number of Invalid Objects”
FROM dba_objects
WHERE status=’INVALID’
AND owner in (‘SYS’,'SYSTEM’);
Prompt
DOC
################################################################

If there are no Invalid objects below will result in zero rows.

################################################################
#
Prompt
set feedback on
SELECT owner,object_name,object_type
FROM dba_objects
WHERE status=’INVALID’
AND owner in (‘SYS’,'SYSTEM’)
ORDER BY owner,object_type;
set feedback off
Prompt
Prompt ================================
Prompt List of Invalid Database Objects
Prompt ================================
Prompt
set head on
SELECT case count(object_name)
WHEN 0 THEN ‘There are no Invalid Objects’
ELSE ‘There are ‘||count(object_name)||’ Invalid objects’
END “Number of Invalid Objects”
FROM dba_objects
WHERE status=’INVALID’
AND owner not in (‘SYS’,'SYSTEM’);
Prompt
DOC
################################################################

If there are no Invalid objects below will result in zero rows.

################################################################
#
Prompt
set feedback on
SELECT owner,object_name,object_type
FROM dba_objects
WHERE status=’INVALID’
AND owner not in (‘SYS’,'SYSTEM’)
ORDER BY owner,object_type;
set feedback off
Prompt
Prompt ==============================================================
Prompt Identifying whether a database was created as 32-bit or 64-bit
Prompt ==============================================================
Prompt
DOC
###########################################################################

Result referencing the string ‘B023′ ==> Database was created as 32-bit
Result referencing the string ‘B047′ ==> Database was created as 64-bit
When String results in ‘B023′ and when upgrading database to 10.2.0.3.0
(64-bit) , For known issue refer below articles

Note 412271.1 ORA-600 [22635] and ORA-600 [KOKEIIX1] Reported While
Upgrading Or Patching Databases To 10.2.0.3
Note 579523.1 ORA-600 [22635], ORA-600 [KOKEIIX1], ORA-7445 [KOPESIZ] and
OCI-21500 [KOXSIHREAD1] Reported While Upgrading To 11.1.0.6

###########################################################################
#
Prompt
SELECT SUBSTR(metadata,109,4) “Metadata”,
CASE SUBSTR(metadata,109,4)
WHEN ‘B023′ THEN ‘Database was created as 32-bit’
WHEN ‘B047′ THEN ‘Database was created as 64-bit’
ELSE ‘Metadata not Matching’
END “Initial DB Creation Info”
FROM sys.kopm$;
Prompt
Prompt ===================================================
Prompt Number of Duplicate Objects Owned by SYS and SYSTEM
Prompt ===================================================
Prompt
Prompt Counting duplicate objects ….
Prompt
SELECT count(1)
FROM dba_objects
WHERE object_name||object_type in
(SELECT object_name||object_type
from dba_objects
where owner = ‘SYS’)
and owner = ‘SYSTEM’;
Prompt
Prompt =========================================
Prompt Duplicate Objects Owned by SYS and SYSTEM
Prompt =========================================
Prompt
Prompt Querying duplicate objects ….
Prompt
SELECT object_name, object_type
FROM dba_objects
WHERE object_name||object_type in
(SELECT object_name||object_type
FROM dba_objects
WHERE owner = ‘SYS’)
AND owner = ‘SYSTEM’;
Prompt
DOC

################################################################################

If any objects found please follow below article.
Note 1030426.6 How to Clean Up Duplicate Objects Owned by SYS and SYSTEM schema
Read the Exceptions carefully before taking actions.

################################################################################
#
Prompt
Prompt ================
Prompt JVM Verification
Prompt ================
Prompt
SET SERVEROUTPUT ON
DECLARE

V_CT NUMBER;
P_VERSION VARCHAR2(10);

BEGIN

– If so, get the version of the JAVAM component
EXECUTE IMMEDIATE ‘SELECT version FROM registry$ WHERE cid=”JAVAVM”
AND status 99′ INTO p_version;

SELECT count(*) INTO v_ct FROM dba_objects
WHERE object_type LIKE ‘%JAVA%’ AND owner=’SYS’;

IF SUBSTR(p_version,1,5) = ‘8.1.7′ THEN
IF v_ct>=6787 THEN
DBMS_OUTPUT.PUT_LINE(‘JAVAVM – Installed properly’);
ELSE
DBMS_OUTPUT.PUT_LINE(‘JAVAVM – Not Installed properly’);
END IF;
ELSIF SUBSTR(p_version,1,5) = ‘9.0.1′ THEN
IF v_ct>=8585 THEN
DBMS_OUTPUT.PUT_LINE(‘JAVAVM – Installed properly’);
ELSE
DBMS_OUTPUT.PUT_LINE(‘JAVAVM – Not Installed properly’);
END IF;
ELSIF SUBSTR(p_version,1,5) = ‘9.2.0′ THEN
IF v_ct>=8585 THEN
DBMS_OUTPUT.PUT_LINE(‘JAVAVM – Installed properly’);
ELSE
DBMS_OUTPUT.PUT_LINE(‘JAVAVM – Not Installed properly’);
END IF;
ELSIF SUBSTR(p_version,1,6) = ‘10.1.0′ THEN
IF v_ct>=13866 THEN
DBMS_OUTPUT.PUT_LINE(‘JAVAVM – Installed properly’);
ELSE
DBMS_OUTPUT.PUT_LINE(‘JAVAVM – Not Installed properly’);
END IF;
ELSIF SUBSTR(p_version,1,6) = ‘10.2.0′ THEN
IF v_ct>=14113 THEN
DBMS_OUTPUT.PUT_LINE(‘JAVAVM – Installed properly’);
ELSE
DBMS_OUTPUT.PUT_LINE(‘JAVAVM – Not Installed properly’);
END IF;
END IF;

EXCEPTION WHEN NO_DATA_FOUND THEN
DBMS_OUTPUT.PUT_LINE(‘JAVAVM – NOT Installed. Below results can be ignored’);

END;
/
SET SERVEROUTPUT OFF
Prompt
Prompt ================================================
Prompt Checking Existence of Java-Based Users and Roles
Prompt ================================================
Prompt
DOC

################################################################################

There should not be any Java Based users for database version 9.0.1 and above.
If any users found, it is faulty JVM.

################################################################################
#

Prompt
SELECT CASE count(username)
WHEN 0 THEN ‘No Java Based Users’
ELSE ‘There are ‘||count(*)||’ JAVA based users’
END “User Existence”
FROM dba_users WHERE username LIKE ‘%AURORA%’ AND username LIKE ‘%OSE%’;
Prompt
DOC

###############################################################

Healthy JVM Should contain Six Roles.
If there are more or less than six role, JVM is inconsistent.

###############################################################
#

Prompt
SELECT CASE count(role)
WHEN 0 THEN ‘No JAVA related Roles’
ELSE ‘There are ‘||count(role)||’ JAVA related roles’
END “Role”
FROM dba_roles
WHERE role LIKE ‘%JAVA%’;
Prompt
Prompt Roles
Prompt
SELECT role FROM dba_roles WHERE role LIKE ‘%JAVA%’;
set head off
Prompt
Prompt =========================================
Prompt List of Invalid Java Objects owned by SYS
Prompt =========================================
SELECT CASE count(*)
WHEN 0 THEN ‘There are no SYS owned invalid JAVA objects’
ELSE ‘There are ‘||count(*)||’ SYS owned invalid JAVA objects’
END “Total Invalid JAVA objects”
FROM dba_objects
WHERE object_type LIKE ‘%JAVA%’
AND status=’INVALID’
AND owner=’SYS’;
Prompt
DOC

#################################################################

Check the status of the main JVM interface packages DBMS_JAVA
and INITJVMAUX and make sure it is VALID.
If there are no Invalid objects below will result in zero rows.

#################################################################
#
Prompt
set feedback on
SELECT owner,object_name,object_type
FROM dba_objects
WHERE object_type LIKE ‘%JAVA%’
AND status=’INVALID’
AND owner=’SYS’;
set feedback off
Prompt
Prompt INFO: Below query should succeed with ‘foo’ as result.
set heading on
select dbms_java.longname(‘foo’) “JAVAVM TESTING” from dual;
set heading off
Prompt

set feedback off head off
select LPAD(‘*** End of LogFile ***’,50) from dual;
set feedback on head on
Prompt
spool off
Prompt
set heading off
set heading off
set feedback off
select ‘Upload db_upg_diag_&&dbname&&timestamp&&suffix from “&log_path” directory’
from dual;
set heading on
set feedback on
Prompt
– – - – - – - – - – - – - – - – Script ends here – - – - – - – - – - – - – -

While creating the ASM instance using DBCA in an 11g RAC environment on 64 bit AIX 5L, we got the ORA-27504 error along with others as shown below.

The problem was caused by incorrect UDP and TCP packet settings and is documented in Metalink note 300956.1

As root, the following changes need to be made

# no -o tcp_sendspace=262144
# no -o tcp_recvspace=262144
# no -o udp_sendspace=65536
# no -o udp_recvspace=262144
# no -o rfc1323=1

Adding these entries to the /etc/rc.net will ensure that these parameters take effect on every machine reboot.

This scenario will show how we can use a combination of FLASHBACK database and also recovery to take a database back in time to undo a TRUNCATE operation and then roll forward the database after the flashback operation to bring it to the current point in time.

INSERT ROWS INTO TABLE MYOBJ – THIS WILL BE TRUNCATED

SQL> insert into scott.myobj select * from all_objects;

50496 rows created.

SQL> /

50496 rows created.

SQL> select count(*) from scott.myobj;

  COUNT(*)
----------
    100992

OBTAIN THE CURRENT SCN – FLASHBACK WILL HAPPEN TO THIS SCN

SQL> select current_scn from v$database;

          CURRENT_SCN
---------------------
          15633908021

TRUNCATE THE TABLE

SQL> truncate table scott.myobj;

Table truncated.

SQL> select count(*) from scott.myobj;

  COUNT(*)
----------
         0

AT THE SAME TIME OTHER CHANGES ARE HAPPENING IN THE DATABASE AND THESE CHANGES WILL BE RECOVERED AFTER THE FLASHBACK IS DONE

SQL> insert into scott.myobj2 select * from scott.myobj2;

356874 rows created.

SQL> /

713748 rows created.

SQL> commit;

Commit complete.

SHUTDOWN THE DATABASE AND PERFORM THE FLASHBACK TO THE SCN BEFORE THE TRUNCATE WAS DONE

SQL> shutdown immediate;
Database closed.
Database dismounted.
ORACLE instance shut down.
SQL> startup mount;
ORACLE instance started.

Total System Global Area  469762048 bytes
Fixed Size                  2084880 bytes
Variable Size             377491440 bytes
Database Buffers           83886080 bytes
Redo Buffers                6299648 bytes
Database mounted.

SQL> FLASHBACK DATABASE TO SCN 15633908021;

Flashback complete.

OPEN THE DATABASE IN READ ONLY MODE AND EXPORT THE TABLE THAT WAS TRUNCATED EARLIER. THIS TABLE WILL BE IMPORTED AFTER THE RECOVERY IS DONE

SQL> alter database open read only;

Database altered.

SQL>  select count(*) from scott.myobj;

  COUNT(*)
----------
         100992

SQL> quit
Disconnected from Oracle Database 10g Enterprise Edition Release 10.2.0.4.0 - 64bit Production
With the Partitioning, OLAP, Data Mining and Real Application Testing options
crashdb:/u03/oradata/crashdb/arch> exp file=scott.dmp tables=myobj

Export: Release 10.2.0.4.0 - Production on Fri Feb 6 09:53:00 2009

Copyright (c) 1982, 2007, Oracle.  All rights reserved.

Username: scott
Password:

Connected to: Oracle Database 10g Enterprise Edition Release 10.2.0.4.0 - 64bit Production
With the Partitioning, OLAP, Data Mining and Real Application Testing options
Export done in US7ASCII character set and AL16UTF16 NCHAR character set
server uses WE8ISO8859P1 character set (possible charset conversion)

About to export specified tables via Conventional Path ...
. . exporting table                          MYOBJ          100992 rows exported
Export terminated successfully without warnings.

NOW SHUTDOWN THE DATABASE,STARTUP MOUNT AND PERFORM THE RECOVERY

SQL> shutdown immediate
Database closed.
Database dismounted.
ORACLE instance shut down.
SQL> startup mount;
ORACLE instance started.

Total System Global Area  696254464 bytes
Fixed Size                  2086616 bytes
Variable Size             184551720 bytes
Database Buffers          503316480 bytes
Redo Buffers                6299648 bytes
Database mounted.
SQL> recover database;
Media recovery complete.
SQL> alter database open;

Database altered.

AS EXPECTED TABLE MYOBJ WHICH WAS TRUNCATED NOW AGAIN HAS 0 ROWS AFTER THE RECOVERY – WE CAN IMPORT THE DUMP WE TOOK AFTER THE FLASHBACK

SQL> select count(*) from scott.myobj;

  COUNT(*)
----------
         0

CONFIRM THAT OTHER COMMITTED CHANGES IN THE DATABASE HAVE BEEN RECOVERED

SQL> select count(*) from scott.myobj2;

  COUNT(*)
----------
  713748

While relinking the Oracle 11g RAC software on 64 bit AIX platform, we got the error as shown below:

Looking at the installation log we found the following lines:

ld: 0706-006 Cannot find or open library file: -l ha_gs_r
ld:open(): A file or directory in the path name does not exist.
ld: 0706-006 Cannot find or open library file: -l ha_em_r
ld:open(): A file or directory in the path name does not exist.
make: 1254-004 The error code from the last command is 255

The cause of this error is that a particluar fileset rsct.basic.rte which is required specifically for a 11g RAC installation has not been installed at the OS level. After the fileset was installed we could relink the software without the same ld errors as mentioned above.

We can check if the fileset has been installed by running the command ‘lslpp’ as shown below.

testdb:/u02/oradata/testdb> lslpp -l rsct.basic.rte
lslpp: Fileset rsct.basic.rte not installed.

After the fileset is installed we can confirm the same…

middba1:/u01/oracle/dump> lslpp -l rsct.basic.rte
 Fileset                      Level  State      Description
  ----------------------------------------------------------------------------
Path: /usr/lib/objrepos
  rsct.basic.rte            2.4.10.0  COMMITTED  RSCT Basic Function

This package is part of the OS Patch IY98287 which is one of the mandatory required patches to be installed on AIX if we are configuring a 11g RAC environment.

This is also documented in Metalink Note 471701.1

This example illustrates Transparent Application Failover (TAF) in a two node AIX 11g RAC configuration.

The database name is racdb and the two instances are racdb1 and racdb2.

Node 1 – middba1 which hosts instance racdb1.
Node2 – middba2 which hosts instance racdb2

We have defined two services racdb1 and racdb2. For service racdb1 the preferred instance is racdb1 and
for service racdb2 the preferred instance is racdb2.

The following srvctl commands illustrates the same.

middba1:/u01/oracle/dump> srvctl status service -d racdb -s racdb1
Service racdb1 is running on instance(s) racdb1

middba1:/u01/oracle/dump> srvctl config service -d racdb -s racdb1
racdb1 PREF: racdb1 AVAIL: racdb2

middba1:/u01/oracle/dump> srvctl status service -d racdb -s racdb2
Service racdb2 is running on instance(s) racdb2

middba1:/u01/oracle/dump> srvctl config service -d racdb -s racdb2
racdb2 PREF: racdb2 AVAIL: racdb1

From a SQL*PLUS client, we establish a session as user SYSTEM using the service racdb1. Note the machine name where this service is running from.

testdb:/u01/oracle> sqlplus system/xxx@racdb1

SQL*Plus: Release 11.1.0.6.0 - Production on Fri Aug 14 10:37:03 2009

Copyright (c) 1982, 2007, Oracle.  All rights reserved.

Connected to:
Oracle Database 11g Enterprise Edition Release 11.1.0.6.0 - 64bit Production
With the Partitioning, Real Application Clusters and Real Application Testing options

SQL> select host_name from v$instance;

HOST_NAME
----------------------------------------------------------------
middba1

We then execute a long running SELECT statement

SQL> select * from sh.sales;

While this statement is running we reboot the node middba1 where this service is currently connected to.

We will note that the screen will momentarily freeze while the service gets relocated to the other functional node racdb2 and then the query continues from where it got interrupted. The session will not get disconnected – so for the end user it is transparent from which instance the query is being serviced.

If we check the status of the cluster resources, we will find that the VIP from node middba1 has now relocated to middba2. Other resources that were originally running from node middba1 are now offline.

middba2:/u01/oracle> crs_stat -t
Name           Type           Target    State     Host
------------------------------------------------------------
ora....SM1.asm application    ONLINE    OFFLINE
ora....A1.lsnr application    ONLINE    OFFLINE
ora....ba1.gsd application    ONLINE    OFFLINE
ora....ba1.ons application    ONLINE    OFFLINE
ora....ba1.vip application    ONLINE    ONLINE    middba2
ora....SM2.asm application    ONLINE    ONLINE    middba2
ora....A2.lsnr application    ONLINE    ONLINE    middba2
ora....ba2.gsd application    ONLINE    ONLINE    middba2
ora....ba2.ons application    ONLINE    ONLINE    middba2
ora....ba2.vip application    ONLINE    ONLINE    middba2
ora.racdb.db   application    ONLINE    ONLINE    middba2
ora....cdb1.cs application    ONLINE    ONLINE    middba2
ora....b1.inst application    ONLINE    OFFLINE
ora....db1.srv application    ONLINE    ONLINE    middba2
ora....cdb2.cs application    ONLINE    ONLINE    middba2
ora....b2.inst application    ONLINE    ONLINE    middba2
ora....db2.srv application    ONLINE    ONLINE    middba2

If we check the status of the service, we will find that the service racdb1 is now running on node middba2 connecting to the instance racdb2.

middba2:/u01/oracle> srvctl status service -d racdb -s racdb1
Service racdb1 is running on instance(s) racdb2

Once the node middba1 which was rebooted finally does come online, we will find that the VIP will relocate
to its original node middba1 once the cluster services are brought online after the reboot.

middba2:/u01/oracle> crs_stat -t
Name           Type           Target    State     Host
------------------------------------------------------------
ora....SM1.asm application    ONLINE    ONLINE    middba1
ora....A1.lsnr application    ONLINE    ONLINE    middba1
ora....ba1.gsd application    ONLINE    ONLINE    middba1
ora....ba1.ons application    ONLINE    ONLINE    middba1
ora....ba1.vip application    ONLINE    ONLINE    middba1
ora....SM2.asm application    ONLINE    ONLINE    middba2
ora....A2.lsnr application    ONLINE    ONLINE    middba2
ora....ba2.gsd application    ONLINE    ONLINE    middba2
ora....ba2.ons application    ONLINE    ONLINE    middba2
ora....ba2.vip application    ONLINE    ONLINE    middba2
ora.racdb.db   application    ONLINE    ONLINE    middba2
ora....cdb1.cs application    ONLINE    ONLINE    middba2
ora....b1.inst application    ONLINE    ONLINE    middba1
ora....db1.srv application    ONLINE    ONLINE    middba2
ora....cdb2.cs application    ONLINE    ONLINE    middba2
ora....b2.inst application    ONLINE    ONLINE    middba2
ora....db2.srv application    ONLINE    ONLINE    middba2

Note however, that the service racdb1 will continue running on node middba2 (instance racdb2)even though the original node which it was running on middba1 has been brought online.

We will have to relocate the service back to its original node using the srvctl command

middba1:/u01/oracle> srvctl relocate service -d racdb -s racdb1 -i racdb2 -t racdb1