Friday, May 20, 2011

Oracle Database Backup Information

Oracle Database Backup Information

Contents

Backup Modes:
Offline or Cold Backup
Online or Hot Backup

Which files to backup
Archive Log Mode
NoArchivelog
Implement a Backup strategy
Guidelines for scheduling Routine backups
Implement Non-routine Backups
Performance
Procedures to enable Archiving Log Mode

Backup Modes

a) Offline or Cold Backup

Copying of the datafiles, control file and online redo log files must be done by using an operating system copy utility. This is a considered as a complete backup of the database. Any changes made after this backup will be unrecoverable if the database is running in NOARCHIVELOG mode. All transactions are recorded in online redo log files whether archiving or not. When redo logs are archived (ARCHIVELOG mode), ORACLE allows you to apply these transactions after restoring files that were damaged (assuming an active Redo log file was not among the files damaged). Whenever the schema of the database is changed i.e., a new datafile is added or a file is renamed or a tablespace is created or dropped, shutdown the database and at least make a copy of the control file and the newly added datafile. A complete backup of the database is preferred.

Procedure
1. Backup the control file to trace file.
2. Shut down application (where applicable).
3. Shut Oracle down cleanly. If a SHUTDOWN ABORT has been issued, the instance must be restarted and closed again.
4. Copy all relevant database files to a staging area. These comprise:

Data files
All Redo Log files, including all mirror group members
Database control file
All associated parameter files ( database initialization parameter file, init.ora, )
The database password file
5. Restart database.
6. Copy files created to offline storage media (i.e. tape). Ideally these copies will be left on disk to minimise the time taken to recover the database.


b) Online or Hot Backup

At sites where database must operate 24 hours per day and when it is not feasible to take offline backups, then an alternative is provided by ORACLE to perform physical backups while the database remains available for both reading and updating. For this kind of backup the database must be in ARCHIVELOG mode. Only data files and current control file need to be backed up. Unlike offline backups, the unit of a online backup is tablespace, and any or all tablespaces can backed up whenever needed. Different datafiles can be backed up at different times. This process should be fully automated if possible. You MUST NEVER BACKUP THE ONLINE REDO LOG FILES ON HOT BACKUPS!!!

Procedure:

Run script to backup all relevant database files, comprising :-
Data files are grouped into Tablespaces, and before any file is backed up, its tablespace must be marked by running the command:
ALTER TABLESPACE xxxx BEGIN BACKUP;
Copy the datafiles that belongs to that tablespace.
End of backup is marked by the command:
ALTER TABLESPACE xxxx END BACKUP;
Database control file (physical backup, also copied to trace file);
All associated parameter files ( database initialization parameter file, init.ora);
The database password file (if used);
Any required application data files.
Perform a 'alter system archive log current;'
Archive Redo Log Files (but NOT the Online Redo Log Files, you should query the v$archived_log view )
Copy files created before to offline storage media (i.e tape).
Not all data files need be backed up in this operation, for example read-only tablespaces only need to be backed-up once after the tablespace is made read-only. All of the above processing can be performed while the database is open. Ideally it should occur during a period of low transaction activity.

What happens between BEGIN BACKUP and END BACKUP?

Once the ALTER TABLESPACE ts_name BEGIN BACKUP is issued, two things happen:

(1) Extra information is recorded in the redo logs. Because of this, is it important that on-line backups are done as quickly as possible, and also, if possible, during a quieter period when there is less update activity on the database.

(2) The status in the datafile header is changed to indicate that the datafile is being backed up. Oracle stops recording the occurrence of checkpoints in the header of the database files. This means that when a database file is restored, it will have knowledge of the most recent checkpoint that occurred BEFORE the backup, not any that occurred during the backup. This way, the system will ask for the appropriate set of redo log files to apply should recovery be needed. Since vital information needed for recovery is recorded in the Redo logs, these REDO LOGS are considered as part of the backup. Hence, while backing up the database in this way the database must be in ARCHIVELOG mode. Status in the datafile header is not reset until END BACKUP is issued.

On END BACKUP, the system ceases writing this extra information to the redo-logs, and recommences noting the occurrence of the checkpoints in each file of the database. The checkpoint in the datafile header is changed during the next log switch after END BACKUP is issued. The above information will allow the tablespace to be recovered as if the database had been offline when the backup took place.

Which files to backup

All the files belonging the database are important. Also backup the control file and datafile immediately after adding it to a tablespace or after creating tablespace if archiving is enabled. If media failure damages a datafile that has not been backed up, recovering its tablespace is not possible. After backing up the newly added datafile, include it in the regular datafile backup rotation.

Identifying the files to backup: Use the 'v$' (dynamic performance tables) to find the file names:

select name from v$datafile
UNION
select name from v$controlfile
UNION
select member from v$logfile;
Archive Log Mode

Archiving is the process of writing data from a filled online redo log file to an archive log file.
An Oracle database can operate in either NOARCHIVELOG or ARCHIVELOG mode. Oracle writes to the current online redo log file until it fills up, it then begins writing to the next one. When the last online redo log file is full, Oracle cycles back to the first one. In ARCHIVELOG mode each log file is saved before being overwritten. The archive process will backup each redo log file as soon as it is filled. The ARCHIVELOG function enables a whole range of possible backup options:

provides protection against media and instance failure
enables 24-hour availability and is mandatory for hot backups
allows point-in-time recovery
recovery possible without losing any data
allow the database to stay up, preserving the data in the SGA
Archiving can be done manually or automatically.
Points to note:

Archived redo log files can be used to help recover an Oracle database using any of the backup modes listed above, assuming that all redo log files that have been used since the backup have been archived
After each redo log is filled the redo log is archived to a backup device
A failed transaction is still a transaction; redo log files keep track of rollback segment extents, so rolled back inserts or deletes affect it just like completed transactions
Repetitive failed load attempts can create massive numbers of redo log files
Until an online redo file is archived it cannot be reused
When it gets stuck, i.e. none of the redo log files are ready for reuse the database comes to a halt
Administration is more complex as the DBA has to keep track of multiple archive log files
Recommendations:

Use ARCHIVELOG MODE. It is mandatory for full database recovery without loss of data. It is required to support Hot Backups and to recover transactions lost since the last backup.
Size your log files carefully.
Too small and the database will be archiving too frequently. This means a performance overhead due to frequent check-pointing. It will result in the creation of a large number of small archive log files making file management and recovery more complex. Too large and the archive process will take too long and the archive log files may end up spanning multiple tape volumes.
Archive to disk and then back up to tape overnight. Archiving directly to tape is complicated and prone to problems.
When recovering. Try to ensure that all of the archive log files that will be needed are on disk. This will greatly speed up the process.
NOARCHIVELOG:

Provides instance recovery only
Online redo log files are overwritten when needed Only the most recent changes to the database (or those that are in the current online redo log files) are available at any give time
Recovery will involve the loss of transactions processed since the last backup or export was done
Hot Backups not supported. The database will be unavailable during backups.
Implement a Backup strategy

Its important to identify exactly what recovery facilities are required by the site This will determine the frequency and type of backup that will have to be taken. The answers to several questions must be established before attempting to implement any backup procedures.

How much data can you afford to lose?
How often and for how long, can the database be off-line to perform backups?
Should recovery be needed, how quickly do you need to recover your data?
Do you need the capability to reverse changes made to the database?
Implementing a backup strategy requires consideration of the following.

How to minimize vulnerabilities during backup and recovery?
How can physical and logical errors be monitored and checked for?
How much data can you afford to lose?
If you can afford to lose up to a day's worth of data, then running in NOARCHIVELOG mode and taking nightly off-line backups is sufficient. Any changes applied to the database after the most recent backup will be unrecoverable.
If you cannot afford to lose transactions performed since the last backup then you must run in ARCHIVELOG mode. The archived redo log files contain a record of all transactions applied since the last backup. These transactions can be reapplied after restoring from backup any datafiles that were damaged.

How often and for how long, can the database be off-line to perform backups?
Many sites operate 24 hours per day. Even at sites where the database can be shutdown for maintenance, a complete backup may take longer than the window available. When it is not feasible to take off-line backups often enough to provide the protection from data loss the site requires, then online backups are the only option.

Should recovery be needed, how quickly do you need to recover your data?
The cost of downtime (or database outage) may be a key factor in configuring the backup strategy. Recovery time using:

Logical backups (export) take the longest

Recovery using an Export file requires initializing the database and rebuilding the system tablespace prior to importing the data.
If the backup contains incremental export files, multiple export files may have to be read in order to reconstruct the database. The more recent your full backup, the faster the recovery will be.
Physical backups provide the fastest recovery times.

Physical backups depends on the type of backup storage device. Maintaining as much of the backup as possible online will help minimise downtime.
Cold backups take as long as it takes to restore all the database files
Hot backups depends on the age of the backed up datafile(s) that have to be restored and hence the number of archived redo log files that have to be applied. The number of redo log files will also be determined by their size.(NB. 100Mb redo log files can be applied in 1/2 hour on an HP98XX) If all archived redo log files can be located in the directory specified by the ARCHIVE_LOG_DEST parameter in the init.ora file, then automatic recovery will not require to prompt for redo log file names.
Do you need the capability to reverse changes made to the database?
Do you wish to be able to protect against inadvertent changes made to both the content and structure of the database. For example, bringing back a dropped table, or removing a datafile added to the wrong tablespace.

How to minimize vulnerabilities during backup and recovery?

backup and recovery procedures must be as simple as possible to reduce the possibility of error.
Any scripts used to implement a backup strategy must test for any read or write failures, must rollback on error and report errors to the console, via a log file and through mail.
The scripts must be able to be interrupted and restarted at any time without causing any 'holes' in the backup.
A desirable feature is the capability to write a header and label onto each tape used for backups as well as producing a file listing of the tape's contents and a printed tape label. The tape label should be attempted to be read prior to any attempt to write to the tape.
If the database structure changes it is less desirable to have a hard coded list of datafiles used by the backup. Ideally the backup script should query the database to find out which datafiles are currently in use. This option may be compromised if only certain datafiles ot tablespaces are backed up on given days.
Since loss of an archived redo log file disables recovery from that point on, maintaining multiple copies of archived redo log files will allow recovery from multiple media failures.
The archived redo log files for a database ought not reside on the same physical disk device as any database file or online redo log file.
If database files are being backed up to disk, a database file residing on the same physical device as its backup copy is not adequately backed up.
How can physical and logical errors be monitored and checked for?

Routinely check the database can be restarted
There are certain parameters that are only checked on startup; errors with rollback segments in particular may only show up during startup operations.
Perform a full export periodically A full system export picks up information that user exports do not; this includes third-party indexes and grants. Export checks that database files are logically readable. [NB: This does not imply that they are logically importable; corrupt records may be exported into the dump file, preventing import.]
The export dump file can be used to retrieve particular tables/users if needed. Its worth running scripts to map the tablespaces to owners, and owners to tablespaces immediately following the export. In the event of a tablespace loss, you would then be able to quickly determine what users/systems will be affected.
Guidelines for scheduling Routine backups

Backup procedures should be automated and scheduled. Do not rely on ad-hoc backups.
Have a valid backup for all tablespaces. Do not forget to backup the system tablespace.
Read only tablespaces need to be backed up once
Backup frequency should be determined by the update activity of each tablespace. More frequent backups of heavily used tablespaces will reduce recovery time in the roll forward phase.
Keep archive logs long enough to recover all changes made to the database since the last valid backup.
Keep two copies of archive logs. If only one copy is available then loss will prevent full recovery.
Send backup files off site to prepare for disaster recovery.
If your production schedule allows, take a regular FULL COLD database backup. This is the easiest state to recover to
Test your backup procedures to make sure that they work. Know how long full database recovery and recovery by tablespace takes. Test your disaster recovery strategy.
Make sure all DBAs responsible for recovery understand the process and are involved in the test exercise.
Implement Non-routine Backups

Non routine backups are required in the following circumstances:

Database Upgrades
Database Schema Changes
Whenever the database structure is changed by adding, renaming or dropping a tablespace, datafile, or log file, a control file backup should be performed. The ALTER DATABASE BACKUP CONTROLFILE TO command can be used to take an online backup of a control file.
When a datafile is added to the database, a backup of the new file be taken immediately. The new file should also be added to any automated backup procedures.
The database can maintain multiple copies of the control file, and so a copy of the control file should be placed on several different disk devices. A control file can be added to the database by shutting down the database, copying the control file, altering the INIT.ORA parameter CONTROL_FILES, and restarting.
Performance

Backup and recovery performance can be improved using the following guidelines:

Backing up database files to disk can speed recovery, since the file need not be restored from tape. Also, backing up to disk often allows backup procedures that run in a shorter amount of time.
The procedure of rolling forward a database or database file from a backup can in many cases be simplified and made faster by keeping on disk all archived redo log files needed to roll forward the least recently backed up database file of a database. For many systems, much of the time necessary for recovery is spent loading archived redo log files from tape.
At times maintenance procedures might be performed that would generate large amounts of archived redo logs. Such procedures might benefit from having archiving disabled during their duration.
Using hot backups, only one tablespace at a time should be in backup mode.
Hot backups should be performed during low user activity. When in backup state, a tablespace's activity is still written to the archive logs. However, it's written block-by-block rather than byte-by-byte. So changing one record in a tablespace that's being backed up will result in that record's entire block being written to the archive area.
Implementing a Backup Strategy Using ARCHIVELOG

The number of online redo log files provide a window of time should the archive destination become full.
When you start archiving, archived logs will be written every time the redo operation is about to overwrite a previously written log file. It will write it to the directory indicated by the LOG_ARCHIVE_DEST parameter in your init.ora file. They will all be the same size (in V6; V7 can have variably sized archive logs) as your redo logs. They will increase in number until they run out of space on their destination device. At that point the database will freeze until you clear more space for them in the LOG_ARCHIVE_DEST location. SO, have a second location ready to receive them.
Do each tablespace one at a time. That is, rather than setting them all offline, then backing them up, then setting them back online, do them each separately. You don't want to risk having a system crash while the entire database is in begin backup state; recovery is a mess. Minimize your window of vulnerability by having only one tablespace in backup state at any one time.
Before you backup the control file, force an archive log switch (use 'alter system archive log current;' instead of ‘alter system switch logfile;’ ). This will update the header information in the control file.

Procedures to enable Archiving Log Mode

Enabling automatic archiving (init.ora) method, Connect to sqlplus as SYS and perform the following:
create pfile form spfile;
Edit the init.ora File and add the following lines:
*.Log_archive_start = True
*.Log_archive_format = "ARCH_%t_%s_%r.dbf"
l*.og_archive_dest_1 = "location=/u01/oradata/TICPBT09/ARCH MANDATORY"
*.log_archive_dest_state_1 = enable
*.log_archive_dest_2 = "service=TICPBP02 OPTIONAL reopen=60"
*.log_archive_dest_state_2 = enable
*.log_archive_min_succeed_dest = 1

For 10g
The LOG_ARCHIVE_START init.ora parameter has been rendered obsolete. In Oracle 10g, if the LOG_ARCHIVE_DEST is not set, archiving will be directed to the flashback recovery area automatically when the database is switch to ARCHIVELOG mode.
*.log_archive_format='ARCH_%t_%s_%r.dbf'
*.log_archive_dest_1='LOCATION=/u02/oradata/ARCH'
*.log_archive_dest_state_1 = enable
*.log_archive_dest_2 = "service=TICPBP02 OPTIONAL reopen=60"
*.log_archive_dest_state_2 = enable
*.log_archive_min_succeed_dest = 1

Shutdown the database:
shutdown immediate;

Start up a new instance and mount, but do not open the database.
create spfile from pfile;
startup mount

Switch the database's archiving mode and open it
alter database archivelog;
alter database open;

Verify your database is now in archivelog mode.
archive log list;

Archive all your redo logs at this point.
archive log all;
or
alter system switch logfile;
or
alter system archive log current;

Ensure these newly created Archive log files are added to the backup process

Parallel Query

Parallelisms, symmetry, equivalence—these words and concepts all have something in common. They demonstrate how the total exceeds the sum of its parts. Goalies may be able to stop every shot aimed at them, but without the rest of the team, a goalie is powerless to win without a great deal of luck. In the world of computers, this concept is applicable in terms of parallelism. Parallel processing allows great computer with a whole team of CPUs, to work together to create an even greater computational power. Parallel processing has existed for many years, but with Oracle8 the power of multiple processors and servers takes advantage of a computer topology that can further empower the database to increase application and server productivity.

The question is: What do I need to be able to take advantage of these parallelism features incorporated within Oracle8? First you need a computer, but will any computer do? Possibly, as long as the computer can support multiple CPUs and you have bought, configured and installed these CPUs, you have the opportunity to enter into the parallel query world. In today’s world this means that you would require a machine and operating system that supports two or more processors, and an operating system that can address and share processing amongst these processors. These machines share memory and disks, but spread their processing power amongst the numerous CPUs. The reason that parallel process provides us with so much of a performance boost is that we often run into processing bottlenecks, not I/O bottlenecks. By implementing solutions that use the Parallel Query feature, you will start to experience new problems. You may then find that I/O is a bottleneck, since the CPUs will now be hungry for more information at a rate that the system had not experienced during serial operation execution. In this presentation we will discuss how best to use the parallel query feature, how best to implement it, and what to do to ensure that you are using this feature at an optimal level.

What is the Parallel Query Option.
The Parallel Query Option is a mechanism that allows a large query to be split up (transparently to the end-user) into a number of smaller queries that can all be run simultaneously. The Parallel Query Option (PQO) should not be confused with the Oracle Parallel Server (OPS) which allows multiple instances of Oracle (usually running on a number of separate machines) to address a single set of database files and uses a high-speed ‘distributed lock manager’ to synchronize the ownership of blocks and locks across the separate SGA’s

Init.ora Parameters
To use PQO, you need to have a number of ‘floating’ processes available in the system that can be called by your normal shadow process (dedicated server). These processes, called the ‘parallel query slaves’ will have names like ora_p000_{SID}, and you can have up to 256 of them.

The parameter parallel_min_servers specifies the number of such processes that you want permanently running and waiting for calls. Since such processes take up machine resources you may want to keep this number low for normal running: if you set parallel_max_servers Oracle will then spawn further processes on demand up to this limit. One odd feature of dynamically spawned processes, however, is that they are owned at the O/S level by the user running the session’s shadow process, and not by any special ‘Oracle id’.

Once you have the capability of running parallel queries, there are two other parameters that you should consider. These relate to the way the system performance will degrade as the load increases.

Consider the problem of a query that takes 3 minutes to complete when running with a degree of parallelism of 15. What happens if the database is so busy that no parallel slaves are free when someone starts the query. Is it acceptable for the query to run serially and take 45 minutes, or should it simply die with a warning message?

Set parallel_min_percent to a number between 0 and 100, and if Oracle is unable to get that percentage of the demanded degree of parallelism it returns an error (ORA-12827 insufficient parallel query slaves available). For example, you want a tablescan at degree 6. You have parallel_min_percent set to 50. If the query can run at least at degree 3 with the current number of slaves available it will run, but it there are insufficient slaves available for running at degree 3 the error is raised.

Associated with this parameter is optimizer_percent_parallel which affects the assumptions that the optimizer makes about the degree of parallelism at which a query is expected to run.

Imagine a query that works best at a high degree of parallelism (20 say) by using hash-joins, but gives an acceptable performance at a lower level of parallelism (5 to 15 say) by switching to nested loops. If the optimizer decides that the default degree of the query is 20, then it will always generate the hashing path, even when the system is busy and there are not enough free slaves around to run the query at degree 20. Setting this parameter to a number between 0 and 100, however, tells the optimizer to reduce the assumed degree of parallelism - setting it to 50 in our example would make the optimizer choose the path that was appropriate to degree 10, hence taking the nested loop path.

Invoking PQO
After setting your parameters and restarting your database, what makes it happen?
The first possibility is to give a table an explicit (or implicit) degree of parallelism:

alter table PX parallel (degree 4);
alter table PX parallel (degree default);

Whenever a query involves this table, the optimizer will try to find a path that uses parallel techniques on the table.
Two points that are specific to 7.3.2:

First, the Cost Based Optimizer will always take over in queries where there are parallel tables available even if the current optimizer goal is Rule. Second, the default keyword translates into ‘the smaller of the number of CPUs and the number of devices across which the table appears to be spread’.

The second possibility for introducing parallelism into a query is to put an explicit hint into it:
select /*+ parallel (px, 4)*/
count(*) from PX;

select /*+ parallel (px, default) */
count(*) from PX;

The third option is to define views that contain hints, so that application code does not have to worry about degrees of parallelism,
create or replace view big_px as
select /*+ parallel (px1, 4) parallel (px2, 6) */
from px1, px2
where px1.id = px2.x_id;

An interesting, and possibly irritating, feature of the view definition above is the question of what happens when there are several parallel tables in a query. In general it is not possible to predict the degree of parallelism that such a query will use, as it depends on the usage, the driving table, and the join paths used. In the view above I could only say that, if no other parallel tables are introduced, the final query will run with degree 1, 4 or 6.
When to use it:
Typical queries may need to retrieve and crunch a large number of rows to return a small result set. The Parallel Query Option can be very effective in reducing the time for such queries - on the plus side you have the options of increasing parallel throughput on disks and breaking up the sort requirements to avoid sorts to disk - on the minus side you need to consider the risk of unsuitable data distributions that result in excessive communication and sort costs.

Small Parent Scan to Large Child:
The obvious use of this option is with the recently introduced HASH JOIN facility where two tables are scanned and the smaller is hashed to be used as a target for the larger and most of the benefit comes from the ease of separating the I/O on tablescanning. However another option (particularly of use in conjunction with partition views) is in scanning a small table and using indexed accesses into a large table where the physical data distribution is suitably packed.


Object Creation:
When running in parallel, particularly with the unrecoverable option, you may find that the time to create an object (table or index) is significantly reduced. But there are a couple of side-effects (see further down) to worry about when chasing this benefit.



Parallel Query Processing

The processing of information using multiple processors requires the Oracle8 Parallel Query feature. Parallel Query allows SQL statements to utilize share processing across these processors simultaneously. When processing SQL on a machine that has only a single processor, all SQL will execute within a single process. Parallel Query allows statements to be divided and utilize multiple processes, resulting in quicker completion of the statement.

This discussion is limited to the Parallel Query and it does not include the use and configuration of the Parallel Server Option. The Parallel Server Option relates to the linking of multiple database servers together, whereas the Parallel Query option can be implemented on a single server that contains multiple processors.

Parallel Query provides significant performance gains for databases that contain large amounts of data. The types of systems that gain the most from parallel operations include:

SMP (symmetric multiprocessor), MPP (massively parallel processor machines) and clusters
High I/O capacity machines (many disks and multiple disk controllers)
Underutilized CPUs (less the 25%)
Availability of memory capacity
If you have a machine that has all of these characteristics, then you are a candidate for implementing parallel processing within your Oracle database. The types of databases that provide the greatest benefit from parallelism are:
Very large databases (VLDBs)
Data warehouses
Financial applications
Generally, the parallel query feature is most useful when the queries require a great deal of time to complete, and when a large number of rows are processed. These database applications share a number of common characteristics. The most common characteristic is size. These applications are information intensive. The amount of storage required is very large, to maximize the retrieval of the information the parallel query feature spreads the SQL among the active CPUs. This load sharing is performed by efficiently splitting up a request over the many processors running on the system. As noted previously, by working as a team the goal of completing an operation can be reached sooner and require a smaller amount of effort.

At execution time Oracle, along with the server’s multiple processors, work together to distribute the database operation statement. Interestingly enough, the splitting of work by the parallel query engine is dynamic; if there is any changes to the server’s configuration, it will be adapted to by Oracle at the time of the statement request.

It is important to note that if your system is already heavily loaded that the gains expected by parallel query will be reduced. Ensure that your server has the available cycles for implementing a large information intensive database. Also verify that you have optimized your current CPU usage, disks and disk controllers before embarking on a parallelism initiative.

The SQL statements and database functions that benefit from parallelization with parallel query are the following:

select statements
not in statements
group by statements
create table ... as select ... statements
create index
nested subqueries in insert, update, and delete statements
update statements
delete statements
rebuild index
parallel data loading with SQL*Loader
parallel replication
alter table move partition
alter table split partition
star transformation
The above statements will take advantage of parallel query through the proper use of these statements. Together with the configuration of the database, it will all provide results in a more timely manner.

Oracle will parallelize operations in the following ways:

When DML and DDL parallelize by block ranges.
When operations access partitioned tables and indexes.
Parallel execution by parallel query server processes.
The database parallelizes the SQL statement at execution time. At this time it divides the table or index into ranges of database blocks and then executes on these ranges in parallel. This processing on the ranges are performed by ROWID and each statement will access the information in parallel for a high and low range of ROWIDs. In the case of partitioned data, the information is not accessed by this range, but by the defined partitions. The information is queried by a set of ROWIDs from within each partition and no scan can overlap two partitions.

Partitions are excellent candidates for parallel execution as these data sets can be easily divided into smaller working groups to allow for efficient information interrogation. Basically each partition becomes a candidate for assignment to an individual parallel query server process. In some cases, the number of parallel processes may be less than the number of partitions; this is due to system limits or table attributes. This is not a problem since a parallel query server process can access multiple partitions. If you access only a single partition during the statement execution, the optimizer will understand not to perform the statement in parallel and will perform the statement serially. Inserts are parallelized during execution, as they will be divided among the parallel query server processes.

Look and Feel of Parallel Query
The parallel query feature has a number of different server processes that manage and execute the processing of SQL statements. The first is the query coordinator (P000 process). This process decides how to distribute the SQL statement among one or many parallel query server processes. The other processes can be identified, as they will appear as Pxxx in the process list. The values of 000 will be the numerical identifier of the parallel query server process, starting with 1 and continuing to the maximum number set by the database configuration. Thus, a configuration with five parallel query server processes will show processes P000, P001, P002, P003, and P004. Configuration and installation of the parallel query feature requires no intervention, since it is part of the base product. By setting the appropriate parameters in your initialization parameter file, you can utilize the parallel features of Oracle8.

The Initialization Parameter File Parallel Style
The configuration of parallel query is a balance of optimizing SQL and configuring the database so that it maximizes the effectiveness of its parallel query server processes. In order to optimize the database configuration, we will investigate some initialization parameter file entries, and suggest how best to set these values to optimize parallel SQL execution.

When the database starts, and the appropriate entries are set in the parameter file, Oracle will create a number of parallel query server processes that may be addressed by the query coordinator. These parallel processes become available for use in order to perform parallel operations. These processes, once assigned to an operation will be retained by the operation until its completion. Once completed, the operation will release the parallel query server process to be available to the next operation. In order to maximize these processes we must look at preparing the database. Let’s look at these parameters.

PARALLEL_MIN_SERVERS
This parameter specifies the minimum number of parallel query server processes that will be initiated by the database at the time of instance startup. To optimize the parallel query server processes for normal database operations, the DBA should consider setting the number of PARALLEL_MIN_SERVERS to the formula shown in the next listing.

PARALLEL_MIN_SERVERS = the likely number of simultaneous parallel operations

By reviewing the information contained in the V$PQ_SYSSTAT data dictionary view, you can identify if the value you have set is too low or too high. The data you are interested in sits in the STATISTIC column with values shown in the next listing.

STATISTIC VALUE
------------------------------ ----------
Servers Busy 0
Servers Idle 0

You are looking for these values to indicate if we have over-committed or under-committed our parallel query server processes.


PARALLEL_MAX_SERVERS
This parameter specifies the maximum number of parallel query server processes that will be spawned when required. At times, when the volume of concurrent operations exceeds the number of current parallel query processes currently running, the query coordinator will start other parallel query server processes up to the number specified in this parameter.

To optimize the parallel query server processes for normal database operations, you should consider setting the number of PARALLEL_MAX_SERVERS to the formula shown in the next listing. The formula is expanded to show the value for a two CPU machine with 30 concurrent users.

PARALLEL_MAX_SERVERS = 2 * # of CPUs * # of concurrent users = 2 * 4 * 30 = 240

When all parallel query server processes are in use and the maximum number of processes have been reached. The parallel query coordinator will react to the request for processes in excess of PARALLEL_MAX_SERVERS be either switching to serial processing or return an error if involved in a replication role.


OPTIMIZER_PERCENT_PARALLEL
This parameter determines how aggressively the cost-based optimizer (CBO) will try to parallelize an operation. By default the value is set to 0, so the optimizer will not consider parallelization when determining the best execution plan. You will need to decide how aggressive you want the optimizer to be when it comes to determining the best balance between the execution and parallelization of an operation. The higher that this value is set, up to a maximum of 100, the harder CBO will work to optimize parallel execution. This will determine a plan to minimize execution time based on parallel processing. The optimal setting for this value is shown in the next listing.

OPTIMIZER_PERCENT_PARALLEL = 100/number of concurrent users

When determining if parallel execution is being performed, ensure that the value is set to 100, this will force the operation into a parallel plan unless a serial plan is faster. Remember that the lower the value is set, the optimizer will favor indexes, when the value is set higher the optimizer will favor full table scans.

PARALLEL_AUTOMATIC_TUNING
You can set this parameter to true. Oracle will then determine the default values for parameters that control parallel execution. In addition to setting this parameter, you must specify the PARALLEL clause for the target tables in the system. Oracle then tunes all subsequent parallel operations automatically.


SHARED_POOL_SIZE
The shared pool size must be reviewed. The parallel processors to send messages back and forth to each other use the shared pool. The reason for the increase in the pool is because parallel operations require execution plans that require twice as much space as serial plans. To optimize your shared pool we recommend that you consider the following formula when determining your shared pool size; our example uses a buffer size of 2k on a machine with a current shared pool entry of 20,000,000, PARALLEL_MAX_SERVERS of 16, and 8 CPUs.

SHARED_POOL_SIZE =
current value + ((3 * message buffer size ) * (CPUs + 2) * PARALLEL_MAX_SERVERS)
= 20000000 + ((3 * 2048) * (8 + 2) * 16)
20000000 + 6144 * 10 * 16 = 20000000 + 983040 = 20983040

ALWAYS_ANTI_JOIN
This parameter is important when the SQL operation uses the not in operator. By setting this parameter you can tell your database to use parallel functionality when performing anti-joins. By default not in is evaluated as a (sequential) correlated subquery, when the parameter is set to NESTED_LOOPS. Instead Oracle will perform a hash-join that will execute in parallel. To tell the database to do parallel hashing set the parameter as follows:

ALWAYS_ANTI_JOIN = hash

The next SQL statement is an example of the familiar not in construct that the CBO with Oracle8 will react to with the ALWAYS_ANTI_JOIN entry set to HASH. Note the HASH_AJ hint.

select *
from individual
where lastname like 'KERZNER%'
and office_id is not null
and office_id not in
(select /*+ hash_aj */ office
from national_office
where office_id is not null
and role = 'SENATORS');

ROW_LOCKING
This parameter tells the database whether to acquire row level locks during an update operation. The parameter should be set as follows.

ROW_LOCKING = ALWAYS

By selecting ALWAYS or DEFAULT (they are the same), you tell then database to only to get row locks when the table is being updated. If your database is set to INTENT, then locks are acquired when you perform a select for update. This may appear to fine, but by setting the parameter to INTENT, all insert, updates and deletes will performed serially.

COMPATIBLE
This parameter tells the database to use the latest features available within Oracle8. We mention here to remind you to set the value to get the all the parallel features available. The parameter should be set as shown in the next listing.

COMPATIBLE = 8.0.0

Parallel Execution
The execution of all SQL statements goes through the same process, when the database has been configured with the parallel query.

The statement is parsed.
If the statement is a candidate for parallelization, the optimizer determines the execution plan of the statement.
The query coordinator determines the method the parallelization of the operation.
What determines the amount of parallelization is the existence of at least one full table scan or index scan in the execution plan. Oracle8 has now allowed us to utilize parallelism during operations that perform index range scans of a partitioned index and full index scans. To find out if your SQL is a candidate for parallelism, you may evaluate the SQL statement with explain plan.

CBO many times selects a full table scan for many SQL statements; full table scans are ideal candidates as they can greatly benefit from parallelization. Even when indexes are used, one can improve the performance by telling the coordinator to use parallel index processing. To tell the query coordinator to initiate parallel processing, you use hints as discussed in the next few sections.

PARALLEL Hint
The parallel hint (make sure you spell it right!) allows you to tell the query coordinator how many concurrent parallel query server processes should be used for the parallel operation. The four main operations, select, insert, update, and delete, can all take advantage of this hint. Remember that you must identify the table to be parallelized at a minimum. If you do not identify the degree of parallelism or the server instances, the optimizer will set them to the default in the database. If any values exceed those set in the database, the hint will be ignored. The next listing is an example of how this hint is used.

select /*+ parallel (product,4) */ date_range, product, unit_price
from product;

PARALLEL_INDEX Hint
To access information in a table via an index range scans of a partitioned index, the parallel_index hint should be used. This hint tells the query coordinator to spread the index scan across multiple parallel query server processes. The next listing show the format of this hint (for a table being read), where DOP stands for degree of parallelism.

/*+ parallel_index (tablename, DOP, parallel server instance split) */
select /*+ parallel_index (product,4) */ date_range, product, unit_price
from product;

The format of the parallel_index hint is (for an index being scanned) is shown next.

/*+ parallel_index (indexname, DOP, parallel server instance split) */
select /*+ parallel_index (date_range_index,4) */ date_range,
product, unit_price
from product;

NOPARALLEL Hint
When you do not want to use the parallel processing of a select statement, the hint noparallel is available. This hint disables any default parallel processing the query coordinator may attempt to initiate. This hint is the same as saying: /*+ parallel (tablename, 1,1) */. The format of this hint is shown in the next listing.

/*+ noparallel (tablename) */

select /*+ noparallel */ date_range, product, unit_price
from product;

Parallel SQL statements
The performance of information manipulation and retrieval has always been and will continue to be an issue to everyone who has had their computer tied up doing the endless query. You must remember the update that started Monday and then ended when the computer crashed three days later. Oracle7 and Oracle8 have provided us with new strategies to improve the performance of our SQL statements. Remember that SQL that is poorly formatted and configured will still run poorly, but if the code is written efficiently then parallelism will improve performance.

Manual parallel processing is a derivative of the parallel query feature. Looking back into our years of experience, well maybe it was last week at a major bank’s data warehouse project, parallelism helped achieve a goal. Start of "dream" sequence. We created six separate processes that addressed a key value range in a 10,000,000-row table, and started these six wonderfully written programs. This program then allowed the Oracle Server to distribute the processing among the multiple parallel query processes on the server. The moral of this dream—anyone can perform manual parallel processing. Was this the best solution? By spreading out the processes among the many processors, we were able to achieve the overall performance required for the task. The question then becomes what was the trade-off versus serial processing of the same task.

The greatest advantage to us was that we could get the transactions per second required to complete this task in our lifetime. Even with parallel hints, the programs would still not provide the performance required.

For every battle won there is a cost; based on the perceived and estimated cost, decisions need to be made. Let’s look at what influence decisions.

Will the separation of tasks provide significant performance increases?
Do we have enough processes to manage the multiple tasks that will be started?
If one process fails does this affect the other processes? Every process will need the ability to be restarted, and no other process will need to depend on another parallel process
Can we separate the information with a reasonable amount of work?
Is this requirement a candidate for manual parallelism?
If we look at all of the above considerations, what makes an operation a better candidate for parallel execution? Let’s weigh a few items that help you decide on parallel execution.
Performance increase is always a factor in determining the way to approach the problem. What performance increase can you expect by enabling parallelism? This question is one that we are often asked and a difficult one to define. If every increase of 2% is considered a major benefit to your organization, then we must do everything possible to improve performance. If your organization requires a 25% improvement, then the choices are not as clear. In this case parallelism this approach may satisfy both schools of thought and should be considered for all performance reviews.
Available processes are a factor of system capacity. If you have the required number of cycles, more parallel query server processes can be started and a better use of system capacity can be achieved. If you do not have the available capacity, but want to pursue this approach, then a hardware augmentation will need to be performed before you embark on further parallelism.
Interdependence issues try to find answers to questions such as "Is it possible to perform a part of the task in isolation without concern for other operations?" as well as "If tasks can be performed without regard for other operations then it would appear to be a better candidate for parallelism".
This leads to the question of what work is required to separate the information to a level where it can be easily manipulated. Oracle8 has almost removed this question from consideration. If you have been diligent and partitioned your data already, this becomes a non-issue. People out there who want to take advantage of partitions and the parallel nature of the information, ask themselves if the change worth the cost. The best answer we can give to you is that using the features that are now available in your database toolkit must always be considered. So should we implement this feature? If your machine supports parallelism, then this option must be considered.

What Is the Degree of Parallelism?
When the database parallelizes an SQL statement, it parallelizes this statement over a number of parallel query server processes. The number of parallel query server processes used by an operation is known as the degree of parallelism. Without the degree of parallelism, the operation will be performed in serial. If you had wanted that, you would not have purchased 12 new processors, more memory and told everyone it was going to help. By understanding the definition of the degree of parallelism, you can now empower everyone in your organization to use all that new hardware and the new and improved database schema when issuing SQL statements.

The degree of parallelism is defined at the statement level. This is done through the use of imbedded hints within the statement or by default within the database. At the time of table or index creation you can specify a default degree of parallelism for the object. By default this may either be the number of disks or CPUs.

There are two types of parallel operations that we must define at this point as they will affect the degree of parallelism that any operation can perform—intra-operations and inter-operations. These two operations can be performed at the same time during statement execution.


The Degree of Parallelism within Operations
The balance between too much parallelism and too little is always a concern to us. Luckily we have a database that is smarter than your average bear. The Oracle query coordinator is that bear. The coordinator determines the degree of parallelism at runtime, based on number of factors. First, the coordinator looks to see if they are dealing with a smarter than normal programmer who has added hints to their SQL statement. In the case that this has not been done, it will check to see if your DBA was awake when the table or index was created. The coordinator will then look for the object’s default degree of parallelism. Once determined, this degree of parallelism will be used by the operation. If you create a table remember to follow the following format, with the parallel-specific section bolded.

SQL create table individual (
2 firstname varchar2(20) not null,
3 lastname varchar2(30) not null,
4 birthdate date not null,
5 gender varchar2(1) not null,
6 initial varchar2(1),
7 favorite_beatle varchar2(6))
8 parallel (degree 4);
Table created.
SQL

The coordinator will only request the number of parallel query server processes defined by the degree of parallelism. The physical number of processes that the coordinator will be able to get will depend upon how many are available at the time of operation execution.

So now we know how and where the database gets its degree of parallelism, but at some point we will need to tell what degree of parallelism to use. Therefore it is necessary to understand how best to tell Oracle what the degree of parallelism is defined for a table, index or hint. By following these rules and selecting the ones that relate to your specific operation, you can define your degree of parallelism, just like the pros.

Number of CPUs
Number of parallel server operations
Number of disks used by a table or index
Number of partitions that will be accessed
Suppose you have 8 CPUs, but your information is stored on 12 disks, the limiting factor here is CPUs. The degree of parallelism that we should use would be 8.

Although you may request eight parallel query server processes to process your statement, you may is not receive this many. Due to limits placed on the database, there may not be a sufficient number of parallel query servers available. If you have defined 25 PARALLEL_MAX_SERVERS, then you can only have 25 parallel processes running. If you exceed this number Oracle will not spawn additional processes, resulting in some statements being executed in serial mode by the query coordinator. In the case when you have specified the PARALLEL_MIN_PERCENT and the minimum fraction of parallelism is not available, you will receive an error during execution.

By defining the degree of parallelism for an SQL operation you can profoundly affect the processing of your information. Specify it too high and you may not get processes that you require when you want them. If the value is too low, you may find that your parallel query server processes are not being maximized and result in lost performance.

When considering your degree of parallelism, you should by default define it such that it optimizes a majority of operations to maximize the utilization of your I/O and CPU. In some cases this may not be the best approach, and you may want to override the default degree of parallelism. For example if your operation requires large amounts of I/O you may want to increase the degree of parallelism for the operation. If your operation requires a large amount of memory, then you may want to decrease the degree of parallelism. To override the default degree of parallelism, you may decide to incorporate hints into your SQL.

Now that we can define our degree of parallelism, we need to be able to use our knowledge, but where do we use these facts. By using parallelism in your SQL statements, you can increase the performance of many SQL statements. Let’s move on to looking at some familiar SQL statements, and how they are parallelized.



Summary
To summarize parallelism, its features, how to leverage its power, and using it in real-life situations, inspect the following points as a summary of what we have covered in this presentation.

Ensure that your hardware will support parallel query feature. Single CPU machines will have performance degraded by using parallel features.
The best database candidates for parallel usage are: data warehouses, VLDB and large data intensive applications.
Set-up Parallel Query feature, by setting appropriate values in the initialization parameter file.
Set the initialization parameter file entry PARALLEL_MIN_SERVERS = the likely number of simultaneous parallel operations
Set the initialization parameter file entry PARALLEL_MAX_SERVERS = 2 * # of CPUs * # of concurrent users
Set the initialization parameter file entry OPTIMIZER_PERCENT_PARALLEL = 100/number of concurrent users, this is dependent upon the weight you want to give to parallel processing in the cost-based optimizer.
Generally, you will be required to increase the parameter SHARED_POOL_SIZE when using parallelism.
Use hints in your SQL operations to tell Oracle to parallelize your operation.
Parallel hints may access data and indexes. When data is accessed by a table scan use parallel hint. When an index scan is used, the PARALLEL_INDEX hint may be used.
Don’t exceed the maximum number of parallel query server processes available on your machine, or your operation will not execute in parallel.
The number of parallel query server processes used by an operation is known as the degree of parallelism.
Tables can be created with a default degree of parallelism.
Hints will take precedence over the default degree of parallelism that has been defined for the table.
When creating a table or inserting records based on data in another table use parallel hints in both the insert and select sections of the SQL command.
Create and rebuild indexes using parallel hints to reduce the time to perform this lengthy task.
It's recommended to use Parallel query for tables with more than a millon rows
A parallel hint overides the instance settings
For a more complete discussion of parallelism, please read Oracle8 Tuning by Abbey, Corey, Dechichio and Abramson, printed by the Oracle Press 1997. Or the Followinf ML Notes:
237287.1 How To Verify Parallel Execution is running