APPLIES TO:
Oracle Database - Enterprise Edition - Version 10.1.0.2 and later
Information in this document applies to any platform.
***Checked for relevance on 15-Dec-2016***
SYMPTOMS
After upgrade, trying to open the database fails with
ORA-00704: bootstrap process failure ORA-00604: error occurred at recursive SQL level 1CHANGES
Upgrade of Database
CAUSE
1. There is a probable of this error when any unsupported operations are tried to force open the database.
2. This error can also occur when system datafile has corrupted blocks. (ORA-01578)
3. Either Bootstrap Objects are corrupted /Missing
SOLUTION
In Oracle, Bootstrap refers to loading of metadata (data dictionary) before we OPEN the database -
If ORA-00704 is accompanied with any of the below errors
A) ORA-39700: database must be opened with UPGRADE option
Ensure that environment variables point to the correct ORACLE_HOME, PATH etc and rerun catupgrd.sql
B) ORA-00942: table or view does not exist
A Particular bootstrap object must be missing
1. Shutdown (abort) the instance and clean up any OS structures used by the instance.
2. Cleanup if any sharedmemory/semaphores exists
Use the following commands to remove the shared memory segments and semaphores owned by the Oracle user.
At the unix command prompt do the following:
A) ipcs -m
This shows the memory segments owned by the Oracle user. If there are some segments owned by Oracle and related to instance which is failing to start then remove
> ipcrm -m <segment id number>
This removes the segment from memory so Oracle no longer has a hold on that piece of memory
C) ipcs -s
Check for any semaphores - Take the segment id of the correct instance which is failing to start and remove
>ipcrm -s <segment id number>
3. Retry the startup.
4. If the error persists then Restore the database from a backup taken prior to the upgrade
C) If ORA-01406: fetched column value was truncated OR any other errors,
Its possible that bootstrap objects are corrupted
Restore the database from a backup taken prior to the upgrade
Note: If the error persists follow up on the accompanying errors
DBRECOVER Recovery Options
For Oracle incidents, start with the DBRECOVER for Oracle trial to verify table visibility, row previews, and export readiness on copied datafiles. For MySQL and InnoDB incidents, DBRECOVER for MySQL is free software and can inspect.ibd files, ibdata1, and database directories locally.
When the case is urgent, preserve the original files first, work from copies, and contact paid emergency support with the database version, platform, error messages, file list, and recovery objective.