Real Time Issues ORACLE APPS

User experienced in Oracle Applications 11i/R12 that user has been assigned a responsibility but not able to see it while accessing it. However , everything seems fine with responsibility?

Workflow Directory Services User/Role Validation

This is often experienced in Oracle Applications 11i/R12 that user has been assigned a responsibility but not able to see it while accessing it. However , everything seems fine with responsibility ,it is not end dated and looks good. To overcome this issue , there is a concurrent request that does sync up of such users and responsibility in 11i version.

Login as System Administrator --> Submit Concurrent Program ""Workflow Directory Services User/Role Validation' with parameters 10000, yes, yes

This request would check all users and assigned responsibilities and should sync up users with attached responsibilities , users should be able to view assigned responsibility now.
p_BatchSize – 10000 (Default Value 10000)
p_Check_Dangling – Yes (Default value No)
Add missing user/role assignments – Yes (Default Value No)
Update WHO columns in WF tables – No (Default Value No)

While applying a patch if that patch is failing because of a pre-req then how you will apply that pre-req patch and resume with the current patch?
Ans: We need to take the backup of FND_INSTALL_PROCESSES and AD_DEFERRED_JOBS tables and restart directory at APPL_TOP/amdin/SID and then use adctrl to quit all the workers.
mv $APPL_TOP/admin/<SID>/restart $APPL_TOP/admin/<SID>/restart.old
create table applsys.FND_INSTALL_PROCESSES_11510 as select * from applsys.FND_INSTALL_PROCESSES
create table applsys.ad_deferred_jobs_11510 as select * from applsys.ad_deferred_jobs
drop table applsys.FND_INSTALL_PROCESSES
drop table applsys.ad_deferred_jobs
Then apply the pre-req patch , after that rename u r restart directory to its original name and create FND_INSTALL_PROCESSES and AD_DEFERRED_JOBS tables from the bcakup tables. Start adpatch session and take the options want to continue previous session.

ad patches must apply with prerequisite check before applying them how to do it?

The adpatch command should look like: adpatch option=prereq
This flag indicate to adpatch to check prerequisite before applying patch.
adpatch checks the prerequisite based on information from patch files and current snapshot on APPL_TOP.

When running adpatch with "prereq" flag, we might get an error message like:
Analyzing prerequisite patch information...
what is snapshot on current APPL_TOP doesn't exists error?
AutoPatch error: This patch has some prerequisites specified, but a "snapshot" of this APPL-TOP's file-system has never been taken, thereby rendering it impossible to check for the prerequisites.
Please take a "snapshot" of this APPL-TOP using "AD Administration" first.

This error message will show up if a snapshot on current APPL_TOP doesn't exists.

To create such snapshot on APPL_TOP: 
1) run adadmin
2) Select "Maintain Applications Files menu"
3) Select "Update current view snapshot"
4) Rerun adpatch

**It might take couple of hours depends on your hardware and APPL_TOP size.

how adpatch check the prerequisites?

1) Check if a snapshot on current APPL_TOP exist.
using sql script - adbkflsn.sql (if not, will terminate with above error message.....)

2) adpatch uploads a ldt file with FNDLOAD utility into system (bug-fixes).
ldt file name is: b[PATCH_NUMER].ldt - comes from patch root directory.

3) Execute the UMS analysis engine based on the snapshot and bug-fixes to check if all prerequisites exists.

Shell Scripting(sh&ksh)

USEFULL LINK FOR LINUX
http://linuxcommand.org/lc3_writing_shell_scripts.php


$# Stores the number of command-line arguments that were passed to the shell program.
$? Stores the exit value of the last command that was executed.
$* is a string containing all arguments.
$1 is the first argument.


SSH FUNCTION

{
if [ ${DB_OS_TYPE} == "SunOS" ]; then
        COMMAND='cat /etc/release | head -1'
    DB_CPU_TEMP=`exec_over_ssh ${ORA_USER} ${ORA_HOST} ". /export/home/$ORA_USER/.profile > /dev/null;opatch lsinventory| grep -i 'DATABASE PATCH'| head -1"`
else
    DB_CPU_TEMP=`exec_over_ssh ${ORA_USER} ${ORA_HOST} ". /home/$ORA_USER/.bash_profile > /dev/null;opatch lsinventory| grep -i 'DATABASE PATCH'| head -1"`
fi



exec_over_ssh()
{
        USER=$1
        HOST=$2
        COMMAND=$3
        OUTPUT=`ssh ${USER}@${HOST} ${COMMAND}`
        echo ${OUTPUT}
}

###################created script by me#
#!/bin/bash
###set -x
user=$1
HOST=$2
COMMAND=$3
exec_ssh()
{
OUTPUT=`ssh ${user}@${HOST} ${COMMAND}`
echo ${OUTPUT}
}
if [ $# == 3 ]
then
exec_ssh
else
echo "enter correect parameter"
exit
fi




REDOLOG VS ARCHIVE LOG


SQL> SELECT distinct member LOGFILENAME FROM V$LOGFILE;

LOGFILENAME
--------------------------------------------------------------------------------
/u01/install/APPS/data/ebsdb/log1.dbf
/u01/install/APPS/data/ebsdb/log2.dbf
/u01/install/APPS/data/ebsdb/log3.dbf

ALTER SYSTEM SWITCH LOGFILE vs
ALTER SYSTEM ARCHIVE LOG CURRENT


hat is the difference between ALTER SYSTEM SWITCH LOGFILE and ALTER SYSTEM ARCHIVE LOG CURRENT, and when do I use each?
Answer:  Yes, both ALTER SYSTEM SWITCH LOGFILE and ALTER SYSTEM ARCHIVE LOG CURRENT will force a log switch, but they do it in different ways! 
Both the SWITCH LOGFILE and ARCHIVE LOG CURRENT write a quiesce checkpoint, a firm place whereby that last redo log is a part of the hot backup, but ARCHIVE LOG CURRENT waits for the writing to complete.  This can take several minutes for multi-gigabyte redo logs.
Conversely, the ALTER SYSTEM SWITCH LOGFILE command is very fast and returns control to the caller in less than a second while ALTER SYSTEM ARCHIVE LOG CURRENT pauses.
As we see below, the ALTER SYSTEM SWITCH LOGFILE is fast because it does not wait for the archiver process (ARCH) to complete writing the online redo log to the archivelog log filesystem:

  1. It issues database checkpoint
  2.  It immediately starts writing to the next redo log
  3.  In the background, the "switch logfile" command tells the ARCH background process to copy the "old" redo log file to the redo log filesystem.  
Here are the important differences between ALTER SYSTEM SWITCH LOGFILE and ALTER SYSTEM ARCHIVE LOG CURRENT:
  •  RAC:  If you are running RAC, the ALTER SYSTEM ARCHIVE LOG CURRENTwill switch the logs on all RAC nodes (instances), whereasALTER SYSTEM SWITCH LOGFILE will only switch he logfile on the instance where you issue the switch command.  Hence, ALTER SYSTEM ARCHIVE LOG CURRENT is a best practice for RAC systems.


ENABLING ARCHIVELOG and SWITCHING LOG

SQL> archive log list
Database log mode              No Archive Mode
Automatic archival             Disabled
Archive destination            /u01/install/APPS/data/ebsdb/archive
Oldest online log sequence     7
Current log sequence           9
SQL> shut immediate;
Database closed.
Database dismounted.
ORACLE instance shut down.
SQL> startup mount;
ORA-32004: obsolete or deprecated parameter(s) specified for RDBMS instance
ORACLE instance started.

Total System Global Area 2147483648 bytes
Fixed Size                  2926472 bytes
Variable Size             553650296 bytes
Database Buffers         1577058304 bytes
Redo Buffers               13848576 bytes
Database mounted.
SQL> alter database archivelog;

Database altered.

SQL> alter database open;

Database altered.

SQL> archive log list;
Database log mode              Archive Mode
Automatic archival             Enabled
Archive destination            /u01/install/APPS/data/ebsdb/archive
Oldest online log sequence     7
Next log sequence to archive   9
Current log sequence           9
SQL> alter switch logfile;
alter switch logfile
      *
ERROR at line 1:
ORA-00940: invalid ALTER command


SQL> ALTER SYSTEM SWITCH LOGFILE ;

System altered.

SQL> archive log list;
Database log mode              Archive Mode
Automatic archival             Enabled
Archive destination            /u01/install/APPS/data/ebsdb/archive
Oldest online log sequence     8
Next log sequence to archive   10
Current log sequence           10
SQL> exit

REALTIME ISSUES - ORACLECLOUD

****DCS-10045:Validation error encountered: Backup type is invalid.****
CONSOLE ERROR:

Status: The database restore operation failed. Run 'dbcli list-jobs' on all hosts in the DB system to check for problems and fix them before rerunning the operation.

[root@testbackupdb ~]# dbcli list-jobs

ID                                       Description                                                                 Created                             Status
---------------------------------------- --------------------------------------------------------------------------- ----------------------------------- ----------
7182911b-1733-4805-b351-081f6be6605b     Authentication key update for DCS_ADMIN                                     May 5, 2019 4:09:58 PM UTC          Success
ce8f602b-84c7-4295-982a-49098dca465d     Provisioning service creation                                               May 5, 2019 4:11:41 PM UTC          Success
47876bcb-50a5-42ac-9648-52221a12a974     SSH keys update                                                             May 5, 2019 5:03:22 PM UTC          Success
9d01bc5f-0831-48ed-a1fc-f0982e54feaa     SSH key delete                                                              May 5, 2019 5:05:14 PM UTC          Success
80f9b24e-1061-4906-88d1-78f6b824b454     create object store:b9a7iawWuBfGYFaNH7MC                                    May 6, 2019 3:14:45 AM UTC          Success
25845fda-ea88-40b0-bb54-3be86ec265f6     create backup config:b9a7iawWuBfGYFaNH7MC_BC                                May 6, 2019 3:15:31 AM UTC          Success
64903937-7519-415b-b658-186cc6bebbdd     update database : BKUPDB                                                    May 6, 2019 3:16:23 AM UTC          Success
f3d0309f-3bf0-4ec2-b3d7-2d2fcde8599f     Server Patching                                                             May 6, 2019 3:19:53 AM UTC          Success
4fa3b016-8448-4535-b901-a5edd6dc2c1c     Create Regular-L0 Backup with TAG-DBTRegular-L01557111640379VSL for Db:BKUPDB in OSS:b9a7iawWuBfGYFaNH7MC May 6, 2019 3:19:56 AM UTC          Success
5e902ca8-f332-4c7f-86ca-afca1590e4b9     Delete Backup for Database name: BKUPDB_fra1k8                              May 6, 2019 3:26:48 AM UTC          Success
251ba542-234e-40f6-a7a9-f3a1ef281cb8     DB Home Prechecks                                                           May 6, 2019 5:05:43 AM UTC          Success
bc5eb7cd-b320-453f-8dea-e296e645df58     Create Longterm Backup with TAG-DBTLongterm1557125463725Zmf for Db:BKUPDB in OSS:b9a7iawWuBfGYFaNH7MC May 6, 2019 6:52:12 AM UTC          Success
43a924a2-6e16-43e0-a2e9-cc4cd1d6b53f     Delete Backup for Database name: BKUPDB_fra1k8                              May 6, 2019 7:05:18 AM UTC          Success
eefab9b7-2b32-4327-9eff-bf55d99ca7eb     Create recovery-pitr : time '05/06/2019 07:10:08' for db : BKUPDB           May 6, 2019 7:10:57 AM UTC          Failure

[root@testbackupdb ~]# dbcli describe-job  --jobid eefab9b7-2b32-4327-9eff-bf55d99ca7eb

Job details
----------------------------------------------------------------
                     ID:  eefab9b7-2b32-4327-9eff-bf55d99ca7eb
            Description:  Create recovery-pitr : time '05/06/2019 07:10:08' for db : BKUPDB
                 Status:  Failure
                Created:  May 6, 2019 7:10:57 AM UTC
                Message:  DCS-10001:Internal error encountered: Failed to run RMAN command. Please refer log at location : testbackupdb: /opt/oracle/dcs/log/testbackupdb/rman/bkup/BKUPDB_fra1k8/rman_restore_2019-05-06_07-12-23-4741329873884168661.log.Failed to do restore validati

Task Name                                Start Time                          End Time                            Status
---------------------------------------- ----------------------------------- ----------------------------------- ----------
task:TaskZLockWrapper_7173               May 6, 2019 7:11:07 AM UTC          May 6, 2019 7:12:37 AM UTC          Failure
task:TaskSequential_7174                 May 6, 2019 7:11:07 AM UTC          May 6, 2019 7:12:37 AM UTC          Failure
Database recovery validation             May 6, 2019 7:11:08 AM UTC          May 6, 2019 7:12:37 AM UTC          Failure

[
ISSUE is #####rman log error#showing inorder recovery of scn#####

validation succeeded for archived log
recovery will be done up to SCN 1761481
Media recovery start SCN is 1760987
Recovery must be done beyond SCN 1761012 to clear datafile fuzziness
could not locate pieces of backup set key 20
validation succeeded for backup piece
Finished restore at 2019/05/06 07:12:36

Recovery Manager complete.

we have used until time "05/06/2019 07:10:08"


SQL> select sequence#,first_change#,next_change# from v$log order by 1;

 SEQUENCE# FIRST_CHANGE# NEXT_CHANGE#
---------- ------------- ------------
        18       1761147      1761155
        19       1761155      1761444
        20       1761444   2.8147E+14


FIX :so i restored using until scn 1761444 

[root@testbackupdb ~]# dbcli list-jobs

ID                                       Description                                                                 Created                             Status
---------------------------------------- --------------------------------------------------------------------------- ----------------------------------- ----------
7182911b-1733-4805-b351-081f6be6605b     Authentication key update for DCS_ADMIN                                     May 5, 2019 4:09:58 PM UTC          Success
ce8f602b-84c7-4295-982a-49098dca465d     Provisioning service creation                                               May 5, 2019 4:11:41 PM UTC          Success
47876bcb-50a5-42ac-9648-52221a12a974     SSH keys update                                                             May 5, 2019 5:03:22 PM UTC          Success
9d01bc5f-0831-48ed-a1fc-f0982e54feaa     SSH key delete                                                              May 5, 2019 5:05:14 PM UTC          Success
80f9b24e-1061-4906-88d1-78f6b824b454     create object store:b9a7iawWuBfGYFaNH7MC                                    May 6, 2019 3:14:45 AM UTC          Success
25845fda-ea88-40b0-bb54-3be86ec265f6     create backup config:b9a7iawWuBfGYFaNH7MC_BC                                May 6, 2019 3:15:31 AM UTC          Success
64903937-7519-415b-b658-186cc6bebbdd     update database : BKUPDB                                                    May 6, 2019 3:16:23 AM UTC          Success
f3d0309f-3bf0-4ec2-b3d7-2d2fcde8599f     Server Patching                                                             May 6, 2019 3:19:53 AM UTC          Success
4fa3b016-8448-4535-b901-a5edd6dc2c1c     Create Regular-L0 Backup with TAG-DBTRegular-L01557111640379VSL for Db:BKUPDB in OSS:b9a7iawWuBfGYFaNH7MC May 6, 2019 3:19:56 AM UTC          Success
5e902ca8-f332-4c7f-86ca-afca1590e4b9     Delete Backup for Database name: BKUPDB_fra1k8                              May 6, 2019 3:26:48 AM UTC          Success
251ba542-234e-40f6-a7a9-f3a1ef281cb8     DB Home Prechecks                                                           May 6, 2019 5:05:43 AM UTC          Success
bc5eb7cd-b320-453f-8dea-e296e645df58     Create Longterm Backup with TAG-DBTLongterm1557125463725Zmf for Db:BKUPDB in OSS:b9a7iawWuBfGYFaNH7MC May 6, 2019 6:52:12 AM UTC          Success
43a924a2-6e16-43e0-a2e9-cc4cd1d6b53f     Delete Backup for Database name: BKUPDB_fra1k8                              May 6, 2019 7:05:18 AM UTC          Success
eefab9b7-2b32-4327-9eff-bf55d99ca7eb     Create recovery-pitr : time '05/06/2019 07:10:08' for db : BKUPDB           May 6, 2019 7:10:57 AM UTC          Failure
eab18e5c-ea75-41dd-a4ed-98aa209f2132     Create detailed Backup Report                                               May 7, 2019 2:16:37 AM UTC          Success
f22341d1-d835-4828-8bf3-b1d8f0158c9b     Create recovery-latest for db : BKUPDB                                      May 7, 2019 2:45:07 AM UTC          Failure
65f2fcf5-776c-4630-ab8f-76975ab0b935     Create recovery-scn : scn 1761444 for db : BKUPDB                           May 7, 2019 2:57:46 AM UTC          Success




[root@testbackupdb ~]# dbcli describe-job --jobid 65f2fcf5-776c-4630-ab8f-76975ab0b935

Job details
----------------------------------------------------------------
                     ID:  65f2fcf5-776c-4630-ab8f-76975ab0b935
            Description:  Create recovery-scn : scn 1761444 for db : BKUPDB
                 Status:  Running
                Created:  May 7, 2019 2:57:46 AM UTC
                Message:

Task Name                                Start Time                          End Time                            Status
---------------------------------------- ----------------------------------- ----------------------------------- ----------
Database recovery validation             May 7, 2019 2:57:58 AM UTC          May 7, 2019 2:59:37 AM UTC          Success
Database recovery                        May 7, 2019 2:59:38 AM UTC          May 7, 2019 2:59:38 AM UTC          Running

[root@testbackupdb ~]# dbcli describe-job --jobid 65f2fcf5-776c-4630-ab8f-76975ab0b935

Job details
----------------------------------------------------------------
                     ID:  65f2fcf5-776c-4630-ab8f-76975ab0b935
            Description:  Create recovery-scn : scn 1761444 for db : BKUPDB
                 Status:  Success
                Created:  May 7, 2019 2:57:46 AM UTC
                Message:

Task Name                                Start Time                          End Time                            Status
---------------------------------------- ----------------------------------- ----------------------------------- ----------
Database recovery validation             May 7, 2019 2:57:58 AM UTC          May 7, 2019 2:59:37 AM UTC          Success
Database recovery                        May 7, 2019 2:59:38 AM UTC          May 7, 2019 3:02:47 AM UTC          Success
Enable block change tracking             May 7, 2019 3:02:47 AM UTC          May 7, 2019 3:02:50 AM UTC          Success
Database opening                         May 7, 2019 3:02:50 AM UTC          May 7, 2019 3:03:44 AM UTC          Success
Database restart                         May 7, 2019 3:03:44 AM UTC          May 7, 2019 3:05:06 AM UTC          Success
Recovery metadata persistance            May 7, 2019 3:05:06 AM UTC          May 7, 2019 3:05:06 AM UTC          Success

[root@testbackupdb ~]#

REALTIME ISSUES - WEBLOGIC

Weblogic :

1.MACHINE IS INCOMPATABLE STATUS (2020154.1)

managedserver.log  ----OUTPUT

<May 2, 2019 10:04:00 PM BST> <Error> <NodeManager> <BEA-300034> <Could not execute command "kill" for server "P6_PRM1" using the node manager. Reason: Server 'P6_PRM1' is not currently running.>
<May 2, 2019 10:04:12 PM BST> <Error> <NodeManager> <BEA-300048> <Unable to start the server P6_PRM1 : Exception while starting server 'P6_PRM1'>
<May 2, 2019 10:06:57 PM BST> <Error> <NodeManager> <BEA-300034> <Could not execute command "kill" for server "P6_PR_REP1_DS" using the node manager. Reason: Server 'P6_PR_REP1_DS' is not currently running.>
<May 2, 2019 10:07:07 PM BST> <Error> <NodeManager> <BEA-300048> <Unable to start the server P6_PR_REP1_DS : Exception while starting server 'P6_PR_REP1_DS'>
<May 2, 2019 10:11:40 PM BST> <Error> <NodeManager> <BEA-300034> <Could not execute command "kill" for server "P6_WEB_SRV1" using the node manager. Reason: Server 'P6_WEB_SRV1' is not currently running.>
<May 2, 2019 10:11:50 PM BST> <Error> <NodeManager> <BEA-300048> <Unable to start the server P6_WEB_SRV1 : Exception while starting server 'P6_WEB_SRV1'>
<May 3, 2019 6:52:12 AM BST> <Error> <HTTP> <BEA-101215> <Malformed Request "/<script>alert(53416)</script>". Request parsing failed, Code: -1>
<May 3, 2019 7:02:07 AM BST> <Error> <RJVM> <BEA-000503> <Incoming message header or abbreviation processing failed
 java.io.InvalidClassException: sun.rmi.server.UnicastRef; Unauthorized deserialization attempt
java.io.InvalidClassException: sun.rmi.server.UnicastRef; Unauthorized deserialization attempt
        at weblogic.utils.io.oif.WebLogicObjectInputFilter.checkLegacyBlacklistIfNeeded(WebLogicObjectInputFilter.java:236)
        at weblogic.utils.io.FilteringObjectInputStream.checkLegacyBlacklistIfNeeded(FilteringObjectInputStream.java:54)
        at weblogic.rjvm.InboundMsgAbbrev$ServerChannelInputStream.resolveClass(InboundMsgAbbrev.java:122)
        at java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1612)
        at java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1517)
        Truncated. see log file for complete stacktrace
>
<May 3, 2019 10:04:00 PM BST> <Error> <NodeManager> <BEA-300034> <Could not execute command "kill" for server "P6_PRM1" using the node manager. Reason: Server 'P6_PRM1' is not currently running.>
<May 3, 2019 10:04:11 PM BST> <Error> <NodeManager> <BEA-300048> <Unable to start the server P6_PRM1 : Exception while starting server 'P6_PRM1'>
<May 3, 2019 10:06:53 PM BST> <Error> <NodeManager> <BEA-300034> <Could not execute command "kill" for server "P6_PR_REP1_DS" using the node manager. Reason: Server 'P6_PR_REP1_DS' is not currently running.>
<May 3, 2019 10:07:03 PM BST> <Error> <NodeManager> <BEA-300048> <Unable to start the server P6_PR_REP1_DS : Exception while starting server 'P6_PR_REP1_DS'>
<May 3, 2019 10:11:30 PM BST> <Error> <NodeManager> <BEA-300034> <Could not execute command "kill" for server "P6_WEB_SRV1" using the node manager. Reason: Server 'P6_WEB_SRV1' is not currently running.>
<May 3, 2019 10:11:40 PM BST> <Error> <NodeManager> <BEA-300048> <Unable to start the server P6_WEB_SRV1 : Exception while starting server 'P6_WEB_SRV1'>

************
<May 3, 2019 10:04:10 PM BST> <Critical> <WebLogicServer> <BEA-000386> <Server subsystem failed. Reason: weblogic.security.SecurityInitializationException: Authentication denied: Boot identity not valid; The user name and/or password from the boot identity file (boot.properties) is not valid. The boot identity may have been changed since the boot identity file was created. Please edit and update the boot identity file with the proper values of username and password. The first time the updated boot identity file is used to start the server, these new values are encrypted.
weblogic.security.SecurityInitializationException: Authentication denied: Boot identity not valid; The user name and/or password from the boot identity file (boot.properties) is not valid. The boot identity may have been changed since the boot identity file was created. Please edit and update the boot identity file with the proper values of username and password. The first time the updated boot identity file is used to start the server, these new values are encrypted.
        at weblogic.security.service.CommonSecurityServiceManagerDelegateImpl.doBootAuthorization(CommonSecurityServiceManagerDelegateImpl.java:960)
        at weblogic.security.service.CommonSecurityServiceManagerDelegateImpl.initialize(CommonSecurityServiceManagerDelegateImpl.java:1054)
        at weblogic.security.service.SecurityServiceManager.initialize(SecurityServiceManager.java:888)
        at weblogic.security.SecurityService.start(SecurityService.java:141)
        at weblogic.t3.srvr.SubsystemRequest.run(SubsystemRequest.java:64)
        Truncated. see log file for complete stacktrace
Caused By: javax.security.auth.login.FailedLoginException: [Security:090303]Authentication



FIX is either boot.properties file or ldap under domain_name/servers/managedserver/data

moved ldap to ldapbkp and created ldap direcotry again by using mkdir command
[XXXXXXXd1@XXXXXXX6 data]$ ls
ldap  ldapbkp  nodemanager  store
[XXXXXXXd1@XXXXXXX6 data]$ pwd
/XXXXXXXd1/primapp/XXXXXXXppm/domain/XXXXXXXppm83/servers/P6_PRM1/data


copied the correct boot.properties file from another running managed server
[XXXXXXXd1@XXXXXXX6 nodemanager]$ ls
boot.properties  boot.propertiesbkp  P6_PRM1.lck  P6_PRM1.pid  P6_PRM1.state  P6_PRM1.statebkp  P6_PRM1.url  startup.properties
[XXXXXXXd1@XXXXXXX6 nodemanager]$


NOTE:Also you can try managedserver.state file remove and strat from console



ORACLE CLOUD RECOVERY ISSUE


****DCS-10045:Validation error encountered: Backup type is invalid.****


[root@testbackupdb ~]# dbcli list-jobs

ID                                       Description                                                                 Created                             Status
---------------------------------------- --------------------------------------------------------------------------- ----------------------------------- ----------
7182911b-1733-4805-b351-081f6be6605b     Authentication key update for DCS_ADMIN                                     May 5, 2019 4:09:58 PM UTC          Success
ce8f602b-84c7-4295-982a-49098dca465d     Provisioning service creation                                               May 5, 2019 4:11:41 PM UTC          Success
47876bcb-50a5-42ac-9648-52221a12a974     SSH keys update                                                             May 5, 2019 5:03:22 PM UTC          Success
9d01bc5f-0831-48ed-a1fc-f0982e54feaa     SSH key delete                                                              May 5, 2019 5:05:14 PM UTC          Success
80f9b24e-1061-4906-88d1-78f6b824b454     create object store:b9a7iawWuBfGYFaNH7MC                                    May 6, 2019 3:14:45 AM UTC          Success
25845fda-ea88-40b0-bb54-3be86ec265f6     create backup config:b9a7iawWuBfGYFaNH7MC_BC                                May 6, 2019 3:15:31 AM UTC          Success
64903937-7519-415b-b658-186cc6bebbdd     update database : BKUPDB                                                    May 6, 2019 3:16:23 AM UTC          Success
f3d0309f-3bf0-4ec2-b3d7-2d2fcde8599f     Server Patching                                                             May 6, 2019 3:19:53 AM UTC          Success
4fa3b016-8448-4535-b901-a5edd6dc2c1c     Create Regular-L0 Backup with TAG-DBTRegular-L01557111640379VSL for Db:BKUPDB in OSS:b9a7iawWuBfGYFaNH7MC May 6, 2019 3:19:56 AM UTC          Success
5e902ca8-f332-4c7f-86ca-afca1590e4b9     Delete Backup for Database name: BKUPDB_fra1k8                              May 6, 2019 3:26:48 AM UTC          Success
251ba542-234e-40f6-a7a9-f3a1ef281cb8     DB Home Prechecks                                                           May 6, 2019 5:05:43 AM UTC          Success
bc5eb7cd-b320-453f-8dea-e296e645df58     Create Longterm Backup with TAG-DBTLongterm1557125463725Zmf for Db:BKUPDB in OSS:b9a7iawWuBfGYFaNH7MC May 6, 2019 6:52:12 AM UTC          Success
43a924a2-6e16-43e0-a2e9-cc4cd1d6b53f     Delete Backup for Database name: BKUPDB_fra1k8                              May 6, 2019 7:05:18 AM UTC          Success
eefab9b7-2b32-4327-9eff-bf55d99ca7eb     Create recovery-pitr : time '05/06/2019 07:10:08' for db : BKUPDB           May 6, 2019 7:10:57 AM UTC          Failure

[root@testbackupdb ~]# dbcli describe-job  --jobid eefab9b7-2b32-4327-9eff-bf55d99ca7eb

Job details
----------------------------------------------------------------
                     ID:  eefab9b7-2b32-4327-9eff-bf55d99ca7eb
            Description:  Create recovery-pitr : time '05/06/2019 07:10:08' for db : BKUPDB
                 Status:  Failure
                Created:  May 6, 2019 7:10:57 AM UTC
                Message:  DCS-10001:Internal error encountered: Failed to run RMAN command. Please refer log at location : testbackupdb: /opt/oracle/dcs/log/testbackupdb/rman/bkup/BKUPDB_fra1k8/rman_restore_2019-05-06_07-12-23-4741329873884168661.log.Failed to do restore validati

Task Name                                Start Time                          End Time                            Status
---------------------------------------- ----------------------------------- ----------------------------------- ----------
task:TaskZLockWrapper_7173               May 6, 2019 7:11:07 AM UTC          May 6, 2019 7:12:37 AM UTC          Failure
task:TaskSequential_7174                 May 6, 2019 7:11:07 AM UTC          May 6, 2019 7:12:37 AM UTC          Failure
Database recovery validation             May 6, 2019 7:11:08 AM UTC          May 6, 2019 7:12:37 AM UTC          Failure

[
ISSUE is #####rman log error#showing inorder recovery of scn#####

validation succeeded for archived log
recovery will be done up to SCN 1761481
Media recovery start SCN is 1760987
Recovery must be done beyond SCN 1761012 to clear datafile fuzziness
could not locate pieces of backup set key 20
validation succeeded for backup piece
Finished restore at 2019/05/06 07:12:36

Recovery Manager complete.

we have used until time "05/06/2019 07:10:08"


SQL> select sequence#,first_change#,next_change# from v$log order by 1;

 SEQUENCE# FIRST_CHANGE# NEXT_CHANGE#
---------- ------------- ------------
        18       1761147      1761155
        19       1761155      1761444
        20       1761444   2.8147E+14


FIX : so i restored using until scn 1761444 

[root@testbackupdb ~]# dbcli list-jobs

ID                                       Description                                                                 Created                             Status
---------------------------------------- --------------------------------------------------------------------------- ----------------------------------- ----------
7182911b-1733-4805-b351-081f6be6605b     Authentication key update for DCS_ADMIN                                     May 5, 2019 4:09:58 PM UTC          Success
ce8f602b-84c7-4295-982a-49098dca465d     Provisioning service creation                                               May 5, 2019 4:11:41 PM UTC          Success
47876bcb-50a5-42ac-9648-52221a12a974     SSH keys update                                                             May 5, 2019 5:03:22 PM UTC          Success
9d01bc5f-0831-48ed-a1fc-f0982e54feaa     SSH key delete                                                              May 5, 2019 5:05:14 PM UTC          Success
80f9b24e-1061-4906-88d1-78f6b824b454     create object store:b9a7iawWuBfGYFaNH7MC                                    May 6, 2019 3:14:45 AM UTC          Success
25845fda-ea88-40b0-bb54-3be86ec265f6     create backup config:b9a7iawWuBfGYFaNH7MC_BC                                May 6, 2019 3:15:31 AM UTC          Success
64903937-7519-415b-b658-186cc6bebbdd     update database : BKUPDB                                                    May 6, 2019 3:16:23 AM UTC          Success
f3d0309f-3bf0-4ec2-b3d7-2d2fcde8599f     Server Patching                                                             May 6, 2019 3:19:53 AM UTC          Success
4fa3b016-8448-4535-b901-a5edd6dc2c1c     Create Regular-L0 Backup with TAG-DBTRegular-L01557111640379VSL for Db:BKUPDB in OSS:b9a7iawWuBfGYFaNH7MC May 6, 2019 3:19:56 AM UTC          Success
5e902ca8-f332-4c7f-86ca-afca1590e4b9     Delete Backup for Database name: BKUPDB_fra1k8                              May 6, 2019 3:26:48 AM UTC          Success
251ba542-234e-40f6-a7a9-f3a1ef281cb8     DB Home Prechecks                                                           May 6, 2019 5:05:43 AM UTC          Success
bc5eb7cd-b320-453f-8dea-e296e645df58     Create Longterm Backup with TAG-DBTLongterm1557125463725Zmf for Db:BKUPDB in OSS:b9a7iawWuBfGYFaNH7MC May 6, 2019 6:52:12 AM UTC          Success
43a924a2-6e16-43e0-a2e9-cc4cd1d6b53f     Delete Backup for Database name: BKUPDB_fra1k8                              May 6, 2019 7:05:18 AM UTC          Success
eefab9b7-2b32-4327-9eff-bf55d99ca7eb     Create recovery-pitr : time '05/06/2019 07:10:08' for db : BKUPDB           May 6, 2019 7:10:57 AM UTC          Failure
eab18e5c-ea75-41dd-a4ed-98aa209f2132     Create detailed Backup Report                                               May 7, 2019 2:16:37 AM UTC          Success
f22341d1-d835-4828-8bf3-b1d8f0158c9b     Create recovery-latest for db : BKUPDB                                      May 7, 2019 2:45:07 AM UTC          Failure
65f2fcf5-776c-4630-ab8f-76975ab0b935     Create recovery-scn : scn 1761444 for db : BKUPDB                           May 7, 2019 2:57:46 AM UTC          Success




[root@testbackupdb ~]# dbcli describe-job --jobid 65f2fcf5-776c-4630-ab8f-76975ab0b935

Job details
----------------------------------------------------------------
                     ID:  65f2fcf5-776c-4630-ab8f-76975ab0b935
            Description:  Create recovery-scn : scn 1761444 for db : BKUPDB
                 Status:  Running
                Created:  May 7, 2019 2:57:46 AM UTC
                Message:

Task Name                                Start Time                          End Time                            Status
---------------------------------------- ----------------------------------- ----------------------------------- ----------
Database recovery validation             May 7, 2019 2:57:58 AM UTC          May 7, 2019 2:59:37 AM UTC          Success
Database recovery                        May 7, 2019 2:59:38 AM UTC          May 7, 2019 2:59:38 AM UTC          Running

[root@testbackupdb ~]# dbcli describe-job --jobid 65f2fcf5-776c-4630-ab8f-76975ab0b935

Job details
----------------------------------------------------------------
                     ID:  65f2fcf5-776c-4630-ab8f-76975ab0b935
            Description:  Create recovery-scn : scn 1761444 for db : BKUPDB
                 Status:  Success
                Created:  May 7, 2019 2:57:46 AM UTC
                Message:

Task Name                                Start Time                          End Time                            Status
---------------------------------------- ----------------------------------- ----------------------------------- ----------
Database recovery validation             May 7, 2019 2:57:58 AM UTC          May 7, 2019 2:59:37 AM UTC          Success
Database recovery                        May 7, 2019 2:59:38 AM UTC          May 7, 2019 3:02:47 AM UTC          Success
Enable block change tracking             May 7, 2019 3:02:47 AM UTC          May 7, 2019 3:02:50 AM UTC          Success
Database opening                         May 7, 2019 3:02:50 AM UTC          May 7, 2019 3:03:44 AM UTC          Success
Database restart                         May 7, 2019 3:03:44 AM UTC          May 7, 2019 3:05:06 AM UTC          Success
Recovery metadata persistance            May 7, 2019 3:05:06 AM UTC          May 7, 2019 3:05:06 AM UTC          Success

[root@testbackupdb ~]#



Oracle application questions

Question 28: why we run datapatch:

The datapatch utility will run the necessary apply scripts to load the modified SQL files into the database. Entires will be added to both DBA_REGISTRY_HISTORY and DBA_REGISTRY_SQLPATCH views automatically.

XXXXXX1> SELECT patch_id, status FROM dba_registry_sqlpatch;

   PATCH_ID STATUS

  22139226 SUCCESS

  21948354 SUCCESS

  20204035 SUCCESS

  22139226 SUCCESS

  30805558 END

  20204035 END

  31219939 END

  31113348 BEGIN

 

Details of Adpreclone

adpreclone.pl dbtier

it will run the below two stages

1)Techstack

1)Database


Techstack:

it will $ORACLE_HOME/clone/Jlib -->directory it will create and update the all library files  (when we apply psu patches like that it will udpate the psu version related class files in this directory)

NOTE: if

-bash-3.2$ pwd
$FND_TOP/sql
-bash-3.2$ ls wfver.sql
wfver.sql   ---> this script will idetify the xmlparser version if psu related issues (need to be run from application node)


related to datafile and required for cloning

creates the driver files ORACLE_HOME/appsutil/driver/instconf.drv

convert the inventory file from binary to xml

$ORACLE_HOME/appsutil/clone/context/db/Sid_context.xml

example: it will save the binary information to xml as like below

</nls_settings>
      <oa_db_server>
         <dbhost oa_var="s_dbhost">xxxxxxp2</dbhost>
         <dbport oa_var="s_dbport" oa_type="PORT">1522</dbport>
         <dbtype oa_var="s_dbtype">VISION</dbtype>
         <dbcset oa_var="s_dbcset">UTF8</dbcset>
         <dbseed oa_var="s_dbseed">Fresh Install</dbseed>
         <dbcomp oa_var="s_dbcomp">oracle.apps.dbseed.fresh</dbcomp>
         <dbprocesses oa_var="s_db_processes">1500</dbprocesses>
         <SESSIONS oa_var="s_db_sessions">400</SESSIONS>
         <dbfiles oa_var="s_dbfiles">1500</dbfiles>
         <dbblockbuffers oa_var="s_dbblock_buffers">0</dbblockbuffers>
         <dbcachesize oa_var="s_dbcache_size">163577856</dbcachesize>
         <dbsharedpool oa_var="s_dbsharedpool_size">300000000</dbsharedpool>
         <dbrollbacksegs oa_var="s_db_rollback_segs">NOROLLBACK</dbrollbacksegs>
         <dbutilfiledir oa_var="s_db_util_filedir" osd="unix">/xxxxxxp2/app/comn/temp, /usr/tmp, /tmp</dbutilfiledir>
         <undotablespace oa_var="s_undo_tablespace">APPS_UNDOTS1</undotablespace>
         <APPS_DATA_FILE_DIR oa_var="s_db_data_file_dir">/xxxxxxp2/oracle/db/11.2.0/appsutil/outbound/xxxxxxp2_xxxxxxp2</APPS_DATA_FILE_DIR>
         <o7dictionaryaccess oa_var="s_o7_dictionary_accessibility">TRUE</o7dictionaryaccess>
         <db_walletdir oa_var="s_dbWalletDir">/xxxxxxp2/oracle/db/11.2.0/appsutil/wallet</db_walletdir>



Database:

creating datbase control file script/datafile location information file

at    $ORACLE_HOME/appsutil/template/adcrdbclone.sql, dbfinfo.lst

example:
$ cat dbfinfo.lst

param|s_undo_tablespace|APPS_UNDOTS1
param|s_database|db112
param|UNDO_MANAGEMENT|AUTO
param|isspfileexists|false
param|s_dbfiles|1500
log|7|/xxxxxxp2/dbdata/redo1/log07.dbf|1610612736
log|8|/xxxxxxp2/dbdata/redo2/log08.dbf|1610612736
log|9|/xxxxxxp2/dbdata/redo1/log09.dbf|1610612736
tmp|TEMP|/xxxxxxp2/dbdata/temp/temp04.dbf
tmp|TEMP|/xxxxxxp2/dbdata/temp/temp03.dbf
tmp|TEMP|/xxxxxxp2/dbdata/temp/temp02.dbf
tmp|TEMP|/xxxxxxp2/dbdata/temp/temp01.dbf
sys|SYSTEM|/xxxxxxp2/dbdata/data1/system04.dbf
sys|SYSTEM|/xxxxxxp2/dbdata/data1/system03.dbf
sys|SYSTEM|/xxxxxxp2/dbdata/data1/system02.dbf
sys|SYSTEM|/xxxxxxp2/dbdata/data1/system01.dbf
dat2|USERS|/xxxxxxp2/dbdata/data1/users01.dbf
dat2|OWAPUB|/xxxxxxp2/dbdata/data1/owad01.dbf
dat2|APPS_TS_TOOLS|/xxxxxxp2/dbdata/data1/apps_ts_tools01.dbf
dat2|xxxxxxSRX|/xxxxxxp2/dbdata/data1/xxxxxxSrx01.dbf
dat2|xxxxxxSRD|/xxxxxxp2/dbdata/data1/xxxxxxSrd01.dbf
dat2|DISCOVERER|/xxxxxxp2/dbdata/data1/discoverer01.dbf
dat2|APPS_UNDOTS1|/xxxxxxp2/dbdata/data1/undots09.dbf
dat2|APPS_UNDOTS1|/xxxxxxp2/dbdata/data1/undots08.dbf
dat2|APPS_UNDOTS1|/xxxxxxp2/dbdata/data1/undots07.dbf
sys|SYSAUX|/xxxxxxp2/dbdata/data1/sysaux01.dbf
dat2|APPS_TS_TX_DATA|/xxxxxxp2/dbdata/data1/APPS_TS_TX_DATA21.dbf
dat2|APPS_TS_MEDIA|/xxxxxxp2/dbdata/data1/APPS_TS_MEDIA03.dbf

example:#cat adcrdbclone.sql  -->because of this we can able to recreate controlfile
define arg1=&1
spool %s_db_oh%/appsutil/log/%s_contextname%/adcrdb_%s_dbSid%.txt
connect / as sysdba
shutdown abort
connect / as sysdba
startup nomount pfile=%s_db_oh%/dbs/init%s_dbSid%.ora;
CREATE CONTROLFILE REUSE SET DATABASE "%s_dbGlnam%"
LOGFILE
  GROUP 7 ('%s_dbhome2%/log07.dbf') SIZE 1610612736,
  GROUP 8 ('%s_dbhome2%/log08.dbf') SIZE 1610612736,
  GROUP 9 ('%s_dbhome2%/log09.dbf') SIZE 1610612736
DATAFILE
  '%s_dbhome1%/system05.dbf',
  '%s_dbhome1%/system04.dbf',
  '%s_dbhome1%/system03.dbf',
  '%s_dbhome1%/system02.dbf',
  '%s_dbhome1%/system01.dbf',
  '%s_dbhome1%/sysaux01.dbf',
  '%s_dbhome3%/ctxd01.dbf',
  '%s_dbhome4%/users01.dbf',
  '%s_dbhome4%/owad01.dbf',
  '%s_dbhome4%/apps_ts_tools01.dbf',
  '%s_dbhome4%/dresrx01.dbf',
  '%s_dbhome4%/dresrd01.dbf',
  '%s_dbhome4%/dresd02.dbf',
  '%s_dbhome4%/dresd01.dbf',
  '%s_dbhome4%/discoverer01.dbf',
  '%s_dbhome4%/APPS_TS_TX_IDX_LG06.dbf',
  '%s_dbhome4%/APPS_TS_TX_IDX_LG05.dbf',

creation of the database driver file

ORACLE_HOME/appsutil/clone/data/driver/data.drv

$ cat data.drv
#=====================================================================+
#  Copyright (c) 2002 Oracle Corporation Belmont, California, USA     |
#                          All rights reserved.                       |
#=====================================================================+
# FILENAME
#       /xxxxxxp2/oracle/db/11.2.0/appsutil/clone/data/driver/data.drv
# DESCRIPTION
#       Template file driver file for applying a clone using AutoClone
#=====================================================================*/
#
# <src top> <src dir> <src name> <file type> <dst dir> <dst name>
# =========================================================================

if installation-type db
  ad <s_clonestage>/data/stage adcrdb.zip UNZIP <s_db_oh>/appsutil/template
endif


Copy JDBC Libraries at ORACLE_HOME/appsutil/clone/jlib/classes12.jar and appsoui

12.2 clone


12.2 Manual Clone – XXXXXXX01 – Multi Node


Index
Incomplete
Incomplete
Incomplete
Incomplete
Incomplete
Incomplete
Incomplete
Incomplete
Incomplete
Incomplete
Incomplete
Incomplete
Incomplete
Incomplete
Incomplete
Incomplete
Incomplete
Incomplete
Incomplete
Incomplete
Incomplete
Incomplete
Incomplete
Incomplete
Incomplete
Incomplete
Incomplete
Incomplete
Incomplete
Incomplete
Incomplete
Incomplete
Incomplete
Incomplete
Incomplete
Incomplete
Incomplete
Incomplete
Release the Instance to the Business
Incomplete

From DT ORACLE USER:

mkdir -p /p01/XXXXXXX01/csf/preclone/${SID}.${DATEFMT}/DT/db/rdbms
cp $ORACLE_HOME/*.env .
cp $TNS_ADMIN/*.ora .
cp $ORACLE_HOME/appsutil/*.xml .
cp $ORACLE_HOME/dbs/init*.ora .
--- To take backup of DBlinks
sh /p01/XXXXXXX01/custom/scripts/budlink.sh

From APPL USER

mkdir -p /p01/XXXXXXX01/csf/preclone/${SID}.${DATEFMT}/AT/fs1-pre
cp $APPL_TOP/*.env .
cp $APPL_TOP/admin/adovars.env .
cp $APPL_TOP/admin/adpltfrm.txt .
cp $APPL_TOP/admin/topfile.txt .
cp $APPL_TOP/admin/*.xml* .
cp -r $FND_TOP/secure/* .
cp -r $FND_TOP/secure/${SID}/* .
cp $OA_HTML/env.txt .
cp -r $MWA_TOP/secure/* .
cp $ORACLE_HOME/*.env* .
cp $TNS_ADMIN/*.ora* .
$FMW_HOME/webtier/instances .
cp –rp $INST_TOP .
mkdir inst_patch
cd inst_patch
cp –rp $PATCH_BASE/inst .

Record the File Structure

cd /p01/XXXXXXX01/csf/preclone/${SID}.${DATEFMT}
sqlplus '/as sysdba'
spool file_structure.lst
select name from v$database;
archive log list
show parameter pfile
select name from v$controlfile;
select file_name from dba_data_files;
select file_name from dba_temp_files;
select group#,member from v$logfile;
sho parameter undo
select tablespace_name from dba_tablespaces where tablespace_name like '%UNDO%';
alter database backup controlfile to trace as '/p01/XXXXXXX01/csf/preclone/${SID}.${DATEFMT}/ctrltrace.sql';
spool off

rman.sh

rman target / cmdfile='/p01/xxxxxx01/backup01/DB-BKP-26Dec2016/rman.cmd' log='/p01/xxxxxx01/backup01/DB-BKP-26Dec2016/db_backup.log'

rman.cmd

run {
ALLOCATE CHANNEL ch02 TYPE DISK;
ALLOCATE CHANNEL ch03 TYPE DISK;
ALLOCATE CHANNEL ch04 TYPE DISK;
ALLOCATE CHANNEL ch05 TYPE DISK;
ALLOCATE CHANNEL ch06 TYPE DISK;
ALLOCATE CHANNEL ch07 TYPE DISK;
ALLOCATE CHANNEL ch08 TYPE DISK;
ALLOCATE CHANNEL ch09 TYPE DISK;
ALLOCATE CHANNEL ch10 TYPE DISK;
ALLOCATE CHANNEL ch11 TYPE DISK;
BACKUP AS COMPRESSED BACKUPSET
FORMAT '/p01/xxxxxx01/backup01/DB-BKP-26Dec2016/%d_db_u%u_s%s_p%p_t%t_db'
FILESPERSET 10
database plus archivelog ;
RELEASE CHANNEL ch02;
RELEASE CHANNEL ch03;
RELEASE CHANNEL ch04;
RELEASE CHANNEL ch05;
RELEASE CHANNEL ch06;
RELEASE CHANNEL ch07;
RELEASE CHANNEL ch08;
RELEASE CHANNEL ch09;
RELEASE CHANNEL ch10;
RELEASE CHANNEL ch11;
}
sql 'alter system archive log current';
run{
ALLOCATE CHANNEL ch00 TYPE DISK;
BACKUP FORMAT '/p01/xxxxxx01/backup01/DB-BKP-26Dec2016/%d_db_u%u_s%s_p%p_t%t_db' TAG 'xxxxxx01_ctrl' CURRENT CONTROLFILE;
RELEASE CHANNEL ch00;
}
exit

nohup sh rman.sh &


Once Completed  Check for any Errors in log file:

egrep -i 'ERROR|error|ORA' db_restore.log

Give Full Permissions to the Directory:

chomod -R 777 /p01/xxxxxx01/backup01/DB-BKP-26Dec2016

Check if same Patches are applied for Source and Target:


oraXXXXXXX01@qservername104(OPatch)$ opatch lsinv | grep -i applied
Patch  24701882     : applied on Fri Apr 14 02:44:42 EDT 2017
Patch  24701840     : applied on Fri Apr 14 02:34:58 EDT 2017
Patch  21967332     : applied on Fri Apr 14 02:24:44 EDT 2017
Patch  24499346     : applied on Fri Apr 14 02:21:42 EDT 2017
Patch  24502781     : applied on Fri Apr 14 02:17:31 EDT 2017
Patch  24315821     : applied on Fri Apr 14 01:58:11 EDT 2017
Patch  24340671     : applied on Fri Apr 14 01:45:02 EDT 2017
Patch  23477849     : applied on Sun Nov 13 02:25:22 EST 2016
Patch  22731026     : applied on Sun Nov 13 02:22:50 EST 2016
Patch  22496904     : applied on Sun Nov 13 02:19:27 EST 2016
Patch  21904072     : applied on Sun Nov 13 02:11:42 EST 2016
Patch  21321429     : applied on Sun Nov 13 02:07:34 EST 2016
Patch  21864513     : applied on Sun Nov 13 02:03:10 EST 2016
Patch  23057868     : applied on Sun Nov 13 01:41:33 EST 2016
Patch  23614158     : applied on Sun Nov 13 01:15:25 EST 2016
Patch  23286156     : applied on Sun Nov 13 00:38:28 EST 2016
Patch  23054319     : applied on Sun Nov 13 00:29:24 EST 2016
Patch  22037014     : applied on Tue Apr 12 08:32:42 EDT 2016
Patch  21028698     : applied on Tue Apr 12 07:35:44 EDT 2016
Patch  19835133     : applied on Sat Nov 21 18:35:48 EST 2015
Patch  16075609     : applied on Sat Nov 21 18:28:28 EST 2015
Patch  19472320     : applied on Fri Nov 20 09:16:41 EST 2015
Patch  17629476     : applied on Fri Nov 20 09:13:09 EST 2015
Patch  17429475     : applied on Fri Nov 20 09:11:55 EST 2015
Patch  16989137     : applied on Fri Nov 20 09:10:36 EST 2015
Patch  12949905     : applied on Fri Nov 20 09:02:17 EST 2015
Patch  4247037      : applied on Fri Nov 20 09:00:06 EST 2015
Patch  4189542      : applied on Thu Nov 19 17:44:57 EST 2015
Patch  19396455     : applied on Fri Oct 16 18:16:24 EDT 2015
oraXXXXXXX01@qservername104(OPatch)$

If same patches are already present, no need to copy the Oracle Binaries. If they differ follow the following

**** Important Note : While cloning ACA Instance please make sure you are copying the Oracle Binaries from any other ACA Instance where Latest patches are applied. (This is because we have separate list of patches for ACA and Exa)

If you are Copying Oracle Binaries to Target:


cd $ORACLE_HOME/appsutil/scripts/$CONTEXT_NAME
perl adpreclone.pl dbTier
cd $ORACLE_HOME/..
nohup tar -cvf - 11.2.0.4 2> /backup/rman_1060/gpspgjp1/BIN_03june2016/11.2.0.4.log | gzip -c > /backup/rman_1060/gpspgjp1/BIN_03june2016/11.2.0.4.tar.gz &

cd $ADMIN_SCRIPTPS_HOME
perl adpreclone.pl appsTier
cd /p01/xxxxxx01/appl/erpapp/fs1
nohup tar -cvf - EBSapps 2> /p01/XXXXXXX01/appl/EBSapps.log | gzip -c > /p01/XXXXXXX01/appl/EBSapps.tar.gz &





Send Outage Mail in the below Format:






on all WebNodes:

cd $ADMIN_SCRIPTS_HOME
adstpall.sh apps/********

On Primary Node:

cd $ADMIN_SCRIPTS_HOME
adstpall.sh apps/********
cd $NE_BASE/sso
--- In All Web Nodes
./oamreg_linux XXXXXXX01 stage apps_Password  undeployag
--- on All Web Nodes (Internal)
./oamreg_linux XXXXXXX01 stage apps_Password  dereg
---- On all DMZ Nodes
./oamreg_linux XXXXXXX01 stage apps_Password  dmzdereg

cd /p01/XXXXXXX01/appl/erpapp
rm -rf fs1 &
rm -rf fs2 &
rm -rf fs_ne &
rm -rf oraInventory &

create pfile='$ORACLE_HOME/init$ORACLE_SID.ora' from spfile;
alter system set cluster_database=FALSE scope=spfile;
srvctl stop database -d XXXXXXX01
sqlplus '/as sysdba'
startup mount restrict
sho parameter cluster_database
drop database;




On Node1:

cd $ORACLE_HOME/..
rm -rf 11.2.0.4 &

On Node2:

cd $ORACLE_HOME/..
rm -rf 11.2.0.4 &


On First Nodes;

cd $ORACLE_HOME/..
nohup gunzip -c /backup/rman_1060/gpspgjp1/BIN_03june2016/11.2.0.4.tar.gz |tar -xvf - > /gpspgjd1/erpapp/11.2.0.4.log &

On Second Node:

cd $ORACLE_HOME/..
nohup gunzip -c /backup/rman_1060/gpspgjp1/BIN_03june2016/11.2.0.4.tar.gz |tar -xvf - > /gpspgjd1/erpapp/11.2.0.4.log &
cd /p01/XXXXXXX01/appl/erpapp/fs1
nohup gunzip -c /p01/XXXXXXX01/appl/EBSapps.tar.gz |tar -xvf - > /p01/XXXXXXX01/appl/EBSapps.log &


export ORACLE_HOME=/p01/XXXXXXX01/oracle/product/11.2.0.4
cd $ORACLE_HOME/rdbms/lib
ar -t libknlopt.a | grep -c kcsm.o
1                                                                                             

1 means RAC , 0 means Non-RAC

make -f ins_rdbms.mk rac_off
make -f ins_rdbms.mk ioracle
ar -t libknlopt.a | grep -c kcsm.o
0                                             

Copy the Existing xml file to Context File Location: (if Present)

cp XXXXXXX011_qservername103.xml $ORACLE_HOME/appsutil/XXXXXXX011_qservername103.xml

export ORACLE_HOME=/p01/XXXXXXX01/oracle/product/11.2.0.4
export ORACLE_SID=XXXXXXX011
export PATH=$ORACLE_HOME/perl/bin:$PATH
which perl
cd $ORACLE_HOME/appsutil/clone/bin
perl adcfgclone.pl dbTechStack

                     Copyright (c) 2011 Oracle Corporation
                        Redwood Shores, California, USA

                        Oracle E-Business Suite Rapid Clone

                                 Version 12.2

                      adcfgclone Version 120.63.12020000.39

Enter the APPS password :

Running:
/p01/XXXXXXX01/oracle/product/11.2.0.4/appsutil/clone/bin/../jre/bin/java -Xmx600M -cp /p01/XXXXXXX01/oracle/product/11.2.0.4/appsutil/clone/jlib/java:/p01/XXXXXXX01/oracle/product/11.2.0.4/appsutil/clone/jlib/xmlparserv2.jar:/p01/XXXXXXX01/oracle/product/11.2.0.4/appsutil/clone/jlib/ojdbc6.jar:/p01/XXXXXXX01/oracle/product/11.2.0.4/appsutil/clone/jlib/obfuscatepassword.jar:/p01/XXXXXXX01/oracle/product/11.2.0.4/appsutil/clone/jlib/ojmisc.jar:/p01/XXXXXXX01/oracle/product/11.2.0.4/appsutil/clone/jlib/java:/p01/XXXXXXX01/oracle/product/11.2.0.4/appsutil/clone/jlib/emCfg.jar oracle.apps.ad.context.CloneContext -e /p01/XXXXXXX01/oracle/product/11.2.0.4/appsutil/clone/bin/../context/db/CTXORIG.xml -validate -pairsfile /tmp/adpairsfile_67562.lst -stage /p01/XXXXXXX01/oracle/product/11.2.0.4/appsutil/clone -dbTechStack 2> /tmp/adcfgclone_67562.err; echo $? > /tmp/adcfgclone_67562.res

Log file located at /p01/XXXXXXX01/oracle/product/11.2.0.4/appsutil/clone/bin/CloneContext_1019180728.log

Provide the values required for creation of the new Database Context file.

Target System Hostname (virtual or normal) [qservername103] : qservername103
Target Instance is RAC (y/n) [y] : n
Target System Database SID : XXXXXXX01
Target System Base Directory : /p01/XXXXXXX01/oracle
Target System utl_file_dir Directory List : /p01/XXXXXXX01/temp/tmpfiles,/p01/XXXXXXX01/csf,/usr/tmp
Number of DATA_TOP's on the Target System [3] : 1
Target System DATA_TOP Directory 1 : /p01/XXXXXXX01/oracle
Target System RDBMS ORACLE_HOME Directory [/p01/XXXXXXX01/oracle/11.2.0] : /p01/XXXXXXX01/oracle/product/11.2.0.4
Do you want to preserve the Display [pwerpl0581pv10:20] (y/n)  : n
Target System Display [qservername103:0.0] : qservername103:1.0
Do you want the target system to have the same port values as the source system (y/n) [y] ? : n
Target System Port Pool [0-99] : 1

Checking the port pool 1
done: Port Pool 1 is free
.
.
.
addlnctl.sh: exiting with status 0

addlnctl.sh: check the logfile /p01/XXXXXXX01/oracle/product/11.2.0.4/appsutil/log/XXXXXXX01_qservername103/addlnctl.txt for more information ...

export ORACLE_HOME=/p01/XXXXXXX01/oracle/product/11.2.0.4
cd $ORACLE_HOME/rdbms/lib
ar -t libknlopt.a | grep -c kcsm.o
1                                                                                             

 1 means RAC , 0 means Non-RAC

make -f ins_rdbms.mk rac_off
make -f ins_rdbms.mk ioracle
ar -t libknlopt.a | grep -c kcsm.o
0                                             

Copy the xml file from preclone to Context File Location:

cp XXXXXXX012_qservername104.xml $ORACLE_HOME/appsutil/XXXXXXX012_qservername104.xml

export ORACLE_HOME=/p01/XXXXXXX01/oracle/product/11.2.0.4
export PATH=$ORACLE_HOME/perl/bin:$PATH
export ORACLE_SID=XXXXXXX012
which perl
cd $ORACLE_HOME/appsutil/clone/bin
perl adcfgclone.pl dbTechStack

                     Copyright (c) 2011 Oracle Corporation
                        Redwood Shores, California, USA

                        Oracle E-Business Suite Rapid Clone

                                 Version 12.2

                      adcfgclone Version 120.63.12020000.39

Enter the APPS password :

Running:
/p01/XXXXXXX01/oracle/product/11.2.0.4/appsutil/clone/bin/../jre/bin/java -Xmx600M -cp /p01/XXXXXXX01/oracle/product/11.2.0.4/appsutil/clone/jlib/java:/p01/XXXXXXX01/oracle/product/11.2.0.4/appsutil/clone/jlib/xmlparserv2.jar:/p01/XXXXXXX01/oracle/product/11.2.0.4/appsutil/clone/jlib/ojdbc6.jar:/p01/XXXXXXX01/oracle/product/11.2.0.4/appsutil/clone/jlib/obfuscatepassword.jar:/p01/XXXXXXX01/oracle/product/11.2.0.4/appsutil/clone/jlib/ojmisc.jar:/p01/XXXXXXX01/oracle/product/11.2.0.4/appsutil/clone/jlib/java:/p01/XXXXXXX01/oracle/product/11.2.0.4/appsutil/clone/jlib/emCfg.jar oracle.apps.ad.context.CloneContext -e /p01/XXXXXXX01/oracle/product/11.2.0.4/appsutil/clone/bin/../context/db/CTXORIG.xml -validate -pairsfile /tmp/adpairsfile_67562.lst -stage /p01/XXXXXXX01/oracle/product/11.2.0.4/appsutil/clone -dbTechStack 2> /tmp/adcfgclone_67562.err; echo $? > /tmp/adcfgclone_67562.res

Log file located at /p01/XXXXXXX01/oracle/product/11.2.0.4/appsutil/clone/bin/CloneContext_1019180728.log

Provide the values required for creation of the new Database Context file.

Target System Hostname (virtual or normal) [qservername104] : qservername104
Target Instance is RAC (y/n) [y] : n
Target System Database SID : XXXXXXX01
Target System Base Directory : /p01/XXXXXXX01/oracle
Target System utl_file_dir Directory List : /p01/XXXXXXX01/temp/tmpfiles,/p01/XXXXXXX01/csf,/usr/tmp
Number of DATA_TOP's on the Target System [3] : 1
Target System DATA_TOP Directory 1 : /p01/XXXXXXX01/oracle
Target System RDBMS ORACLE_HOME Directory [/p01/XXXXXXX01/oracle/11.2.0] : /p01/XXXXXXX01/oracle/product/11.2.0.4
Do you want to preserve the Display [pwerpl0581pv10:20] (y/n)  : n
Target System Display [qservername104:0.0] : qservername104:1.0
Do you want the target system to have the same port values as the source system (y/n) [y] ? : n
Target System Port Pool [0-99] : 1

Checking the port pool 1
done: Port Pool 1 is free
.
.
.
addlnctl.sh: exiting with status 0

addlnctl.sh: check the logfile /p01/XXXXXXX01/oracle/product/11.2.0.4/appsutil/log/XXXXXXX01_qservername104/addlnctl.txt for more information ...

Change Group & Mode of oracle binary on all nodes:

cd $ORACLE_HOME/bin
ls -ltr oracle
-rwsr-s--x 1 oracle oinstall 240410831 Apr 18 02:15 oracle
chgrp asmadmin oracle
ls -ltr oracle
-rwsr-s--x 1 oracle asmadmin 240410831 Apr 18 02:15 oracle
chmod 6751 oracle

cd $ORACLE_HOME/bin
unlink lbuilder
ln -s $ORACLE_HOME/nls/lbuilder/lbuilder lbuilder
ls -ltr lbuilder

Copy pfile from preclone:
cp init$ORACLE_SID.ora $ORACLE_HOME/dbs
cd $ORACLE_HOME/dbs
grep -i convert init$ORACLE_SID.ora
*.db_file_name_convert='+DATAC1','+DATAC1'
*.log_file_name_convert='+DATAC1','+DATAC1'


Sqlpus ‘/as sysdba’
Create spfile=’+DATAC1/XXXXXXX01/parameterfile/spfileXXXXXXX01.ora’ from pfile=’ $ORACLE_HOME/dbs/init$ORACLE_SID.ora';

SQL> startup nomount
rman.sh
rman auxiliary / cmdfile='/p01/XXXXXXX01/backup01/RMAN_14NOV2016/rman_restore.cmd' log='/p01/XXXXXXX01/backup01/RMAN_14NOV2016/rman_restore.log'

rman_restore.cmd
run {
ALLOCATE auxiliary CHANNEL ch01 TYPE DISK;
ALLOCATE auxiliary CHANNEL ch02 TYPE DISK;
ALLOCATE auxiliary CHANNEL ch03 TYPE DISK;
ALLOCATE auxiliary CHANNEL ch04 TYPE DISK;
ALLOCATE auxiliary CHANNEL ch05 TYPE DISK;
ALLOCATE auxiliary CHANNEL ch06 TYPE DISK;
DUPLICATE DATABASE TO XXXXXXX01
BACKUP LOCATION '/p01/xxxxxx01/backup01/DB_BKP_14NOV16'
NOFILENAMECHECK;
RELEASE CHANNEL ch01;
RELEASE CHANNEL ch02;
RELEASE CHANNEL ch03;
RELEASE CHANNEL ch04;
RELEASE CHANNEL ch05;
RELEASE CHANNEL ch06;
}

nohup sh rman.sh &

FOR PCA Instances:

CREATE RMAN SCRIPT FOR RESTORE AND RECOVERY:


Run the following query in Source to get the output used for preparing Restore script:

set head off;
set lines 900 
select 'SET NEWNAME FOR DATAFILE ' ||FILE_ID|| ' TO ' || '''' || '/p01/xxxxxx04/oradata/data1/' || substr(FILE_NAME,(instr(FILE_NAME,'/',-1,1))+1)|| ''''||';' from dba_data_files where tablespace_name not in ('APPS_TS_TX_IDX','APPS_UNDOTS1','APPS_UNDOTS2','APPS_UNDOTS3','APPS_UNDOTS4','UNDOTBS1','UNDOTBS2','UNDOTBS3','UNDOTBS4')
UNION
select 'SET NEWNAME FOR DATAFILE ' ||FILE_ID|| ' TO ' || '''' || '/p01/xxxxxx04/oradata/undo/' || substr(FILE_NAME,(instr(FILE_NAME,'/',-1,1))+1)|| ''''||';' from dba_data_files where tablespace_name in ('APPS_UNDOTS1','APPS_UNDOTS2','APPS_UNDOTS3','APPS_UNDOTS4','UNDOTBS1','UNDOTBS2','UNDOTBS3','UNDOTBS4')
UNION
select 'SET NEWNAME FOR DATAFILE ' ||FILE_ID|| ' TO ' || '''' || '/p01/xxxxxx04/oradata/index/' || substr(FILE_NAME,(instr(FILE_NAME,'/',-1,1))+1)|| ''''||';' from dba_data_files where tablespace_name in ('APPS_TS_TX_IDX')
UNION
select 'SET NEWNAME FOR TEMPFILE ' ||FILE_ID|| ' TO ' || '''' || '/p01/xxxxxx04/oradata/temp/' || substr(FILE_NAME,(instr(FILE_NAME,'/',-1,1))+1)|| ''''||';' from dba_temp_files;
spool off;

Place the Output below channels allocation in Restore Script:

.
.
.
ALLOCATE auxiliary CHANNEL ch06 TYPE DISK;
ALLOCATE auxiliary CHANNEL ch07 TYPE DISK;
SET NEWNAME FOR DATAFILE 1 TO '/p01/xxxxxx01/oradata/data1/system.1728.916054081';
SET NEWNAME FOR DATAFILE 10 TO '/p01/xxxxxx01/oradata/data1/apps_ts_media.1990.916054957';
SET NEWNAME FOR DATAFILE 100 TO '/p01/xxxxxx01/oradata/data1/gepsbackd.1781.916055643';
SET NEWNAME FOR TEMPFILE 5 TO '/p01/xxxxxx01/oradata/temp/temp.2291.916152947';
SET NEWNAME FOR TEMPFILE 6 TO '/p01/xxxxxx01/oradata/temp/temp.2290.916152947';
SET NEWNAME FOR TEMPFILE 7 TO '/p01/xxxxxx01/oradata/temp/temp.2289.916152947';
DUPLICATE DATABASE TO xxxxxx01
BACKUP LOCATION '/p01/xxxxxx01/backup02/xxxxxx01_clone_25Jan2017/db'
NOFILENAMECHECK;
RELEASE CHANNEL ch01;
RELEASE CHANNEL ch02;
RELEASE CHANNEL ch03;
RELEASE CHANNEL ch04;
RELEASE CHANNEL ch05;
RELEASE CHANNEL ch06;
.
.
.

Edit few undo tablespaces such that they reside in data1 mount point as below (Undo MountPoint Issue)

oraxxxxxx01@pwercd01vn046(rman_restore_26Jan16)$ grep -i undo rman.cmd
SET NEWNAME FOR DATAFILE 220 TO '/p01/xxxxxx01/oradata/undo/undotbs3.2296.916154507';
SET NEWNAME FOR DATAFILE 221 TO '/p01/xxxxxx01/oradata/data1/undotbs4.2297.916154563';
SET NEWNAME FOR DATAFILE 222 TO '/p01/xxxxxx01/oradata/undo/undotbs2.2314.916356275';
SET NEWNAME FOR DATAFILE 229 TO '/p01/xxxxxx01/oradata/data1/undotbs4.2425.922017983';
SET NEWNAME FOR DATAFILE 230 TO '/p01/xxxxxx01/oradata/undo/undotbs1.2426.922028839';
SET NEWNAME FOR DATAFILE 232 TO '/p01/xxxxxx01/oradata/data1/undotbs4.2574.929794643';
SET NEWNAME FOR DATAFILE 233 TO '/p01/xxxxxx01/oradata/data1/undotbs3.2575.929794785';
SET NEWNAME FOR DATAFILE 234 TO '/p01/xxxxxx01/oradata/data1/undotbs3.2576.929794799';
SET NEWNAME FOR DATAFILE 235 TO '/p01/xxxxxx01/oradata/undo/undotbs2.2577.929794929';
SET NEWNAME FOR DATAFILE 236 TO '/p01/xxxxxx01/oradata/data1/undotbs1.2585.930155543';
SET NEWNAME FOR DATAFILE 242 TO '/p01/xxxxxx01/oradata/undo/undotbs3.2696.933030533';
SET NEWNAME FOR DATAFILE 3 TO '/p01/xxxxxx01/oradata/data1/undotbs1.1722.916054081';
SET NEWNAME FOR DATAFILE 5 TO '/p01/xxxxxx01/oradata/data1/undotbs2.1809.916054081';
oraxxxxxx01@pwercd01vn046(rman_restore_26Jan16)$



 

Check the following once Database is up:

show parameter pfile
SELECT * FROM V$RECOVER_FILE;
select name,user,open_mode from v$database;
archive log list
select name from v$controlfile;
select file_name from dba_data_files;
select file_name from dba_temp_files;
select group#,member from v$logfile;
select node_name from apps.fnd_nodes;
sho parameter undo
select tablespace_name from dba_tablespaces where tablespace_name like '%UNDO%';
select file_name,tablespace_name,BYTES/1024/1024 from dba_data_files where tablespace_name like '%UNDO%';
select file_name, tablespace_name, bytes/1024/1024 from dba_temp_files;
show parameter db_name
show parameter db_unique_name
--- DB Name DB Unique name should be in Small case, if not change in pfile.ora file and bounce DB.
show parameter service
alter system set service_names='ebs_patch,XXXXXXX01' scope=both;
select NAME from dba_services;
update dba_services set NAME='XXXXXXX01' where NAME='XXXXXXX01';
--- if Undo/Temp are not in Target Mount Point do the following:
create temporary tablespace TEMPDUP tempfile '+DATAC1' size 200m;
alter database default temporary tablespace TEMPDUP;
drop tablespace TEMP including contents and datafiles;
create temporary tablespace TEMP tempfile '+DATAC1' size 200m;


startup
sho parameter undo
sho parameter pfile
select instance_name from gv$instance
select tablespace_name from dba_tablespaces where tablespace_name like '%UNDO%';
alter system set cluster_database=TRUE scope=spfile;
alter system set instance_number=1 SID='XXXXXXX011' scope=spfile;
alter system set thread=1 SID='XXXXXXX011' scope=spfile;
alter system set instance_number=2 SID='XXXXXXX012' scope=spfile;
alter system set thread=2 SID='XXXXXXX012' scope=spfile;
alter system set undo_tablespace='APPS_UNDOTS11' SID='XXXXXXX011' scope=spfile;
alter system set undo_tablespace='APPS_UNDOTS12' SID='XXXXXXX012' scope=spfile;
select GROUP#,THREAD#,MEMBERS,STATUS  from v$log;
alter database enable public thread 2;
alter database enable public thread 3;
shut immediate

Convert Non-RAC to RAC

cd $ORACLE_HOME/rdbms/lib
ar -t libknlopt.a | grep -c kcsm.o
0
make -f ins_rdbms.mk rac_on
make -f ins_rdbms.mk ioracle
ar -t libknlopt.a | grep -c kcsm.o
1

Edit the pfile with Location of spfile:

Node1:

cd $ORACLE_HOME/dbs
cat init$ORACLE_SID.ora
spfile='+DATAC1/XXXXXXX01/parameterfile/spfileXXXXXXX01.ora'

Node2:

cd $ORACLE_HOME/dbs
cat init$ORACLE_SID.ora
spfile='+DATAC1/XXXXXXX01/parameterfile/spfileXXXXXXX01.ora'

Start using srvctl:

srvctl start database -d XXXXXXX01

UPDATE PRODUCT_USER_PROFILE SET CHAR_VALUE='ENABLED' WHERE CHAR_VALUE='DISABLED';
COMMIT;
update apps.fnd_user set email_address = null;
update apps.per_all_people_f set email_address = null;
update apps.po_vendor_sites_all set fax = null, fax_area_code = null, attribute12 = null;
update apps.PO_VENDOR_SITES_ALL  set attribute11 = NULL where attribute11 is not NULL;
UPDATE apps.po_vendor_sites_all SET email_address = NULL WHERE email_address IS NOT NULL;
commit;
Truncate table applsys.fnd_concurrent_processes;
Truncate table applsys.fnd_concurrent_requests;
update applsys.FND_CONCURRENT_QUEUES set node_name=null, target_node=null where concurrent_queue_name like 'FNDICM';
update apps.icx_parameters set session_cookie_domain=NULL;
update applsys.FND_CONCURRENT_QUEUE_SIZE set MIN_PROCESSES=5 where concurrent_queue_id in (SELECT a.CONCURRENT_QUEUE_id FROM applsys.fnd_concurrent_queues a, applsys.FND_CONCURRENT_QUEUE_SIZE b WHERE  a.CONCURRENT_QUEUE_id=b.CONCURRENT_QUEUE_id and b.min_processes > 40);
update applsys.fnd_cp_services a set developer_parameters= (select b.developer_parameters||':-mx64m' from applsys.fnd_cp_services b where a.service_id=b.service_id and b.developer_parameters like 'J%'and a.service_handle not like 'FNDOPP%') where a.developer_parameters like 'J%' and developer_parameters not like '%-mx64m%' and a.service_handle not like 'FNDOPP%';
update applsys.wf_notifications set status='CLOSED' where status='OPEN';
update applsys.wf_notifications set MAIL_Status='SENT'  where mail_status='MAIL';
update apps.alr_alerts set enabled_flag = 'N' where alert_name like 'GEPS%';
update hr.per_all_people_f set email_address = null;
UPDATE ASO.ASO_QUOTE_HEADERS_ALL SET ATTRIBUTE4 = NULL;
update applsys.wf_local_roles set email_address = null;
commit;

Clean up fnd_nodes:

conn apps/*****
exec fnd_conc_clone.setup_clean;
EXEC FND_CONC_CLONE.TARGET_CLEAN;

Run AutoConfig on Both DB Nodes in the Order : 1 - 2 - 1


Make the value of s_update_scan as TRUE before Running Autoconfig

grep -i s_update_scan $CONTEXT_FILE
         <scanUpdateFlag oa_var="s_update_scan">TRUE</scanUpdateFlag>

cd $ORACLE_HOME/appsutil/bin
adconfig.sh

Repeat the Same in Second Node:


grep -i s_update_scan $CONTEXT_FILE
         <scanUpdateFlag oa_var="s_update_scan">TRUE</scanUpdateFlag>

cd $ORACLE_HOME/appsutil/bin
adconfig.sh

Check connectivity on both the node :


sqlplus apps/****@XXXXXXX01
sqlplus apps/****@XXXXXXX011
sqlplus apps/****@XXXXXXX012

On 1st Node:
SQL> conn apps/****@XXXXXXX01
Connected.
SQL> conn apps/****@XXXXXXX011
Connected.
SQL> conn apps/****@XXXXXXX012
Connected.

On 2nd node:
SQL> conn apps/****@XXXXXXX01
Connected.
SQL> conn apps/****@XXXXXXX011
Connected.
SQL> conn apps/****@XXXXXXX012
Connected.

Connect to Primary Application Node:

cd /p01/XXXXXXX01/appl/erpapp
mkdir fs1

Verify the Inventory location and group of oraInst.loc in /etc and 10.1.2_Home

appXXXXXXX01@servername116(erpapp)$ cat /etc/oraInst.loc
inventory_loc=/p01/XXXXXXX01/appl/erpapp/oraInventory
inst_group=oaaXXXXXXX01

appXXXXXXX01@servername116(erpapp)$ cat /p01/XXXXXXX01/appl/erpapp/fs1/EBSapps/10.1.2/oraInst.loc
inventory_loc=/p01/XXXXXXX01/appl/erpapp/oraInventory
inst_group=oaaXXXXXXX01

Make sure env file .bash_profile is deleted/notsourced


cd /p01/XXXXXXX01/appl/erpapp/fs1/EBSapps/comn/clone/bin
perl adcfgclone.pl appsTier dualfs

                     Copyright (c) 2002, 2015 Oracle Corporation
                        Redwood Shores, California, USA
                        Oracle E-Business Suite Rapid Clone
                                 Version 12.2
                      adcfgclone Version 120.63.12020000.56

Enter the APPS password :
Enter the Weblogic AdminServer password :
Do you want to add a node (yes/no) [no] : no
Running: Context clone...
Log file located at /p01/XXXXXXX01/appl/erpapp/fs1/EBSapps/comn/clone/bin/CloneContext_0424230009.log

Provide the values required for creation of the new APPL_TOP Context file.
Target System Hostname (virtual or normal) [servername116] :
Target System Database SID : XXXXXXX01
Target System Database Server Node [servername116] : perald41-scan
Target System Database Domain Name [domain.com] :
Target System Base Directory : /p01/XXXXXXX01/appl/erpapp
Target System Base Directory set to /p01/XXXXXXX01/appl/erpapp
Target System Current File System Base set to /p01/XXXXXXX01/appl/erpapp/fs1
Target System Other File System Base set to /p01/XXXXXXX01/appl/erpapp/fs2
Target System Fusion Middleware Home set to /p01/XXXXXXX01/appl/erpapp/fs1/FMW_Home
Target System Other File System Fusion Middleware Home set to /p01/XXXXXXX01/appl/erpapp/fs2/FMW_Home
Target System Web Oracle Home set to /p01/XXXXXXX01/appl/erpapp/fs1/FMW_Home/webtier
Target System Other File System Web Oracle Home set to /p01/XXXXXXX01/appl/erpapp/fs2/FMW_Home/webtier
Target System Appl TOP set to /p01/XXXXXXX01/appl/erpapp/fs1/EBSapps/appl
Target System Other File System Appl TOP set to /p01/XXXXXXX01/appl/erpapp/fs2/EBSapps/appl
Target System COMMON TOP set to /p01/XXXXXXX01/appl/erpapp/fs1/EBSapps/comn
Target System Other File System COMMON TOP set to /p01/XXXXXXX01/appl/erpapp/fs2/EBSapps/comn
Target System Instance Home Directory [/p01/XXXXXXX01/appl/erpapp] :
Target System Current File System Instance Top set to /p01/XXXXXXX01/appl/erpapp/fs1/inst/apps/XXXXXXX01_servername116
Do you want to preserve the Display [pwerlp01vn37:1.0] (y/n)  : n
Target System Display [servername116:0.0] : servername116:1.0       
Target System Root Service [enabled] :
Target System Web Entry Point Services [enabled] :
Target System Web Application Services [enabled] :
Target System Batch Processing Services [enabled] :
Target System Other Services [enabled] :
Do you want the target system to have the same port values as the source system (y/n) [y] ? : n
Target System Port Pool [0-99] : 1
Checking the port pool 1
done: Port Pool 1 is free
Report file located at /p01/XXXXXXX01/appl/erpapp/fs1/inst/apps/XXXXXXX01_servername116/admin/out/portpool.lst

UTL_FILE_DIR on database tier consists of the following directories.
1. /p01/XXXXXXX01/temp/tmpfiles
2. /p01/XXXXXXX01/custom/pa/out

Choose a value which will be set as APPLPTMP value on the target node [1] :
The new APPL_TOP context file has been created :
  /p01/XXXXXXX01/appl/erpapp/fs1/inst/apps/XXXXXXX01_servername116/appl/admin/XXXXXXX01_servername116.xml
Check Clone Context logfile /p01/XXXXXXX01/appl/erpapp/fs1/EBSapps/comn/clone/bin/CloneContext_0424230009.log for details.

Creating Patch file system context file.....
Log file located at /p01/XXXXXXX01/appl/erpapp/fs1/EBSapps/comn/clone/bin/CloneContextPatch_0424230116.log
Target System Other File System Instance Top set to /p01/XXXXXXX01/appl/erpapp/fs2/inst/apps/XXXXXXX01_servername116
Target System Port Pool [0-99] : 3
Checking the port pool 3
done: Port Pool 3 is free
Report file located at /p01/XXXXXXX01/appl/erpapp/fs2/inst/apps/XXXXXXX01_servername116/admin/out/portpool.lst

The new APPL_TOP context file has been created :
  /p01/XXXXXXX01/appl/erpapp/fs2/inst/apps/XXXXXXX01_servername116/appl/admin/XXXXXXX01_servername116.xml
Check Clone Context logfile /p01/XXXXXXX01/appl/erpapp/fs1/EBSapps/comn/clone/bin/CloneContextPatch_0424230116.log for details.

FMW Pre-requisite check log file location : /p01/XXXXXXX01/appl/erpapp/fs1/EBSapps/comn/clone/FMW/logs/prereqcheck.log
Running: FMW pre-req check...

Configuring: Run file system....
LogFile located at /p01/XXXXXXX01/appl/erpapp/fs1/inst/apps/XXXXXXX01_servername116/admin/log/clone/run/RCloneApplyAppstier_04242301.log
<Apr 25, 2017 12:02:07 AM EDT> <Warning> <JNDI> <BEA-050001> <WLContext.close() was called in a different thread than the one in which it was created.>
<Apr 25, 2017 12:02:07 AM EDT> <Warning> <JNDI> <BEA-050001> <WLContext.close() was called in a different thread than the one in which it was created.>
<Apr 25, 2017 12:02:07 AM EDT> <Warning> <JNDI> <BEA-050001> <WLContext.close() was called in a different thread than the one in which it was created.>

Configuring: Patch file system....
LogFile located at /p01/XXXXXXX01/appl/erpapp/fs1/inst/apps/XXXXXXX01_servername116/admin/log/clone/patch/RCloneApplyAppstier_04250002.log
<Apr 25, 2017 1:46:58 AM EDT> <Warning> <JNDI> <BEA-050001> <WLContext.close() was called in a different thread than the one in which it was created.>
<Apr 25, 2017 1:46:58 AM EDT> <Warning> <JNDI> <BEA-050001> <WLContext.close() was called in a different thread than the one in which it was created.>
<Apr 25, 2017 1:46:58 AM EDT> <Warning> <JNDI> <BEA-050001> <WLContext.close() was called in a different thread than the one in which it was created.>
<Apr 25, 2017 1:46:58 AM EDT> <Warning> <JNDI> <BEA-050001> <WLContext.close() was called in a different thread than the one in which it was created.>

Do you want to startup the Application Services for XXXXXXX01? (y/n) [n] : n
Services not started








Start Admin Server in Run and Patch Filesystems:

. /p01/XXXXXXX01/appl/erpapp/EBSapps.env run
cd $ADMIN_SCRIPTS_HOME
adadminsrvctl.sh start

. /p01/XXXXXXX01/appl/erpapp/EBSapps.env patch
cd $ADMIN_SCRIPTS_HOME
adadminsrvctl.sh start forcepatchfs

. /p01/XXXXXXX01/appl/erpapp/EBSapps.env run
perl $AD_TOP/patch/115/bin/adProvisionEBS.pl ebs-delete-managedserver -contextfile=$CONTEXT_FILE -managedsrvname=oacore_server5 -servicetype=oacore -managedsrvport=7203 -logfile=$EBS_DOMAIN_HOME/servers/oacore_server5/logs/oacore_server5.log

. /p01/XXXXXXX01/appl/erpapp/EBSapps.env patch
perl $AD_TOP/patch/115/bin/adProvisionEBS.pl ebs-delete-managedserver -contextfile=$CONTEXT_FILE -managedsrvname=oacore_server5 -servicetype=oacore -managedsrvport=7205 -logfile=$EBS_DOMAIN_HOME/servers/oacore_server5/logs/oacore_server5.log

Login to Run/Patch Weblogic Console and delete oaea_servers

Run:

http://servername116.domain.com:7002/console

Patch:

http://servername116.domain.com:7004/console

Make Nessessary Changes in the Run/Patch Context-Files: (Refer from Pre-Clone)


Below are some of the Important Parameters to be changed:

egrep -i 's_javamailer_imaphost|s_javamailer_imapdomainname|s_javamailer_reply_to|s_javamailer_imap_user' $CONTEXT_FILE

egrep -i 's_webentryurlprotocol|s_webentryhost|s_webentrydomain|s_active_webport|s_login_page|s_external_url|s_endUserMonitoringURL' $CONTEXT_FILE

egrep -i 's_dbport|s_dbhost|s_applcsf|s_appltmp|s_applptmp|s_apps_jdbc_connect_descriptor' $CONTEXT_FILE

egrep -i 's_admin_ui_access_nodes|s_oacore_nodes|s_forms_nodes|s_oafm_nodes|s_forms-c4ws_nodes|s_oaea_nodes' $CONTEXT_FILE

egrep -i 's_appldcp|s_concstatus|s_cp_twotask|s_isConc|s_isConcDev|s_isAdmin' $CONTEXT_FILE

appXXXXXXX01@servername116(admin)$ egrep -i 's_javamailer_imaphost|s_javamailer_imapdomainname|s_javamailer_reply_to|s_javamailer_imap_user' $CONTEXT_FILE
         <hostname oa_var="s_javamailer_imaphost" customized="yes">mail</hostname>
         <domain oa_var="s_javamailer_imapdomainname" customized="yes">ad.domain.com</domain>
         <username oa_var="s_javamailer_reply_to" customized="yes">mail.XXXXXXX01@domain.com</username>
         <username oa_var="s_javamailer_imap_user" customized="yes">mail.XXXXXXX01@domain.com</username>


appXXXXXXX01@servername116(admin)$ egrep -i 's_webentryurlprotocol|s_webentryhost|s_webentrydomain|s_active_webport|s_login_page|s_external_url|s_endUserMonitoringURL' $CONTEXT_FILE
         <EndUserMonitoringURL oa_var="s_endUserMonitoringURL">https://servername1.domain.com/oracle_smp_chronos/oracle_smp_chronos_sdk.gif</EndUserMonitoringURL>
         <externURL oa_var="s_external_url">https://servername1.domain.com</externURL>
         <webentryhost oa_var="s_webentryhost">servername1</webentryhost>
         <webentrydomain oa_var="s_webentrydomain">domain.com</webentrydomain>
         <login_page oa_var="s_login_page">https://servername1.domain.com/OA_HTML/AppsLogin</login_page>
         <webentryurlprotocol oa_var="s_webentryurlprotocol">https</webentryurlprotocol>
      <activewebport oa_var="s_active_webport" oa_type="DUP_PORT" base="8000" step="1" range="-1" label="Active Web Port">443</activewebport>

                 
                 
appXXXXXXX01@servername116(admin)$ egrep -i 's_dbport|s_dbhost|s_applcsf|s_appltmp|s_applptmp|s_apps_jdbc_connect_descriptor' $CONTEXT_FILE
         <dbhost oa_var="s_dbhost">perald41-scan</dbhost>
         <jdbc_url oa_var="s_apps_jdbc_connect_descriptor">jdbc:oracle:thin:@(DESCRIPTION=(ADDRESS_LIST=(LOAD_BALANCE=YES)(FAILOVER=YES)(ADDRESS=(PROTOCOL=tcp)(HOST=perald41-scan)(PORT=1521)))(CONNECT_DATA=(SERVICE_NAME=XXXXXXX01)))</jdbc_url>
         <APPLCSF oa_var="s_applcsf">/p01/XXXXXXX01/csf</APPLCSF>
         <APPLTMP oa_var="s_appltmp">/p01/XXXXXXX01/temp/tmpfiles</APPLTMP>
         <APPLPTMP oa_var="s_applptmp" osd="UNIX">/p01/XXXXXXX01/temp/tmpfiles</APPLPTMP>
      <dbport oa_var="s_dbport" oa_type="EXT_PORT" base="1521" step="1" range="-1" label="Database Port">1521</dbport>

                 
appXXXXXXX01@servername116(admin)$ egrep -i 's_admin_ui_access_nodes|s_oacore_nodes|s_forms_nodes|s_oafm_nodes|s_forms-c4ws_nodes|s_oaea_nodes' $CONTEXT_FILE
                        <admin_ui_access_nodes oa_var="s_admin_ui_access_nodes">localhost</admin_ui_access_nodes>
                        <oacore_nodes oa_var="s_oacore_nodes">servername116.domain.com:7202,servername117.domain.com:7202</oacore_nodes>
                        <forms_nodes oa_var="s_forms_nodes">servername116.domain.com:7402,servername117.domain.com:7402</forms_nodes>
                        <oafm_nodes oa_var="s_oafm_nodes">servername116.domain.com:7602,servername117.domain.com:7602</oafm_nodes>
                        <forms-c4ws_nodes oa_var="s_forms-c4ws_nodes">servername116.domain.com:7802,servername117.domain.com:7802</forms-c4ws_nodes>
                        <oaea_nodes oa_var="s_oaea_nodes">servername116.domain.com:6802,servername117.domain.com:6802</oaea_nodes>

                                 
appXXXXXXX01@servername116(admin)$ egrep -i 's_appldcp|s_concstatus|s_cp_twotask|s_isConc|s_isConcDev|s_isAdmin' $CONTEXT_FILE
         <TIER_ADMIN oa_var="s_isAdmin">YES</TIER_ADMIN>
         <TIER_NODE oa_var="s_isConc">YES</TIER_NODE>
         <TIER_NODEDEV oa_var="s_isConcDev">YES</TIER_NODEDEV>
         <CP_TWOTASK oa_var="s_cp_twotask">XXXXXXX01</CP_TWOTASK>
         <APPLDCP oa_var="s_appldcp">ON</APPLDCP>
            <oa_service_status oa_var="s_concstatus">enabled</oa_service_status>

Once Completed Run Autoconfig on Run/Patch:

. /p01/XXXXXXX01/appl/erpapp/EBSapps.env run
cd $ADMIN_SCRIPTS_HOME
adautocfg.sh

Disable the Trigger: (From DB Node)

sqlplus '/as sysdba'
alter trigger SYSTEM.EBS_LOGON DISABLE;
. /p01/XXXXXXX01/appl/erpapp/EBSapps.env patch
cd $ADMIN_SCRIPTS_HOME
adautocfg.sh

Once completed. Enabled the Trigger

alter trigger SYSTEM.EBS_LOGON ENABLE;

Start the Services in Run File System. Do Complete Sanity before proceeding to Further Steps


Run Pre-Clone in Run FileSystem:

cd $COMMON_TOP
mv clone clone_bkp
cd $ADMIN_SCRIPTS_HOME
perl adpreclone.pl appsTier

·         mkdir -p /p01/XXXXXXX01/appl/erpapp/pair_files/26Apl2017
·         cp $INST_TOP/appl/admin/XXXXXXX01_servername116_run.txt /p01/XXXXXXX01/appl/erpapp/pair_files/26Apl2017/XXXXXXX01_servername116_run.txt
·         cd /p01/XXXXXXX01/appl/erpapp/pair_files/26Apl2017
·         cp XXXXXXX01_servername116_run.txt XXXXXXX01_servername117_run.txt

Edit the Newly Created Pair File with the following changes:

vi XXXXXXX01_servername117_run.txt

[Instance Specific]

s_temp=/p01/XXXXXXX01/appl/erpapp/fs1/inst/apps/XXXXXXX01_servername117/temp
s_contextname=XXXXXXX01_servername117
s_hostname=servername117
s_domainname=domain.com
s_cphost=servername116
s_webhost=servername117
s_config_home=/p01/XXXXXXX01/appl/erpapp/fs1/inst/apps/XXXXXXX01_servername117
s_inst_base=/p01/XXXXXXX01/appl/erpapp
s_display=servername117:1.0
s_forms-c4ws_display=servername117:0.0
s_ohs_instance=EBS_web_XXXXXXX01_OHS2
s_webport=8001
s_http_listen_parameter=8001
s_https_listen_parameter=4444

[Services To be Enabled on the Secondary Application Tier Node]

s_web_applications_status=enabled
s_web_entry_status=enabled
s_apcstatus=enabled
s_root_status=enabled
s_batch_status=enabled
s_other_service_group_status=enabled
s_adminserverstatus=disabled
s_web_admin_status=disabled

Add the Entry:
patch_s_port_pool=3

Make sure env is not Sourced
echo $TWO_TASK

cd /p01/XXXXXXX01/appl/erpapp/fs1/EBSapps/comn/clone/bin
./adclonectx.pl addnode contextfile=/p01/XXXXXXX01/appl/erpapp/fs1/inst/apps/XXXXXXX01_servername116/appl/admin/XXXXXXX01_servername116.xml pairsfile=/p01/XXXXXXX01/appl/erpapp/pair_files/26Apl2017/XXXXXXX01_servername117_run.txt outfile=/p01/XXXXXXX01/appl/erpapp/fs1/inst/apps/XXXXXXX01_servername117/appl/admin/XXXXXXX01_servername117.xml  dualfs=yes

Follow similar steps in Rest of the Application Nodes


Once completed. Make nessasary changes in the Context file of both Run/Patch Filesystem in All Nodes followed by Autoconfig on all Nodes.

For Run:

. /p01/XXXXXXX01/appl/erpapp/EBSapps.env run
cd $ADMIN_SCRIPTS_HOME
adadminsrvctl.sh start

egrep -i 's_javamailer_imaphost|s_javamailer_imapdomainname|s_javamailer_reply_to|s_javamailer_imap_user' $CONTEXT_FILE
egrep -i 's_webentryurlprotocol|s_webentryhost|s_webentrydomain|s_active_webport|s_login_page|s_external_url|s_endUserMonitoringURL' $CONTEXT_FILE
egrep -i 's_dbport|s_dbhost|s_applcsf|s_appltmp|s_applptmp|s_apps_jdbc_connect_descriptor' $CONTEXT_FILE
egrep -i 's_admin_ui_access_nodes|s_oacore_nodes|s_forms_nodes|s_oafm_nodes|s_forms-c4ws_nodes|s_oaea_nodes' $CONTEXT_FILE
egrep -i 's_appldcp|s_concstatus|s_cp_twotask|s_isConc|s_isConcDev|s_isAdmin' $CONTEXT_FILE

For Patch:

. /p01/XXXXXXX01/appl/erpapp/EBSapps.env patch
cd $ADMIN_SCRIPTS_HOME
adadminsrvctl.sh start forcepatchfs

egrep -i 's_javamailer_imaphost|s_javamailer_imapdomainname|s_javamailer_reply_to|s_javamailer_imap_user' $CONTEXT_FILE
egrep -i 's_webentryurlprotocol|s_webentryhost|s_webentrydomain|s_active_webport|s_login_page|s_external_url|s_endUserMonitoringURL' $CONTEXT_FILE
egrep -i 's_dbport|s_dbhost|s_applcsf|s_appltmp|s_applptmp|s_apps_jdbc_connect_descriptor' $CONTEXT_FILE
egrep -i 's_admin_ui_access_nodes|s_oacore_nodes|s_forms_nodes|s_oafm_nodes|s_forms-c4ws_nodes|s_oaea_nodes' $CONTEXT_FILE
egrep -i 's_appldcp|s_concstatus|s_cp_twotask|s_isConc|s_isConcDev|s_isAdmin' $CONTEXT_FILE

Start Services in all the Application Nodes and Perform Full Sanity before proceeding with further steps


col directory_path  for a100
set pages 1000 lines 150
select directory_name,directory_path from dba_directories;

update dir$ set OS_PATH=replace(OS_PATH,'&OLD_VALUE','&NEW_VALUE') where OS_PATH like '%&OLD_VALUE%';

And verifiy once Directories are Upadated….Need to check Physical exixtence of Path Also.

Start Admin Server of Run and Patch Filesystems:

For Run:

. /p01/XXXXXXX01/appl/erpapp/EBSapps.env run
cd $ADMIN_SCRIPTS_HOME
adadminsrvctl.sh start

For Patch:

. /p01/XXXXXXX01/appl/erpapp/EBSapps.env patch
cd $ADMIN_SCRIPTS_HOME
adadminsrvctl.sh start forcepatchfs
All Passwords are placed in /p01/XXXXXXX01/appl/erpapp/.postclone Location

Change Passwords of Sys and System

sqlplus '/as sysdba'
alter user sys identified by manager;
alter user system identified by manager;

FNDCPASS apps/exaoq01nucl3r 0 Y system/manager SYSTEM APPLSYS exaoq01nucl3r

Execute the following predefined scripts

exec_FNDCPASS_custom.sh                        --- For Custom User Password change
exec_FNDCPASS_std.sh                                --- For Standard User Password change
exec_ALTER_cust.sql                                      --- For Custom Schema Password change (from db user)

Login to Weblogic Console:

Run:

http://servername116.domain.com:7002/console

Follow the below Steps to change apps password in Weblogic Console:

Click Lock & Edit in Change Center
In the Domain Structure tree, expand Services, then select Data Sources
On the “Summary of JDBC Data Sources” page, select EBSDataSource
On the “Settings for EBSDataSource” page, select the Connection Pool tab.
Enter the new password in the “Password” field.
Enter the new password in the “Confirm Password” field.
Click Save.
Click Activate Changes in Change Center.
Then go to OAEADatasource
In the Domain Structure tree, expand Services, then select Data Sources.
On the “Summary of JDBC Data Sources” page, select OAEADatasource
On the “Settings for EBSDataSource” page, select the Connection Pool tab.
Enter the new password in the “Password” field.
Enter the new password in the “Confirm Password” field.
Click Save.
Click Activate Changes in Change Center.

Follow the Same for Patch Filesystem:

http://servername116.domain.com:7004/console

postclone_std.sql                                            --- Post Clone Script
instance_spec_prof.sql                                 --- Post Clone Script

update applsys.fnd_profile_option_values set profile_option_value='GE NUCLEAR EXA Instance - XXXXXXX01 - Cloned @ '||sysdate where profile_option_id='125';

SELECT value FROM XDO_CONFIG_VALUES WHERE property_code = 'SYSTEM_TEMP_DIR';
VALUE
--------------------------------------------------------------------------------
/p01/XXXXXXX01/temp/tmpfiles

update XDO_CONFIG_VALUES set value='/p01/servername01/temp/tmpfiles' WHERE  property_code = 'SYSTEM_TEMP_DIR';
SELECT NAME ,DISPLAY_NAME FROM WF_SYSTEMS;
NAME                                                   DISPLAY_NAME
--------------------------------------------------------------------------------
XXXXXXX01.DOMAIN.COM          XXXXXXX01.DOMAIN.COM

update WF_SYSTEMS set DISPLAY_NAME='XXXXXX01.DOMAIN.COM' where DISPLAY_NAME='XXXXXX01.DOMAIN.COM';

We need to update profiles to Target which are pointing to Source

Query to Check:

select a.profile_option_name, b.profile_option_value, a.profile_option_id, a.application_id
from fnd_profile_options a, fnd_profile_option_values b
where a.application_id = b.application_id
and a.profile_option_id = b.profile_option_id    
and b.profile_option_value like '%&profile%';

Check with sid,SID,hostname and bigip

Query to Update:

update apps.fnd_profile_option_values set profile_option_value=replace(profile_option_value,'&OLD_VALUE','&NEW_VALUE')
where profile_option_value like '%&OLD_VALUE%';

cd $APPL_TOP/geps

find . -type l -exec ls -ls '{}' \; |grep -i "fndcpesr" |awk '{print "ls -ltr "$(NF-2)}'
– to check links
find . -type l -exec ls -ls '{}' \; |grep -i "fndcpesr" |awk '{print "ln -s $FND_TOP/bin/fndcpesr "$(NF-2)}
- to make links
find . -type l -exec ls -ls '{}' \; |grep -i "fndcpesr" |awk '{print "rm -f "$(NF-2)}'
– to remove the links

Once completed, give full permissions to geps directory:


chmod -R 777 $APPL_TOP/geps

To check the existing DB-Links:

set pages 2000
set lines 200
col OWNER for a10
col DB_LINK for a35
col USERNAME for a20
col HOST for a40
select OWNER, DB_LINK, USERNAME, HOST, CREATED from dba_db_links order by created desc;

To drop the DB-Links:

drop database link APPS_DOMAIN.COM;

Re-Create the DB-Links using:

CREATE DATABASE LINK JS1_TO_JO1.DOMAIN.COM CONNECT TO GEPSREADONLY IDENTIFIED BY gepsreadonly123 USING '(DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=pwerpl0596dv01.cloud.domain.com)(PORT=1522))(CONNECT_DATA=(SERVICE_NAME=gpspgjo1)(INSTANCE_NAME=gpspgjo1)))';

Validate:

select name from v$database@JS1_TO_JO1.DOMAIN.COM;

mkdir $NE_BASE/sso
cd $NE_BASE/sso
wget --no-check-certificate https://oamsso.com/downloads/webgate.zip
wget --no-check-certificate https://oamsso.com/downloads/oamreg_linux.zip
unzip webgate.zip -d webgate
unzip oamreg_linux.zip

./oamreg_linux XXXXXXX01 stage <apps Password>  undeployag --- on servername116
./oamreg_linux XXXXXXX01 stage <apps Password>  dereg -- on servername116 and servername117
./oamreg_linux XXXXXXX01 stage <apps Password>  dmzdereg -- on servername118  and servername119

./oamreg_linux XXXXXXX01 stage <apps Password>  deployag node1 -- -- on servername116
./oamreg_linux XXXXXXX01 stage <apps Password>  deployag node2 -- -- on servername117
./oamreg_linux XXXXXXX01 stage <apps Password>  deployag node3 -- -- on servername118
./oamreg_linux XXXXXXX01 stage <apps Password>  deployag node4 -- -- on servername119
./oamreg_linux XXXXXXX01 stage <apps Password>  reg -- on servername116 and servername117
./oamreg_linux XXXXXXX01 stage <apps Password>  dmzreg -- on servername118  and servername119

Once completed run autoconfig on Primary Node



Edit the mod_wl_ohs.conf File as per the Instance:


cd $FMW_HOME/webtier/instances/EBS_web_XXXXXXX01_OHS1/config/OHS/EBS_web_XXXXXXX01
cp  mod_wl_ohs.conf mod_wl_ohs.conf-original
vi mod_wl_ohs.conf
Give the entires of both the nodes in all the Components:

Example:

Before

        <Location /forms>
                SetHandler weblogic-handler
                WLCookieName JsessionIDForms
                WebLogicCluster servername116.domain.com:7402
                WLTempDir ${ORACLE_INSTANCE}/tmp
        </Location>

After

        <Location /forms>
                SetHandler weblogic-handler
                WLCookieName JsessionIDForms
                WebLogicCluster servername116.domain.com:7402,servername117.domain.com:7402
                WLTempDir ${ORACLE_INSTANCE}/tmp
        </Location>


Login as Primary Node and run the below scripts : -


sqlplus apps/********

SQL> @$FND_TOP/patch/115/sql/txkChangeProfH.sql SERVRESP
Changing the hierarchy type for the  Profile APPS_WEB_AGENT
Profile APPS_WEB_AGENT hierarchy type has been
successfully changed to SERVRESP
Changing the hierarchy type for the  Profile APPS_SERVLET_AGENT
Profile APPS_SERVLET_AGENT hierarchy type has been
successfully changed to SERVRESP
Changing the hierarchy type for the  Profile APPS_JSP_AGENT
Profile APPS_JSP_AGENT hierarchy type has been
successfully changed to SERVRESP
Changing the hierarchy type for the  Profile APPS_FRAMEWORK_AGENT
Profile APPS_FRAMEWORK_AGENT hierarchy type has been
successfully changed to SERVRESP
Changing the hierarchy type for the  Profile ICX_FORMS_LAUNCHER
Profile ICX_FORMS_LAUNCHER hierarchy type has been
successfully changed to SERVRESP
Changing the hierarchy type for the  Profile ICX_DISCOVERER_LAUNCHER
Profile ICX_DISCOVERER_LAUNCHER hierarchy type has been
successfully changed to SERVRESP
Changing the hierarchy type for the  Profile ICX_DISCOVERER_VIEWER_LAUNCHER
Profile ICX_DISCOVERER_VIEWER_LAUNCHER hierarchy type has been
successfully changed to SERVRESP
Changing the hierarchy type for the  Profile HELP_WEB_AGENT
Profile HELP_WEB_AGENT hierarchy type has been
successfully changed to SERVRESP
Changing the hierarchy type for the  Profile APPS_PORTAL
Profile APPS_PORTAL hierarchy type has been
successfully changed to SERVRESP
Changing the hierarchy type for the  Profile CZ_UIMGR_URL
Profile CZ_UIMGR_URL hierarchy type has been
successfully changed to SERVRESP
Changing the hierarchy type for the  Profile QP_PRICING_ENGINE_URL
Profile QP_PRICING_ENGINE_URL hierarchy type has been
successfully changed to SERVRESP
Changing the hierarchy type for the  Profile TCF:HOST
Profile TCF:HOST hierarchy type has been
successfully changed to SERVRESP
Disconnected from Oracle Database 11g Enterprise Edition Release 11.2.0.4.0 - 64bit Production
With the Partitioning, Real Application Clusters, Automatic Storage Management, OLAP,
Data Mining and Real Application Testing options
appXXXXXXX01@servername116(geps)$

Update Node Trust Level

To change the value of the Node Trust Level profile option value to External for a particular node, perform the following steps:


·         Login to Oracle E-Business Suite as sysadmin user using the internal URL
·         Select the System Administrator Responsibility
·         Select Profile / System
·         From the 'Find system profile option Values' window, select the server that you want to designate as the external web tier
·         Query for %NODE%TRUST%. You will see a profile option named 'Node Trust Level'. The value for this profile option at the site level will be Normal. Leave this setting unchanged.
·         Set the value of this profile option to External at the server level. The site level value should remain set to Normal





Record the output of:

select node_name,server_id from fnd_nodes where support_web='Y';

Perform the below:

cd $FMW_HOME/Oracle_EBS-app1/deployment_plans/accessgate
cp plan.xml plan.xml.original
rm plan.xml

Go to each application tier node and do this

Login as applmgr
mkdir $HOME/.sso
cd $FMW_HOME/Oracle_EBS-app1/deployment_plans/accessgate
cp plan.xml.original $HOME/.sso/plan.xml
ln -s $HOME/.sso/plan.xml plan.xml
vi $HOME/.sso/plan.xml

Search for string APPL_SERVER_ID

      <name>app_APPL_SERVER_ID</name>
      <value>081877EC50246AE0E053823D2F03371136087014321141155826551352395400</value>

Replace the value of APPL_SERVER_ID with the server_id value from fnd_nodes


From servername116:

cp customXXXXXXX01_servername116.env $INST_TOP/appl/admin

From servername117:

cp customXXXXXXX01_servername117.env $INST_TOP/appl/admin

Run autoconfig on extranet and then on intranet


Run autoconfig on extranet tier 1
Run autoconfig on extranet tier 2
Run autoconfig on intranet tier 2
Run autoconfig on intranet tier 1
Start all services on intranet tier and Extranet.


cd $INST_TOP/appl/admin
ls -ltr *.env
. ./customXXXXXXX01_servername116.env

cd $INST_TOP/admin/install
grep -i mwa.telnetportnumber mwa.cfg
mwa.TelnetPortNumber=10206,10208,10210

cd $INST_TOP/admin/scripts
ls -ltr mwactl.sh
./mwactl.sh start 10206
./mwactl.sh start 10208
./mwactl.sh start 10210

ps -fu $LOGNAME|grep -i mwa