You can use Oracle Fleet Patching and Provisioning to provision and maintain your Oracle Grid Infrastructure homes.

Oracle Fleet Patching and Provisioning enables mass deployment and maintenance of standard operating environments for databases, clusters, and user-defined software types.

Oracle FPP enables you to install clusters, and provision, patch, scale, and upgrade Oracle Grid Infrastructure, Oracle Restart, and Oracle Database homes. The supported releases are 11.2.0.4, 12.1, 12.2, 18c, and later releases. You can also provision applications and middleware using Oracle Fleet Patching and Provisioning.

Oracle Fleet Patching and Provisioning is a service in Oracle Grid Infrastructure that you can use in either of the following modes:

• Central Oracle Fleet Patching and Provisioning Server

  The Oracle Fleet Patching and Provisioning Server stores and manages standardized images, called gold images. Gold images can be deployed to any number of nodes across the data center. You can create new clusters and databases on the deployed homes and can use them to patch, upgrade, and scale existing installations.

  The Oracle Fleet Patching and Provisioning Server can manage the following types of installations:

  • Software homes on the cluster hosting the Oracle Fleet Patching and Provisioning Server itself.
  • Installations running Oracle Grid Infrastructure 11g Release 2 (11.2.0.4) and later releases.
  • Oracle Fleet Patching and Provisioning Clients running Oracle Grid Infrastructure 12c Release 2 (12.2) and later releases.
  • Installations running without Oracle Grid Infrastructure.

  The Oracle Fleet Patching and Provisioning Server can provision new installations, and manage existing installations, without requiring any changes to the existing installations. The Oracle Fleet Patching and Provisioning Server can automatically share gold images among peer servers to support enterprises with geographically distributed data centers.

• Oracle Fleet Patching and Provisioning Client

  The Oracle Fleet Patching and Provisioning Client can be managed from the Oracle Fleet Patching and Provisioning Server, or directly by executing commands on the client itself. The Oracle Fleet Patching and Provisioning Client is a service built into the Oracle Grid Infrastructure and is available in Oracle Grid Infrastructure 12c Release 2 (12.2) and later releases. The Oracle Fleet Patching and Provisioning Client can retrieve gold images from the Oracle Fleet Patching and Provisioning Server, upload new images based on the policy, and apply maintenance operations to itself.

Fleet Patching and Provisioning has several methods to provision and, optionally, configure Oracle Grid Infrastructure and Oracle Restart grid infrastructure homes.

• To inform Fleet Patching and Provisioning the nodes on which to install Oracle Grid Infrastructure, and to configure Oracle Grid Infrastructure, you provide directions in a response file, as in the following example:

  $ rhpctl add workingcopy -workingcopy GI_HOME_11204_WCPY -image GI_HOME_11204 –responsefile /u01/app/rhpinfo/GI_11204_install.txt
    {authentication_option}

  The preceding command provisions the GI_HOME_11204_WCPY working copy based on the GI_HOME_11204 gold image to a target specified in the GI_11204_install.txt response file. In addition to identifying the target nodes, the response file specifies information about the Oracle Grid Infrastructure configuration, such as Oracle ASM and GNS parameters.

  Note:

  The oracle.install.crs.rootconfig.executeRootScript=xxx response file parameter is overridden and always set to false for Fleet Patching and Provisioning, regardless of what you specify in the response file.
• To provision an Oracle Grid Infrastructure home to a node or cluster that is currently running Oracle Grid Infrastructure:

  $ rhpctl add workingcopy -workingcopy GI_HOME_12201_PATCHED_WCPY -image GI_HOME_12201_PSU1 –client CLUST_002 -softwareonly

  The preceding command provisions a new working copy based on the GI_HOME_12201_PSU1 gold image to the Fleet Patching and Provisioning Client (that is running Oracle Grid Infrastructure 12c release 2 (12.2)) named CLUST_002. When you provision to a target that is not running Oracle Grid Infrastructure 12c release 2 (12.2) (such as, a target running Oracle Grid Infrastructure 12c release 1 (12.1) or Oracle Grid Infrastructure 11g release 2 (11.2)), use the -targetnode parameter instead of -client.

• Specify a target node on which you want to provision an Oracle Restart grid infrastructure, as follows:

  $ rhpctl add workingcopy -workingcopy SIHA_GI -image goldimage -targetnode remote_node_name -responsefile Oracle_Restart_response_file {authentication_option}

Fleet Patching and Provisioning provides three methods to patch Oracle Grid Infrastructure software homes: rolling, non-rolling, and in batches.

When you install Oracle Grid Infrastructure or when you upgrade an older version to this current version, the Fleet Patching and Provisioning Server is configured automatically in local mode.

$ srvctl enable rhpserver
$ srvctl start rhpserver

$ srvctl stop rhpserver
$ srvctl remove rhpserver

• The independent automaton for patching Oracle Grid Infrastructure performs all of the steps necessary to switch from one home to another. Because the automaton is not aware of gold images, moving the database requires two home paths, as follows:

  $ rhpctl move gihome –sourcehome Oracle_home_path -destinationhome Oracle_home_path

Fleet Patching and Provisioning has error prevention and automated recovery options to assist you during maintenance operations.

During maintenance operations, errors must be avoided whenever possible and, when they occur, you must have automated recovery paths to avoid service disruption.

If you are using Fleet Patching and Provisioning, then you can use a single command to upgrade an Oracle Grid Infrastructure home.

$ rhpctl upgrade gihome -sourcewc GIOH11204 -destwc GIOH12201

Use the rhpctl add workingcopy command to provision a working copy of a database home on a Fleet Patching and Provisioning Server, Client, or target.

• Run the rhpctl add workingcopy command on a Fleet Patching and Provisioning Server, similar to the following example:

  $ rhpctl add workingcopy -image db12c -path /u01/app/dbusr/product/12.2.0/db12201
    -client client_007 -oraclebase /u01/app/dbusr/ -workingcopy wc_db122_1

  The preceding command example creates a working copy named wc_db122_1 on all nodes of the Fleet Patching and Provisioning Client cluster named client_007. The gold image db12c is the source of the workingcopy. The directory path locations that you specify in the command must be empty.

Create an Oracle Database on a working copy.

• After you create a working copy of a gold image and provision that working copy to a target, you can create an Oracle Database on the working copy using the rhpctl add database command, similar to the following command example, which creates an Oracle Real Application Clusters (Oracle RAC) database called db12201 on a working copy called wc_db122_1:

  $ rhpctl add database –workingcopy wc_db122_1 –dbname db12201 -node client_007_node1,client_007_node2 -dbtype RAC -datafileDestination DATA007_DG

To patch an Oracle database, you move the database home to a new home, which includes the patches you want to implement.

1. Create a working copy of the Oracle Database that you want to patch, in this case DB122.
2. Apply the patch to the working copy you created.
3. Test and validate the patched working copy.
4. Use the rhpctl add image command to create a gold image (for example, DB122_PATCH) from the patched working copy.

   Note:

   The working copy you specify in the preceding command must be hosted on the Fleet Patching and Provisioning Server in Fleet Patching and Provisioning-managed storage.
5. Delete the patched working copy with the patched Oracle Database using the rhpctl delete workingcopy command.

   Note:

   Do not remove directly using the rm command or some other method, because this does not update the Fleet Patching and Provisioning inventory information.
6. Create a working copy from the patched gold image, (DB122_PATCH).
7. Move myorcldb to the working copy you just created.
8. When you are confident that you will not need to roll back to the working copy on which the database was running at the beginning of the procedure, delete that working copy using the rhpctl delete workingcopy command.

The independent local-mode automaton updates Oracle Database homes, including Oracle Database single-instance databases in a cluster or standalone (with no Oracle Grid Infrastructure), an Oracle RAC database, or an Oracle RAC One Node database.

• The independent automaton for Oracle Database patching performs all of the steps necessary to switch from one home to another. Because the automaton is not aware of gold images, moving the database requires two home paths, as follows:

  $ rhpctl move database -sourcehome Oracle_home_path -desthome destination_oracle_home_path

In addition to Oracle Grid Infrastructure and Oracle Database homes, Oracle Fleet Patching and Provisioning supports patching the Oracle Exadata components: database nodes, storage cells, and InfiniBand switches.

1. Import an image of the EXAPATCHSOFTWARE image type, using the rhpctl import image command on an Oracle Fleet Patching and Provisioning Server, similar to the following:

   $ rhpctl import image -image EXA1 -imagetype EXAPATCHSOFTWARE -path /tmp/ExadataPatchBundle
     -version 12.1.2.2.3.160720

   Note:

   • You can only run the rhpctl import image command on an Oracle Fleet Patching and Provisioning Server.

   • You must rename the patch zip files to include the words storage, database and, iso, so that Oracle Fleet Patching and Provisioning can distinguish among them.

2. If this is the first time you are using Oracle Fleet Patching and Provisioning to patch Oracle Exadata, then use the rhpctl add workingcopy command, as follows:

   $ rhpctl add workingcopy -image image_name -root {[-dbnodes dbnode_list]
     [-cells cell_list] [-ibswitches ibswitch_list]} [-fromnode node_name]
     [-unkey] [-smtpfrom "address"] [-smtpto "addresses"] [-precheckonly]
     [-modifyatprereq] [-resetforce] [-force]

   The preceding command stores the list of nodes (database nodes, cells, and InfiniBand switches) and the version of each node in Oracle Fleet Patching and Provisioning on the working copy, in addition to the type of node and the type of image.

3. For subsequent patching operations, after you create a gold image with updated database, storage cell, and switch files, run the rhpctl update workingcopy command to patch one, two, or all three Oracle Exadata components, as follows:

   $ rhpctl update workingcopy -image image_name -root {[-dbnodes dbnode_list] [-cells cell_list]
     [-ibswitches ibswitch_list]} [-fromnode node_name] [-unkey] [-smtpfrom "address"]
     [-smtpto "addresses"] [-precheckonly] [-modifyatprereq] [-resetforce] [-force]

   Note:

   The name of the working copy remains the same throughout the life cycle of patching the given Oracle Exadata target.

   You can choose to patch only the database nodes, cells, or InfiniBand switches or any combination of the three. Patching occurs in the following order: InfiniBand switches, cells, database nodes.

4. Display the list of nodes and their images, as follows:

   $ rhpctl query workingcopy -workingcopy working_copy_name

5. Delete the working copy, as follows:

   rhpctl delete workingcopy -workingcopy working_copy_name

Fleet Patching and Provisioning provides two options for upgrading Oracle Database. Both options are performed with a single command.

Run a zero-downtime upgrade using Oracle GoldenGate for replication.

• You can run the upgrade command on the Fleet Patching and Provisioning Server to upgrade a database hosted on the server, an Oracle Database 12c release 1 (12.1.0.2) target cluster, or a database hosted on a Fleet Patching and Provisioning Client 12c release 2 (12.2.0.1) or 18c. You can also run the command a Fleet Patching and Provisioning Client 18c to upgrade a database hosted on the client, itself.
  Use the upgrade command similar to the following:

  $ rhpctl zdtupgrade database -dbname sierra -destwc DB_Wcopy_121 -ggsrcwc
    GG_Wcopy_11g -ggdstwc GG_Wcopy_12c -targetnode 12102_cluster_node -root

  In the preceding command, 12102_cluster_node refers to the Oracle Grid Infrastructure 12c release 1 (12.1.0.2) cluster hosting the database you want to upgrade.

Run a zero-downtime upgrade using Oracle Data Guard for replication.

• The following example of a zero-downtime upgrade using Oracle Data Guard upgrades an Oracle Database 11g release 2 (11.2.0.4), sierra, running on the target cluster, which includes a node, targetclust003, to an Oracle Database 12c release 1 (12.1.0.2) (the destination working copy, which was provisioned from a Gold Image stored on the Fleet Patching and Provisioning Server named rhps.example.com):

  $ rhpctl zdtupgrade database -dbname sierra -destwc WC121DB4344 -clonedatadg DBDATA -targetnode node90743 -root
  
  Enter user "root" password:
  node90753.example.com: starting zero downtime upgrade operation ...
  node90753.example.com: verifying patches applied to Oracle homes ...
  node90753.example.com: verifying if database "sierra" can be upgraded with zero downtime ...
  node90743: 15:09:10.459: Verifying whether database "sierra" can be cloned ...
  node90743: 15:09:10.462: Verifying that database "sierra" is a primary database ...
  node90743: 15:09:14.672: Verifying that connections can be created to database "sierra" ...
  < ... >
  node90743: 15:14:58.015: Starting redo apply ...
  node90743: 15:15:07.133: Configuring primary database "sierra" ...
  ####################################################################
  node90753.example.com: retrieving information about database "xmvotkvd" ...
  node90753.example.com: creating services for snapshot database "xmvotkvd" ...
  ####################################################################
  node90743: 15:15:33.640: Macro step 1: Getting information and validating setup ...
  < ... >
  node90743: 15:16:02.844: Macro step 2: Backing up user environment in case upgrade is aborted  ...
  node90743: 15:16:02.848: Stopping media recovery for database "xmvotkvd" ...
  node90743: 15:16:05.858: Creating initial restore point "NZDRU_0000_0001" ...
  < ... >
  node90743: 15:16:17.611: Macro step 3: Creating transient logical standby from existing physical standby ...
  node90743: 15:16:18.719: Stopping instance "xmvotkvd2" of database "xmvotkvd" ...
  node90743: 15:16:43.187: Verifying that database "sierra" is a primary database ...
  < ... >
  node90743: 15:19:27.158: Macro step 4: Upgrading transient logical standby database ...
  node90743: 15:20:27.272: Disabling service "sierrasvc" of database "xmvotkvd" ...
  node90743: 16:36:54.684: Macro step 5: Validating upgraded transient logical standby database ...
  node90743: 16:37:09.576: Creating checkpoint "NZDRU_0301" for database "xmvotkvd" during stage "3" and task "1" ...
  node90743: 16:37:09.579: Stopping media recovery for database "xmvotkvd" ...
  node90743: 16:37:10.792: Creating restore point "NZDRU_0301" for database "xmvotkvd" ...
  node90743: 16:37:11.998: Macro step 6: Switching role of transient logical standby database ...
  node90743: 16:37:12.002: Verifying that database "sierra" is a primary database ...
  < ... >
  node90743: 16:39:07.425: Macro step 7: Flashback former primary database to pre-upgrade restore point and convert to physical standby ...
  node90743: 16:39:08.833: Stopping instance "sierra2" of database "sierra" ...
  < ... >
  node90743: 16:41:17.138: Macro step 8: Recovering former primary database ...
  node90743: 16:41:19.045: Verifying that database "sierra" is mounted ...
  < ... >
  node90743: 17:20:21.378: Macro step 9: Switching back ...
  < ... >
  ####################################################################
  node90753.example.com: deleting snapshot database "xmvotkvd" ...

You can customize zero-downtime upgrades using the user-action framework of Fleet Patching and Provisioning.

• To register a plugin to be run during a zero-downtime upgrade, run the following command:

  $ rhpctl add useraction -useraction user_action_name -actionscript script_name
    {-pre | -post} -optype {ZDTUPGRADE_DATABASE | ZDTUPGRADE_DATABASE_SNAPDB |
     ZDTUPGRADE_DATABASE_DBUA | ZDTUPGRADE_DATABASE_SWITCHBACK}

  You can specify run-time input to the plugins using the -useractiondata option of the rhpctl zdtupgrade database command.

Provision Oracle Grid Infrastructure software on two nodes that do not currently have a Grid home, and then configure Oracle Grid Infrastructure to form a multi-node Oracle Grid Infrastructure installation.

• From the Fleet Patching and Provisioning Server, run the command:

  $ rhpctl add workingcopy -workingcopy GI19c -image GI_HOME_19c –responsefile /u01/app/rhpinfo/GI_19c_install.rsp {authentication_option}

  GI19c is the working copy based on the image GI_HOME_19c

  /u01/app/rhpinfo/GI_19c_install.rsp is the response file location.

  Cluster Verification Utility checks for preinstallation configuration as per requirements. Fleet Patching and Provisioning configures Oracle Grid Infrastructure.

This procedure provisions Oracle Database 19c software and creates Oracle Database instances.

1. From the Fleet Patching and Provisioning Server, provision the Oracle Database home software:

   $ rhpctl add workingcopy -image db19c -path /u01/app/dbusr/product/19.0.0/db19c
     -client client_001 -oraclebase /u01/app/dbusr/ -workingcopy db19wc 

   The command provisions the working copy db19wc to the specified path on the cluster client_001, from the image db19c.
2. Create the database instance:

   $ rhpctl add database -workingcopy db19wc -dbname db -dbtype RAC

   The command creates an Oracle RAC database instance db . You can use the add database command repeatedly to create more instances on the working copy.

You can provision a pluggable database (PDB) on an existing container database (CDB) running in a provisioned database working copy.

• The following command example creates a PDB called pdb19c on a CDB called raccdb19c, which is on a working copy called wc_db19c:

  $ rhpctl addpdb database -workingcopy wc_db19c -cdbname raccdb19c -pdbname pdb19c

• Use the rhpctl deletepdb database command to delete a PDB from an existing CDB on a working copy.
  The following command example deletes a PDB called pdb19c on a CDB called raccdb19c, which is on a working copy called wc_db19c:

  $ rhpctl deletepdb database -workingcopy wc_db19c -cdbname raccdb19c -pdbname pdb19c

This procedure uses Fleet Patching and Provisioning to upgrade your Oracle Grid Infrastructure cluster from 18c to 19c.

1. Provision a working copy of the Oracle Grid Infrastructure 19c software:

   $ rhpctl add workingcopy -workingcopy GI9c -image GI_HOME_19c {authentication_option}

   GI19c is the working copy based on the image GI_HOME_19c.

2. Upgrade your target cluster to the GI19c working copy:

   rhpctl upgrade gihome -sourcewc GI18c -destwc GI19c

   Rapid Home Provisioning identifies the cluster to upgrade based on the name of the source working copy, and upgrades to the working copy GI19c.

This procedure explains how to patch Oracle Grid Infrastructure without changing the Grid home path.

• To move a non-OLFS-compliant Grid home to an OLFS Grid home, from the Fleet Patching and Provisioning Server, run two commands similar to the following:

  $ rhpctl add workingcopy -workingcopy GI_HOME_19c_PSU1 -aupath
    /u01/app/orabase/product/19.1.0/aupath -image image_name
    -client client_name -softwareonly

  $ rhpctl move gihome -srcwc GI_HOME_19c -destwc GI_HOME_19c_PSU1 -agpath
    /u01/app/orabase/product/19.1.0/agpath -path
    /u01/app/orabase/product/19.1.0/grid

• To move an OLFS-compliant Grid home to a patched version, run two commands similar to the following:

  $ rhpctl add workingcopy -workingcopy gihome19c_psu1_lpm -aupath
    /u01/app/orabase/product/19.1.0/aupath -image image_name -client
    client_name -softwareonly

  $ rhpctl move gihome -sourcewc gihome19c_lpm -destwc gihome19c_psu1_lpm

This procedure patches Oracle Grid Infrastructure and Oracle Databases on the cluster to the latest patch level without cluster downtime.

1. Prepare target Oracle homes as follows:
   1. Provision software-only Grid home on the cluster to be patched:

      $ rhpctl add workingcopy -workingcopy GI_HOME_19c_PATCHED_WCPY 
        -image GI_HOME_19c_PSU1 –client CLUSTER_005 -softwareonly

   2. Provision each release Database home, without database instances, to be patched:

      $ rhpctl add workingcopy -workingcopy DB_HOME_19c_PATCHED_WCPY 
        -image DB_HOME_19c_PSU1
      $ rhpctl add workingcopy -workingcopy DB_HOME_18c_PATCHED_WCPY 
        -image DB_HOME_18c_PSU5

2. Patch Oracle Grid Infrastructure and all Oracle RAC Databases on node1 as follows:

   $ rhpctl move gihome -sourcewc GI_HOME_19c_WCPY -destwc GI_HOME_19c_PATCHED_WCPY -auto
     -dbhomes DB_HOME_18c_WCPY=DB_HOME_18c_PATCHED_WCPY,DB_HOME_19c_WCPY=DB_HOME_19c_PATCHED_WCPY  -targetnode node1 {authentication_option}

   When you run the command, you move your active Oracle Grid Infrastructure from working copy GI_HOME_19c_WCPY to GI_HOME_19c_PATCHED_WCPY, Oracle RAC Database db1 from DB_HOME_19c_WCPY to DB_HOME_19c_PATCHED_WCPY, and Oracle RAC Database db2 from DB_HOME_18c_WCPY to DB_HOME_18c_PATCHED_WCPY.

This procedure explains how to patch Oracle Database 19c with the latest patching without bringing down the database.

1. To patch an Oracle Database home managed by Fleet Patching and Provisioning, and there exist working copies of the source and destination databases, run:

   rhpctl move database -sourcewc db19c -patchedwc db19c_psu

   db19c is the source working copy of the database being patched

   db19c_psu is the working copy of the Oracle Database software with patches applied, based on the image DB19c_PSU.

2. To patch an unmanaged Oracle Database home (source working copy does not exist because the Oracle home is not managed by Fleet Patching and Provisioning), run:

   rhpctl move database -sourcehome /u01/app/orabase/product/19.0.0/dbhome_1
    -patchedwc db19c_psu -targetnode node1

   targetnode specifies the node on which the database to be upgraded is running, because the source Oracle Database is on a 19c cluster.

   /u01/app/orabase/product/19.0.0/dbhome_1 is the path of the database being patched

   db19c_psu is the working copy of the Oracle Database software with patches applied, based on the image DB19c_PSU.

   Use the saved gold image for standardized patching of all your databases of release 19c to the same patch level.
3. If for some reason, you want to rollback the patches applied to a managed Oracle Database home, run:

   rhpctl move database -sourcewc db19c_psu 
   -patchedwc db19c -ignorewcpatches

   db19c is the working copy of the unpatched database to which you want to roll back.

   db19c_psu is the working copy of the Oracle Database software with patches applied, based on the image DB19c_PSU.

This procedure describes how to upgrade an Oracle database from Oracle Database 18c to 19c with a single command, using Fleet Patching and Provisioning, both for managed and unmanaged Oracle homes.

1. From the Fleet Patching and Provisioning Server, run one of the following commands as per your source and destination database:
   1. To upgrade an Oracle home managed by Fleet Patching and Provisioning, and there exist working copies of the source and destination databases, run:

      $ rhpctl upgrade database -dbname test_database -sourcewc db18c -destwc db19c
        {authentication_option}

      test_database is the name of the database being upgraded.

      db18c is the source working copy of the pre-upgrade database.

      db19c is the working copy of the upgraded Oracle Database software.

   2. To upgrade an unmanaged Oracle home (source working copy does not exist because the Oracle home is not managed by Fleet Patching and Provisioning), run:

      $ rhpctl move database -sourcehome /u01/app/orabase/product/18.0.0/dbhome_1
        -destwc db19c -targetnode node1 {authentication_option}

      /u01/app/orabase/product/18.0.0/dbhome_1 is the path of the database being upgraded.

      db19c is the working copy of the upgraded Oracle Database software.

      targetnode specifies the node on which the database to be upgraded is running, because the source Oracle Database is on a 18c cluster.

You can add a node to your two-node cluster by using Fleet Patching and Provisioning to add the node, and then extend an Oracle RAC database to the new node.

1. From the Fleet Patching and Provisioning Server, run the following command to add a node to the existing Oracle Grid Infrastructure working copy:

   rhpctl addnode gihome -workingcopy GI_HOME_19c_WCPY -newnodes n3:n3-vip {authentication_option}

   The command extends the cluster by adding node3.
2. Add instances to the administrator-managed Oracle RAC database on the new node:

   rhpctl addnode database -workingcopy DB_HOME_19c_WCPY -dbname db321 -node n3 {authentication_option}

   The command extends the database home on the node3 and creates database db321 on this node.

Create gold images of software home and store them on the Fleet Patching and Provisioning Server, to use later to provision Oracle homes.

1. Import an image from an installed Oracle home on the Fleet Patching and Provisioning Server:

   rhpctl import image -image db19c -path /share/software/19c/dbhome -imagetype ORACLEDBSOFTWARE 

   The gold image of imagetype Oracle Database 19c software is created and stored on the Fleet Patching and Provisioning Server.

   You can also create gold images of Oracle Grid Infrastructure or any other software by specifying -imagetype as ORACLEGISOFTWARE, ORACLEGGSOFTWARE, or SOFTWARE respectively.

2. Import an image from an installed Oracle home on a Fleet Patching and Provisioning Client by running the following command from the Fleet Patching and Provisioning Client:

   rhpctl import image -image db19c -path /u01/app/dbusr/product/19.0.0/

   The command creates and adds the image db19c based on the local Oracle home installed in the specified path.

You can use Fleet Patching and Provisioning user actions to perform many tasks, such as installing and configuring any type of software and running scripts.