Disaster Recovery, Security, and High Availability

Follow

Revision: 2.7
Date: Jan 9, 2015

Question

What are the disaster recovery steps for [Enterprise] Random Password Manager (E/RPM) and how can the solution be made highly available?”

Disaster Recovery Preparation

As in any application a good backup strategy for your system and its system state information is imperative and should be implemented for all systems. While this holds true for E/RPM, it is important to understand that some components of E/RPM are much more important than others.

E/RPM is divided into three components:

  • Management Application (or Console), including Deferred Processor & Encryption Key
  • Database
  • Website



The database is the single most important component to E/RPM. All systems lists, jobs, job settings, password, and other information about the systems are stored here in the database.

If you have implemented a backup solution for your system which includes all system state information and the website, database, and management application are all installed on the same system, this will be sufficient for recovery. If the database, management application, and website are on separate systems then you will need to be aware how the individual components of E/RPM work.

The E/RPM database must be part of the normal backup regiment. As E/RPM uses Microsoft SQL or Oracle for the database, you will use programs and APIs for database backup that are native to Microsoft or Oracle; Lieberman Software does not provide a backup mechanism to manage Microsoft SQL or Oracle database backups.

Generally, backup the program’s database at least as often as scheduled password change jobs run. It is recommended to perform nightly full/complete/normal backups of the production database. The database is comparably small – typically no more than a couple hundred megabytes but can be gigabytes depending on the scenario.

It is highly recommended to turn on encryption for the stored passwords in the database. If password encryption for the password information in the database is enabled, it will be necessary to archive the encryption key that E/RPM uses. The encryption settings are located under SETTINGS | ENCRYPTION SETTINGS within the management application. From the Encryption Settings dialog, choose to export the encryption key. This encryption key only needs to be exported as often as the encryption key is changed.

Disaster Recovery

Once the database is restored, E/RPM can be reattached to the database. If reinstalling E/RPM, the installer will prompt for the E/RPM database – simply choose the same database. If you had enabled encryption previously, you will need to re-import the encryption key. This can be done from the SETTINGS | ENCRYPTION SETTINGS within the management application.
The website functions independently of the management application. This means a failure of the management application will not render the passwords inaccessible. If the website should become unavailable for any reason and restoration is not possible, simply re-deploy the website from the management application. Refer to the E/RPM installation guide for specific steps. All settings will remain in-tact and all delegations will still remain in-tact.

Ultimately, the backup of the program database and encryption key is all that is required for E/RPM to be restored to any system. Without both of these items, it will not be possible to gain access to the random passwords stored in the database.

As of ERPM 4.83.8, licensing is stored in the database rather than locally. So when multiple consoles are present, but at least one console experiences total host system disaster, it may be preferred to properly license the any one of the other hosts or install a new console.

If the original host is lost and a new host must be brought online to support management console functionality, a new license will be required for the new host if the name is different. If a key has already been supplied to you, go to HELP | REGISTER to input the key. If a key has not been supplied to you then contact your account manager for a replacement key (replacement meaning the original licensed console will not be restored to a functional state). The original encryption key will be required for this installation to maintain access to the original database for password retrieval and management purposes.


E/RPM is agnostic of the database or database server it connects to. This means you are free to move the database to any system at any time, though if the name of the database server changes, the management application and websites will need to be redirected to the new database. This can be done by changing the database options from SETTINGS | DATABASE CONFIGURATION for the management console, and then updating the website connection settings by going to SETTINGS | MANAGE WEB APPLICATION | MANAGE WEB APPLICATION INSTANCES and choosing to update the instance with current options.

High Availability


As in any application a good backup strategy for your system and its system state information is imperative and should be implemented for all systems. High availability is more than just a good backup strategy; it is ensuring that the services which provide the data are available to you with the least downtime and as little interruption to service as possible. This means that items like mirroring and clustering need to be addressed.

The Database

The database is the single most important component to E/RPM. All systems lists, jobs, job settings, password, and other information about the systems are stored here in the database.

For high availability when using Microsoft SQL Server, options for high availability are Database Mirroring and Clustering. For high availability for Oracle, use Mirroring (Active Data Guard) or clustering (RAC). Mirroring is cheaper than clustering but requires more work in a Disaster Recovery scenario. For steps on how to configure mirroring or clustering, see the Microsoft or Oracle documentation associated with your database.

Using mirroring, if the database fails a secondary server with the same information is readily available. In this scenario, redirect E/RPM and its website(s) to the mirrored database. To do this, go to SETTINGS | DATASTORE CONFIGURATION and input the new database server name. Some companies further this process by having a monitor examine the health of the database servers. If the master mirror fails, the DNS records are automatically redirected to the secondary mirror. This process ensures no program reconfiguration is required.

Using clustering, if the active node of the database fails, the secondary server will take over automatically and there will be no discernible interruption to service and there will not be a need to reconfigure E/RPM or the website(s) in any way shape or form.

The Website

The website works independently of the management application. This means that even if the management application crashes, the website will still be able to function and serve requested passwords. To avoid loss of this functionality, Lieberman Software recommends the use of Network Load Balancing (NLB) for the website. NLB will require each of your web servers to have two IP addresses – one for each system and a common one for the NLB cluster. For specifics about setting NLB for your version of Windows, please see your Microsoft documentation.

When using NLB, the website is referenced through a single name (just like clustered databases) and if one is busy or off line, the other(s) will take over. Be sure to turn off session state management within the IIS website/virtual directory settings.

The Management Application

For the management application, there is presently no built-in clustering solution available. Rather, if an enterprise license or DR application was purchased, you can install the application multiple times on multiple systems and direct them to the same database. If you do not choose to obtain an enterprise license or DR application and are only able to install one licensed application, in the event of disaster of the system hosting the application, there will still be no interruption to password recovery or availability. This is because all of the data is stored in the database and password recovery is supplied through the website. In the absence of the management application, management of systems lists and job creation will be unavailable until the application is reinstalled and attached back to the original database. Once the management application is reinstalled reconnected to the original database, all groups, systems and jobs will be completely intact.

The installation process for the management application is comprised of accepting the End-User License Agreement and choosing the installation directory. This will take very little time – as long as it takes you to click NEXT, NEXT, NEXT, NEXT, FINISH.

Total Failure

The question will come up: “What if I didn’t backup or all my backups failed, and the database completely failed and I didn’t do clustering, mirroring, log shipping, or similar – what happens to my stored passwords? The answer is: it depends on the target system.

For trusted systems, simply begin randomizing passwords again using your domain authority. For untrusted systems (standalone devices, etc), it may require a reset of the password or authoritative restore of a base password using various products.
Like any important system, it is always recommended to test the backups and examine and monitor system health. E/RPM integrates with various SIEM systems such as Microsoft System Center Operation Manager and ArcSight Enterprise Security Manager for such monitoring and alerting.

Security

As previously mentioned, the database is the single most important component to E/RPM. All systems lists, jobs, job settings, password, and other information about the systems are stored in the database. This means your foremost goal will be to secure the database and how it can be managed or connected to.

First, if using Microsoft SQL Server, implement the use of integrated security for the database. This will allow limiting who has access to the database even if they have access to the management application as each user must then be authenticated to the database. If using SQL authentication, then it will always appear as the SQL account is the one accessing the database and accountability will be greatly minimized.

Next is to control who has access to the management application. By default, anyone who is an administrator on the system where the management application is installed will have the ability to launch the tool (though security on the database will prevent access to the data). This however may not be the desired behavior. To control which administrators have the ability to even launch the management application, go to SETTINGS | DELGATIONS | DELEGATE CONSOLE ACCESS and define which user(s) will have the rights to launch the console.

If two-factor authentication is configured for the user and the machine, E/RPM can also require the user(s) to use their two-factor authentication to gain access to the management application and/or password recovery website.

Change the default password recovery access password from within the management application and configure event sinks to alert on the attempted access to the dialog. The steps for each of these items are outlined in the E/RPM admin guide.
Although the website does not retrieve a clear text password from the database when encryption is enabled, the website does not include its own protection mechanisms when passing passwords to the user's browser and is reliant on the methods implemented within IIS. This means configuration of SSL encryption within the IIS server is of paramount importance. Further, IIS supports the use of user based certificates and these can be used to authenticate users as well.

With E/RPM 4.x and later, the website can employ the use of two-factor authentication. This requires the user to be configured for two-factor authentication and the user to be required to use them within the website which is one of the delegation options. See the help manual included with E/RPM or view the documentation on the product website for exact steps on how to configure two-factor authentication.

When passwords are recovered in the website, one of the configuration options when setting up the website is to send an administrative alert to this effect. This will alert the specified parties that these passwords are being recovered. This is not turned on by default but is highly recommended. See the instruction manual included with the tool or view the documentation on the product website for exact steps on how to configure these alerts.

Applies To:

Enterprise Random Password Manager (ERPM)
Random Password Manager (RPM)

Was this article helpful?
1 out of 1 found this helpful

Comments

Powered by Zendesk