Hybrid Data Replication

Replicate a database from a source to a target (where both the source and target can be Cloud-based or on-premises) using both a cloud based and on-premises Data Integration Platform.

Pattern Definition

Problem How do I replicate data from a database to a database in the Cloud or on-premise with maximum flexibility?
Context This pattern is applicable when there is a desire to replicate data. There may be multiple Cloud-hosted databases and multiple on-premises databases.
Forces
  • Need a strategic and comprehensive platform
  • Want to extend an established on-premises Data Integration Platform
  • Access to on-premises database may be too slow.
  • Quickly attach to existing on-premises Data Integration Flows
Solution This data pattern employs a Data Integration Platform both on premises and in the Cloud. While this may increase the cost and complexity of the overall solution it also provides the greatest flexibility to replicate data.
ConsequencesPros:
  • Provides the greatest flexibility
  • Encapsulates on-premises data replication. On premises to Cloud communication only required when replicating on premises to Cloud.
Cons:
  • Potentially increased complexity to having only one Data Integration Platform.
  • Potential performance impact due to data going through two Data Integration Platforms.
  • May require different skill sets for on-premises and Cloud-based Data Integration Platform
  • One-way replication does not support a master-master requirement.

Interaction View

Relational Data Replication

A static or synchronized copy of an on-premises database replicated to a Database in the Cloud. Database using data integration technologies both in the Cloud and on premises. 

Interactions

  1. On-Premises Data Integration Platform captures the data from source database. It can be configured for initial loading of the source data as well as to capture the changed data on the source with the target. It can also be configured to propagate any DDL changes on those databases where DDL change support is available.
  2. The on-premises Data Integration Platform sends the data to the Cloud-based Data Integration Platform.
  3. The Transmitted data and DDL changes are applied to the target database.

Notes