Replication

BETA: This is a beta feature. Production use is highly discouraged.

The component Sync provides the ability to replicate data between Cloudrexx websites.

Architecture

The Sync component listens to model changes and pushes them to any configured remote (another Cloudrexx installation having Sync set up) through the RESTful-API (DataAccess). For proper foreign key handling, the Sync component injects a wrapper command (called sync) on top of the RESTful-API command v1. Meaning, instead of pushing data directly to /api/v1/<outputModule>/<dataAccessName>/<elementId> on the remote, data is pushed to /api/sync/v1/<outputModule>/<dataAccessName>/<elementId> instead.

╔════════════════════════════╗
║ Cloudrexx #1               ║
║                            ║
║        ┌──────────┐        ║
║        │ Database │        ║
║        └──────────┘        ║
║             │              ║
║   ┌────────────────────┐   ║
║   │ DoctrineRepository │   ║
║   └────────────────────┘   ║
║             │              ║
║       ┌────────────┐       ║
║       │ DataSource │       ║
║       └────────────┘       ║
║             │              ║
║       ┌────────────┐       ║
║       │ DataAccess │       ║
║       └────────────┘       ║
║             │              ║
║         ┌──────┐           ║
║         │ Sync │           ║
║         └──────┘           ║
║             │              ║
╚════════════════════════════╝
              │
              ⇅ HTTP
              │
╔════════════════════════════╗
║             │              ║
║         ┌──────┐           ║
║         │ Sync │           ║
║         └──────┘           ║
║             │              ║
║       ┌────────────┐       ║
║       │ DataAccess │       ║
║       └────────────┘       ║
║             │              ║
║       ┌────────────┐       ║
║       │ DataSource │       ║
║       └────────────┘       ║
║             │              ║
║   ┌────────────────────┐   ║
║   │ DoctrineRepository │   ║
║   └────────────────────┘   ║
║             │              ║
║        ┌──────────┐        ║
║        │ Database │        ║
║        └──────────┘        ║
║                            ║
║ Cloudrexx #2               ║
╚════════════════════════════╝

Supported DataSource

The Sync component does currently only support the replication of the DataSource DoctrineRepository.

Setup

To enable the replication of model data two steps have to be taken:

1. Enable RESTful-API on Replica
2. Enable Sync on Primary

Enable RESTful-API on Replica

On the Cloudrexx installation that will act as the replica, do as follows:

1. Create an API endpoint for the model that shall be replicated:

   Replace MODEL by the FQCN of the model to replicate. E.g.: Cx\\Core\\Routing\\Model\\Entity\\RewriteRule
   Replace ENDPOINT by a unique name to identify the endpoint. E.g.: RewriteRule

   INSERT INTO `contrexx_core_data_source`
       SET `identifier` = 'MODEL',
           `options` = 'a:0:{}',
           `type` = 'doctrineRepository';
   INSERT INTO `contrexx_core_module_data_access`
       SET `data_source_id` = (
               SELECT `id` FROM `contrexx_core_data_source` WHERE `identifier` = 'MODEL'
           ),
           `name` = 'ENDPOINT',
           `field_list` = 'a:0:{}',
           `access_condition` = 'a:0:{}',
           `allowed_output_methods` = 'a:0:{}';
   

2. Create a new API-Key (under Administration > RESTful API) and assign the newly created endpoint (e.g. RewriteRule) having write access

Enable Sync on Primary

On the Cloudrexx installation that will act as the primary, do as follows:

1. Create the same API endpoint as the one created on the replica:

   Replace MODEL and ENDPOINT by the same values used on the replica.

   INSERT INTO `contrexx_core_data_source`
       SET `identifier` = 'MODEL',
           `options` = 'a:0:{}',
           `type` = 'doctrineRepository';
   INSERT INTO `contrexx_core_module_data_access`
       SET `data_source_id` = (
               SELECT `id` FROM `contrexx_core_data_source` WHERE `identifier` = 'MODEL'
           ),
           `name` = 'ENDPOINT',
           `field_list` = 'a:0:{}',
           `access_condition` = 'a:0:{}',
           `allowed_output_methods` = 'a:0:{}';
   

2. Enable replication of the newly created endpoint:

   Replace MODEL by the same value used to create the endpoint.

   INSERT INTO `contrexx_core_module_sync`
   SET
       `data_access_id` = (
           SELECT `id` FROM `contrexx_core_data_source` WHERE `identifier` = 'MODEL'
       ),
       `to_uri` = '',
       `api_key` = '',
       `active` = '1';
   

3. Register the replica:

   Replace HOST by the domain name of the replica (for local environments using cx env see Connect multiple ENVs to each other).
   Replace API_KEY by the previously generated API-Key on the replica.

   INSERT INTO `contrexx_core_module_sync_host`
       SET `host` = 'http://HOST',
           `active` = 1,
           `api_key` = 'API_KEY',
           `api_version` = 1,
           `url_template` = '',
           `state` = 0;
   

4. Enable replication of the newly created endpoint to the replica:

   Replace MODEL by the same value used to create the endpoint.
   Replace HOST by the name of the replica (must be identical to the one used in the previous step).

   INSERT INTO `contrexx_core_module_sync_host_entity`
   SET `sync_id` = (
           SELECT `id` FROM `contrexx_core_module_sync`
           WHERE `data_access_id` = (
               SELECT `id` FROM `contrexx_core_data_source` WHERE `identifier` = 'MODEL'
           ) AND `active` = '1'
       ),
       `host_id` = (
            SELECT `id` FROM `contrexx_core_module_sync_host` WHERE `host` = 'http://HOST'
       ),
       `entity_id` = '*';
   

Special Case: Calendar

As the model of the Calendar component is not purely a DoctrineRepository DataSource (Remember: DoctrineRepository is the only supported DataSource), but instead a mix of LegacyDatabaseRepository and DoctrineRepository, the setup is a bit different.

1. On the replica do as follows:
   1. Add a new API-Key and assign all calendar-* endpoints (with write access)
2. On the primary do as follows:
   1. Enable endpoint (calendar-event) for replication:

      UPDATE `contrexx_core_module_sync`
          SET `active` = '1'
          WHERE `data_access_id` = (
              SELECT `id` FROM `contrexx_core_module_data_access` WHERE `name` = 'calendar-event'
          );
      

   2. Enable publishing function in Calendar component:

      UPDATE `contrexx_module_calendar_settings`
          SET `value` = '1'
          WHERE `name` = 'publicationStatus';
      

   3. Register the replica:

      Replace HOST by the domain name of the replica (for local environments using cx env see Connect multiple ENVs to each other).
      Replace API_KEY by the previously generated API-Key on the replica.

      INSERT INTO `contrexx_core_module_sync_host`
          SET `host` = 'http://HOST',
              `active` = 1,
              `api_key` = 'API_KEY',
              `api_version` = 1,
              `url_template` = '',
              `state` = 0;
      

   4. Register replica as publishing-target in Calendar component:

      Replace HOST by the name of the replica (must be identical to the one used in the previous step).

      INSERT INTO `contrexx_module_calendar_host`
          SET `title` = 'Replica',
              `uri` = 'http://HOST',
              `cat_id` = 0,
              `key` = '',
              `confirmed` = 0,
              `status` = 0);
      

Bidirectional Replication

Note: For easier understanding the terms primary and replica refer to the Cloudrexx installations acting as primary and replica of the initial instructions from above. Meaning that when setting up the bidirectional replication (according to the following instructions) the term primary technically refers to the installation acting as the replica and replica to the installation acting a primary.

To enable bidirectional replication, simply set the replication up the other way round. Meaning:

1. Create a new API-Key (but this time on primary) and assign the previously created endpoint (e.g. RewriteRule) having write access
2. Enable synchronization of the endpoint on the replica
3. Register the primary on the replica as replication target
4. On the replica, enable replication of the endpoint to primary

Important: Bidirectional replication only works properly, if the HOST on both sides are set to the main domain (configuration option Main domain under Administration > Global Configuration > System) of the other side.