For a licensed exchange, "staying online" is not a UX issue—it is a licensing issue. Under the RMO DAX (Recognized Market Operator – Digital Asset Exchange) framework, Securities Commission Malaysia requires operational resilience: the ability to recover trading and protect customer assets within committed times during hardware failure, cloud outage, cyberattack, or data-center loss. This article walks through a complete BCP/DR design, from RTO/RPO tiering and multi-site architecture to matching-engine and wallet failover, backup and reconciliation, incident response, customer-asset protection, testing, insurance, and reserves, with emphasis on how to turn these capabilities into auditable programs rather than slideware.

1. SC Requirements for Operational Resilience

SC focuses on three layers of operational resilience in RMO DAX guidelines, on-site inspection checklists, and license-renewal reviews. First, critical systems must have a written BCP covering all identified critical business functions—matching, clearing, wallets, deposits and withdrawals, KYC, and customer support. Second, platforms must run regular tests and keep complete records, with test outcomes feeding a remediation loop. Third, customer assets must remain safe and traceable during outages, and platforms must be able to demonstrate afterward to SC and auditors that assets were not lost, misappropriated, or mis-posted. Regulators reject "best effort"; they require defined RTO/RPO targets, owners, escalation paths, and notification timelines to both regulator and customers for every critical function. BCP documents must be reviewed at least annually and updated after major architecture changes, listing-rule changes, or replacements of key outsourcers, and produced to SC on request within set deadlines. Beyond RMO DAX, platforms must also align with capital-markets legislation, AML requirements, and SC guidance on cybersecurity and technology risk management, integrating resilience with internal control, information security, and outsourcing governance so documents do not contradict each other.

2. RTO/RPO Tiering

Not every system needs sub-second recovery, and blindly running everything active-active pushes cost to unsustainable levels. We recommend four tiers by business impact. Tier one is core trading—matching engine, real-time risk, market data, order gateway—with RTO of minutes and near-zero RPO, meaning matched trades cannot be lost. Tier two is funds and clearing—wallets, reconciliation, crediting, withdrawal review—with RTO of tens of minutes and RPO of seconds to minutes; brief delay is acceptable but no accounting errors. Tier three is accounts and support—login, KYC, tickets, notifications—with RTO of hours. Tier four is non-critical systems such as marketing, reporting, and BI, with RTO of a day or more. Tiering must be based on the actual loss per hour of outage, customer and regulatory tolerance, and cost-benefit of recovery—not on tech-team preference. Each target should be written into an internal SLA, and disclosable portions should be surfaced in the customer agreement and status page so market expectations match platform capability. Tiering is not set-and-forget: volume growth, new products (futures, earn), and changing customer mix can elevate previously non-critical systems, so the tiering table and business-impact analysis should be rechecked at least annually.

3. Active-Active and Multi-Site DR

Core trading should run active-active across two data centers or availability zones in the same city or cloud region. Both carry real traffic, and load balancing fails over in seconds while low-latency synchronous replication keeps matching state strongly consistent. Active-active handles facility and AZ failures but not whole-region network loss, a broad cloud-region outage, or a city-scale disaster. A remote DR site in a separate geography is therefore required, usually with asynchronous replication, second-to-minute RPO, and minute-to-ten-minute RTO to take over after a regional failure. Multi-cloud or hybrid deployment further reduces vendor lock-in but must honestly account for cross-cloud latency, clock synchronization, image consistency, and operational complexity; do not sacrifice matching performance on the altar of "multi-cloud." The design principle is "failure is default": every component has a peer, any single failure is isolated and failed over automatically rather than repaired overnight by humans. Traffic management should support health checks, weighted cutover, canary rollout, and one-circuit-breaker emergency stops so that the failover itself does not trigger a second incident.

4. Matching Engine Failover

The matching engine is the most stateful and hardest component to fail over. The primary holds the in-memory order book and positions; a hot standby replays the write-ahead log in real time but does not serve clients. On primary failure, the standby promotes only after confirming log completeness and catching up to the primary's last acknowledged position; clients reconnect through the gateway or config center, unacknowledged orders are resubmitted by clients, and acknowledged trades are determined by the log. Promotion must prevent split-brain—two nodes both believing they are primary—which would cause duplicate fills and corrupted balances. The typical pattern is a lease plus a distributed lock so that only the node holding a valid lease and the newest log can promote; the old primary, when it recovers, must rejoin as a standby and resync. In extremis, a brief trading halt with manual state confirmation is preferable to resuming trading on inconsistent data. Matching failover must also be coordinated with downstream clearing, risk, wallets, market data, and notifications; otherwise matching recovers but the rest does not, producing a half-state where orders can be placed but not filled, or filled but not credited.

5. Wallet and Private Key DR

The first principle of wallet DR is that DR must not weaken key security. Cold-wallet multisig with geographically dispersed backups is itself part of BCP; no single region, person, or provider should be able to move assets unilaterally. MPC key shares should be stored across regions, cloud providers, and physical media, with share generation, rotation, and recovery written into policy and audited periodically. Hot-wallet nodes run multi-instance so one failure does not block broadcast or receipt scanning; keys and shares never leave the secure domain—HSM, MPC node, or signer—and the DR environment replicates signing workflows and node state, never plaintext keys. Recovery drills must verify the full scenario: with an entire region dark, the platform can still sign and withdraw from the surviving region while multisig approval policy, risk limits, and insider controls remain in force. Access rights, approvers, and audit logs in the DR site must be as strict as production so DR does not become a security weak spot. For chain-level reorganizations and node failures, platforms also need node redundancy, parallel chain nodes, and eventual-consistency reconciliation.

6. Backup, Reconciliation, and Recovery

Backups must satisfy both RPO and verifiable recoverability. Production databases take full plus incremental backups with continuous log shipping; ledger, trade, and deposit/withdrawal records are additionally written to WORM or object-locked storage with retention long enough to meet regulatory traceability. Backups alone are insufficient: you must be able to rebuild systems from backup and reconcile. After restoring a complete environment from backup, reconcile on-chain balances, user balances, trades, and money movement line by line so the four agree, with any discrepancy traceable to a specific transaction. Run automated daily reconciliation producing an independent report, and conduct periodic "rebuild the full environment from backup" drills in DR; otherwise recoverability is unknown. Backup security is equally important: offline copies must be encrypted, split-custody, and stored offsite so ransomware cannot encrypt production and backup together. The recovery process must enforce separation of duties so no single administrator can delete both backups and logs. Recovery time itself must be validated against RTO, especially as data volumes grow; sharding, snapshots, and parallel restore are common controls for keeping recovery time in check.

7. DDoS, Cloud Outage, and Cyber Response

DDoS is the most frequent cause of outages for exchanges in Southeast Asia, and both frequency and peak size keep rising. Defense should be layered: upstream anti-DDoS scrubbing absorbs volumetric attacks, CDN and WAF mitigate application-layer CC attacks, the API gateway enforces granular rate limiting and abnormal-traffic blocking, and backend services are only exposed to scrubbed origins. Cloud-region outages are handled through multi-site and cross-region failover; DNS TTLs should be kept low for cutover, while DNS-provider and certificate availability should themselves be monitored to avoid a single point of failure. For intrusion and ransomware, the core principles are least privilege, network segmentation, key rotation, privileged-access approval, and offline backups—even if production is encrypted, a clean offline copy enables recovery and limits the impact of data exfiltration. Every external dependency needs a fallback and a degradation switch: switch payment rails or suspend fiat onramps if a payment partner is down; route KYC to a manual-review queue if the KYC provider is down; switch market-data sources and halt affected symbols if a feed is anomalous; switch chain nodes and raise confirmation thresholds if nodes misbehave. Each degradation should be written into a runbook so on-call staff act on procedure instead of improvising under stress.

8. Incident Response and Communications

BCP is process and organization as much as technology. Platforms should establish an incident command system: the on-call SRE or operations engineer is first responder, a war room is opened by severity, and roles for technical handling, internal communications, external communications, legal/compliance, and customer support are clearly defined with escalation timelines and decision authority. Severity is usually set by impact, fund risk, and regulatory sensitivity; a P1 requires everyone engaged within minutes and executive involvement within a defined window. External communications must go out within committed time—for example 30 to 60 minutes after confirming a major incident—through the status page, app push, official social channels, and media, explaining impact, progress, and an ETA to avoid silence-driven panic and rumor. Reports to SC must follow license and guideline deadlines: an initial notification within the required window, followed by root-cause analysis and remediation after closure, with a proactive debrief for major incidents. Internally, keep a complete timeline, decision log, action record, and communications log for both post-mortem and later demonstration to auditors and regulators that the response was compliant.

9. Customer Asset Protection During Outages

Customer-asset protection is the ultimate point of operational resilience. First, assets must be segregated: customer fiat and crypto are held separately from platform proprietary assets, with cold-wallet ratios, hot-wallet limits, and multisig policies publicly verifiable. Second, during an outage the platform must still be able to complete critical deposit confirmations and withdrawal approvals, or have clearly announced delays and compensation rules so on-chain deposits are not lost because the platform is down. Third, the platform should be able to prove pre- and post-outage solvency through mechanisms such as periodic Merkle-tree proof-of-reserves, independent custody reports, or audit opinions evidencing 100% backing. Fourth, the customer agreement must clearly define liability for outage, rollback, and compensation, including force majeure, chain-level anomalies, and third-party failures, to prevent post-incident disputes. Fifth, customers who lose money because they cannot act during an outage need a clear claims and compensation channel funded by a risk reserve or insurance according to incident nature, rather than a generic "system maintenance" brush-off. Writing all of this into policy is step one; the real test is executing it under incident conditions.

10. Testing and Continuous Improvement

A BCP that has never been tested does not exist. Testing should be layered. Layer one is tabletop exercises, where leadership and key roles walk through an incident scenario to validate process, roles, and communication templates. Layer two is component failover drills, physically shutting down a database, node, or availability zone in a canary or off-peak window to validate alerts, automated failover, and rollback. Layer three is full-stack DR cutover, moving the entire production system to the DR site for a period and back, covering matching, wallets, clearing, gateways, and notifications. Layer four is red-blue/purple teaming and chaos engineering, deliberately injecting failures and simulating attacks to uncover hidden weaknesses. Every drill needs a plan, observation metrics, rollback plan, and post-mortem; issue lists feed remediation with named owners and due dates. A reasonable cadence is tabletops quarterly, component drills one to two times per quarter, full-stack cutover at least annually, plus ad-hoc drills after major architecture changes. Drill records should be retained for years as evidence for SC on-site inspections.

11. Insurance, Reserves, and Third-Party Dependencies

Technical controls cannot cover every loss; insurance and reserves are the last line of resilience. A risk reserve is typically funded from a percentage of trading fees, liquidation surpluses, and dedicated capital, used to cover customer losses from platform outages, breaches, and insider fraud; its funding, usage conditions, and approval process should be published. Commercial insurance can be layered: crime/custody insurance for key compromise and theft, cyber liability for attacks and business interruption, and D&O coverage for management decisions, with limits sized to cold-wallet holdings, daily volume, and regulatory expectations. Third-party dependencies must be brought into BCP as well: cloud providers, anti-DDoS vendors, KYC, payment rails, market-data feeds, chain nodes, and fiat gateways are all potential failure points. Contracts should define SLAs, data ownership, exit and migration terms, and emergency cooperation; critical providers need alternatives and evidence of their own resilience (SOC 2, ISO 27001, regulatory licenses). Critical third parties should be reassessed and reviewed regularly so the platform is not robust while a single supplier outage takes everything down.

12. SoonTech BCP/DR Implementation

SoonTech's white-label exchange platform for Malaysian licensed operators treats operational resilience as a first-class capability. The matching engine supports primary-hot-standby failover with WAL replication and lease-based leader election, deployable in either active-active or remote-DR topologies composed to the client's RTO/RPO targets. The wallet layer integrates with MPC and multisig solutions, with key shares across regions and providers and signing workflows in DR preserving the same policy without lowering the security bar. Built-in backup, WORM retention, and automated reconciliation modules support daily four-way matching across on-chain balances, ledger, trades, and money movement, with exportable audit reports. Monitoring and incident-response modules cover infrastructure, application, business metrics, and on-chain events, with severity-based alerting, war-room tooling, a status page, and customer-notification templates to shorten time-to-communicate. On the compliance side, the platform ships BCP document templates, drill records, incident-report formats, and data-retention interfaces expected by SC on-site inspections, and can integrate with the client's SOC, custodian, and insurers. SoonTech's delivery team works with clients through business-impact analysis, RTO/RPO tiering, architecture review, drill facilitation, and regulator-facing material, rather than simply handing over software.

13. Enterprise Implementation Recommendations

For platforms applying for or holding a Malaysian RMO DAX license, we recommend a four-step path. Step one is business-impact analysis: identify critical business functions, dependencies, single points of failure, and maximum tolerable outage, and produce a tiered inventory. Step two is target architecture: choose active-active, warm DR, or cold backup per tier, avoiding over-investment in non-critical systems while ensuring core systems meet regulatory and customer expectations. Step three is process and policy: turn incident response, communications, testing, backup recovery, and third-party management into executable runbooks and train every on-call engineer. Step four is continuous validation through drills, monitoring, audit, and post-mortems so BCP becomes an ongoing operational capability rather than a one-off project. Executive sponsorship matters: resilience should sit alongside capital, insurance, and compliance at the management table because it determines whether the platform survives an incident. Avoid two extremes—fancy architecture without process and drills, or documentation that is never exercised. The combination is what SC wants to see and what actually protects customers.

FAQ

Q1: Who should set RTO and RPO?

A: Business, compliance, and technology should define them jointly, with executive approval. Tech provides feasibility and cost data, but setting targets in isolation divorces them from business and regulatory reality. Targets go into an SLA and are revisited after major changes.

Q2: Does active-active eliminate the need for a remote DR site?

A: No. Active-active handles data-center or availability-zone failures but not city-scale disasters or broad cloud-region outages. Licensed exchanges with core systems still typically need a remote DR site to cover regional failures.

Q3: Should the DR site carry traffic in normal times?

A: Ideally yes—either a small share of production traffic or through regular drills—so versions, configuration, and data stay aligned with primary. An idle cold standby often fails to cut over when actually needed; warm or active-active standbys are more reliable.

Q4: Does DR increase key-leakage risk?

A: Not if designed correctly. DR replicates signing workflows and nodes, never plaintext keys; MPC shares are cross-region with the same multisig policy, so DR does not lower the security bar. It removes the risk that a single regional outage blocks withdrawals entirely.

Q5: Do drills affect live users?

A: Component drills run in canary or off-peak with second-level rollback; full cutovers are validated in staging or shadow first and then scheduled. Always verify in isolation before touching production, and announce in advance when user impact is possible.

Q6: Who pays for user losses from system faults?

A: The customer agreement defines liability; a risk reserve and insurance cover losses from platform faults. Platform system risk should not be passed to users; compensation follows incident nature, contract terms, and regulatory requirements.

Conclusion

Operational resilience is the baseline capability of a licensed exchange: regulators want verifiable recovery, not polished documents. Linking RTO/RPO tiering, multi-site architecture, matching and wallet failover, backup and reconciliation, incident process, customer-asset protection, regular drills, and insurance and reserves into a closed loop is what lets a platform protect customers, pass inspections, and keep operating when failure actually happens. Technology is only one part; process, people, drills, and continuous improvement matter just as much.

🌐 Build secure and scalable Web3 platforms with SoonTech.

Explore our solutions for White Label Crypto Exchanges, Prediction Markets, MPC Wallets, Matching Engines, Liquidity Integration, and Compliance.