What Does the Future Hold for CDPs?

Composable Customer Data Platforms (CDPs) leverage cloud data environments  and ancillary services to deliver incremental CDP functionality.  All this without the requirement of locking in a single CDP Vendor.

At Transparent we have seen the shift of Composable CDPs.  It has shifted from a discussion topic amongst techies to a topic of deep curiosity when speaking with our clients. The primary questions we hear are: “Should I go with a composable CDP or a conventional SaaS CDP? and How are they different? Unfortunately, the answer to these questions is typically: “Well, it depends”. 

In this article we provide an introduction to the various components of a customer data ecosystem.  These include conventional CDPs, Cloud Data Ecosystems and Composable CDPS.  We hone in on the main differences between these solutions. We also share some key considerations you may use when determining if a Composable CDP is right for your organization. 

The Technical Landscape 

Let’s start by defining some of the terms we will use in this discussion. The following provides a high-level description of a CDP, SaaS CDPs, CDEs, and the Composable CDPs. 

CDP Customer Data Platform

• An integrated set of software and data services.  CDPs enable the aggregation of customer data from multiple sources in order to establish a unified view of customers for analysis and activation.
• CDPs create comprehensive and unified consumer profiles.  They do so by using first-party identity data collected directly from interactions with consumers across the brand’s owned touchpoints.

Figure 1: CDP within a MarTech/AdTech Stack

• These profiles are further enriched by integrating second-party data obtained through partnerships and third-party data from external sources. 
• As a result, the CDP creates a robust 360-degree understanding of consumers.
• The Marketing organization can now use these 360-degree profiles to create demographic segments for use in targeting specific groups of customers or to drive product development.  

Conventional (SaaS) CDPs

• The functionality to support customer data management and activations (as described above) is provided by a CDP vendor as a turnkey solution. 
• The CDP functionality is delivered to the customer through an integrated set of software and infrastructure; delivered to the brand as Software as a Service (SaaS). 

Figure 2 SaaS CDP Architecture

• SaaS delivery means that all infrastructure, supporting operating systems, data management services, data storage and software are developed, maintained and supported by the CDP Vendor.
• New CDP functionality is developed and delivered to the SaaS vendor’s customers based on the vendor’s product roadmap. 
• SaaS CDPs allow an organization to outsource the costly data infrastructure and technical resources required for the development and operations of the CDP. 
• There exists today a wide range of CDP vendors, each with individual strengths in functionality. In fact, it is likely that no single SaaS CDP will provide the perfect solution to all brand needs and use cases.  Therefore it is possible (not frequent) for a brand to have two or more CDPs to achieve all of the functionality they require. 

Cloud Data Ecosystems*

• Cloud Data Ecosystems (CDE) provide a federated repository of large volumes of unstructured, semi-structured and structured data in native formats.  All while creating segments of this data in structured databases for high performance consumption by systems and software.
• CDEs provide for the storage and retrieval of data using cost-effective technologies.  This functionality returns considerable savings to organizations that were storing all their data in data warehouses.

Figure 3 Cloud Data Ecosystem

• CDE Databases provide correlation and presentation of smaller sets of this data for use by systems (CDPs ESPs CRMs) as well as by humans for analytics, reporting and dashboards. (Think Databases)
• CDEs are also composable in nature in that the data stored within the ecosystem varies from company to company therefore the functionality also can evolve over time. 
• The use of CDEs within corporate enterprise is commonplace today and is a valid cost avoidance solution for organizations that have large amounts of transaction and event data within their customer data environment. 
• For further information on Cloud Data Ecosystem please refer to the references at the end of this article. 

*Note: We use the generic “Cloud Data Ecosystems” to describe the functionality provided by Data Services companies such as Data Bricks or Snowflake 

Figure 4 SaaS CDP and CDE

SaaS CDP paired with a Cloud Data Ecosystem

A frequent challenge our clients face is the complexity and scale of the data created by systems that generate the data required to feed the CDP. Examples of two large data generators are retail transactions and web/app event collection. We have seen cases where a single sales transaction generates dozens of line items across dozens of tables.  Or a single web or app user session generates hundreds of semi-structured records. Data sources such as these present several problems within the CDP data flows. 

• The data structure within the source system is designed to support the source system’s mission. While this complexity is important to the source system, this complexity can create risk to the CDP’s data integrity.
• Defining and creating data extracts from the source systems for consumption by the CDP can be an arduous and time consuming task. Additionally, each change to the source system’s data structure could lead to rework. 
• The development of these data extracts requires time and expertise from each source system’s developers. Keep in mind that these individuals will likely have limited availability.
• Custom data extracts developed for the source system may not be available to other potential data consumers within the brand.  Doing so would create the risk of data silos.
• Many systems store transaction data across multiple tables each with many attributes required to manage the transaction. The CDP does not require the same level of fidelity and must translate this complex data structure for its consumption. This translation can create risk to the CDP’s data integrity. 
• If ingested directly, dozens of tables in the source system equate to dozens of tables in the CDP. The CDP must now select the nuggets of data in order to generate the CDP’s version of the transaction. This data mapping can create risk to the CDP’s data integrity.
• If the SaaS CDP relies on structured data bases for storage of source data, then the CDP vendor is going to pass the expense of this costly infrastructure on to the brand. We have seen cases where brands are forced to periodically delete source data to avoid data overage charges.  If the source data was formatted for the CDP and not stored elsewhere, the historic data is no longer readily available for consumption. 

Many SaaS CDP adopters may be tempted to focus on the current problem: “get data into the CDP”. However, let’s step back. Looking at the long term value of brand data will result in a more cost effective and trusted CDP data flow design. Resulting in sustainable and less costly data flows.

• CDEs allow the brand to store large quantities of data in lower-cost infrastructure while retaining the source record’s integrity.
• CDEs provide the ability to create consumer (CDP, Analytics, Dashboards) views of the data.  Enabling brands to view specific data for each consumer’s requirements, while retaining the source record’s integrity.
• Prestaging source system data within the CDE can reduce the time to extract and package data for consumption by the CDP. Saving costs upfront in engineering time and avoiding unplanned CDP data storage costs over the years. 
• This architecture allows organizations to create a single source of truth for all customer data.  Doing so enables multiple systems and use cases (including the CDP) to consume this data from this single source of truth.
• This single source of truth and data sharing mechanisms increases the ability to govern customer privacy data. 
• As the CDE can contain other business data such as inventory, sales and revenue, CDP Data can be combined with these other forms of performance data to develop broad business insights into consumer responses to marketing tactics. 

CDEs do require a considerable level of data architecture and engineering talent to build and operate. However, the addition of the lower cost mass storage available within a CDEs to any Martech ecosystem is also a good alternative for retention of CDP data that must be archived from CDP

Composable CDPs with a CDEs

• The typical candidate for a composable CDP is a brand that has an existing CDE within its data enterprise. 
• A CDE as described in the previous sections provides the foundational data management and storage functionality that typically does not come with a SaaS CDP.  
• With a composable CDP, the CDE provides the only set customer data used by the CDP applications. 
• CDP functionality is achieved by integrating individual customer data software modules from discrete vendors into the CDEs’s information and integration bus.
• CDP software modules create new attributes within the CDEs databased with little to no duplication of data.  Doing so reduces the risk of having multiple copies of customer data within the ecosystem.

Figure 5 Composable CDP & Existing CDEs

• This approach allows organizations to add CDP functionality on top of an existing CDE. Doing so enables organization to leverage their existing data management infrastructure for marketing use cases.
• Brands can take a modular approach to acquiring CDP functionality, given the wide range of composable software capabilities available for customer data management. Individual modules of CDP functionality can be added or swapped out as needs change.
• It is important to remember that CDEs and composable CDPs require a strong set of technical resources to configure and operate.
• Earlier, we mentioned that a single CDP may not fully fit a brand’s customer data requirements.  This has led to some companies purchasing 2 or more CDPs to meet their needs.  With Composable CDPs, it is feasible for a company to fill gaps in their current CDP functionality by using discrete applications rather than an entire CDP.

Factors when Considering Composable CDPs

1. IT and Data Engineering Resources 

• The # 1 consideration for any organization considering a composable CDP is the availability of dedicated (internal or outsourced) IT and Data engineering resources.
• Composable CDPs are a DIY adventure and require significant engineering resources. 

2. The Existence of an Existing CDE

• Composable CDP proponents assume that the adopting organization already has robust CDE and ancillary data management services. 
• The key advantage of a composable CDP is that it leverages existing investments.  Specifically those associated with data aggregation, storage and sharing achieved through a robust CDEs.
• The absence of an existing CDE for customer data does not equate to a full stop on the composable trail.  However, the development of a CDE will have to be included in any plan to deploy a composable solution.
• Again, CDEs are composable.  Therefore start with the minimum functionality required to collect and store your customer data then add more data management functionality as needed.

3. Lead time to Capability

• How fast do you want to achieve a minimum viable product (MVP)? 
• Have an existing, mature CDE? A composable approach may reduce lead time. 
• A majority of the lead time associated with activating a CDP is associated with the work of identifying data sources, preparing data extracts from their source systems and the determination of data aggregates for loading into the CDP.  Having all of these data sources resident within the CDEs reduces the data prep lead time.

4. Incremental Capability 

• Unclear on what functionality you need? The composable approach allows incremental insertion of functionality without committing to an extended service contract with a single CDP provider. Discrete modules of functionality can be evaluated and purchased incrementally. 
• Changing needs: The marketplace has a wide range of offerings in each functional area to choose from and the list of offerings is changing everyday. As individual functions within the CDP are offered by discrete vendors, a brand can change out modules of the CDP as needs change.
• Even with a SaaS CDP in place, Brands can now utilize individual composable CDP offerings to fill these gaps reducing the need to buy multiple SaaS CDPS.

Conclusion

Organizations with an existing CDE and data management infrastructure should consider a Composable CDP architecture, when shopping for a CDP.   These composable CDPs allow organizations to explore from a wide range of bespoke tools rather than locking in with a single source SaaS CDP. While composable CDPs do require a robust data management structure built around a CDE, the ROI for using CDEs to achieve a CDP still makes a strong case for consideration.

Click here to find out how Transparent Partners helps its clients  navigate the evolution of composable CDPs and CDEss for customer data management.