The Curious Case of Salesforce and Workday: Data Integration in the Cloud

The growing enterprise adoption of Salesforce SFA/CRM, Workday HR, Netsuite ERP, Oracle on Demand, Force.com for apps and Amazon Web Services for e-commerce will result in more fragmented enterprise data scattered across the cloud.

Automating the moving, monitoring, securing and synchronization of data is no longer a “nice-to-have” but “must-have” capability.

Data quality and integration issues — aggregating data from the myriad sources and services within an organization — are CIOs and IT Architects top concern about SaaS and the main reason they hesitate to adopt it (Data security is another  concern). They have seen this hosted data silo and data jungle problem too many times in the past. They know how this movie is likely to unfold.

Developing strategic (data governance), tactical (consistent data integration requirements) or operational (vendor selection) strategies to deal with this emerging “internal-to-cloud” data quality problem is a growing priority in my humble opinion. Otherwise most enterprises are going to get less than optimal value from various SaaS solutions. Things are likely to get out of control pretty quickly.

Problem/Situation Summary for Cloud Data Integration

Most SaaS implementations (like Salesforce and Workday) begin at a division/departmental level driven by a specific line-of-business (LoB) need, later expanding to other departments or to an enterprise level. Because each division stores its own business-critical data in various backend systems, small SaaS deployments have simple data integration requirements that can be met with point solutions implemented with minimal IT support.

Larger SaaS application and cloud infrastructure deployments (like force.com) have more complex data and service integration requirements and must fit into a company’s enterprise integration architecture. As a result, they require close collaboration between IT and the business. These deployments must be implemented, integrated, and managed by IT using sophisticated techniques.

To make the most of your company’s investments in cloud solutions, you need to make sure that the data within these solutions is accurate, complete, and up-to-date. Data must be available precisely when your users need it. And data must be fully synchronized with your on-premise applications and databases, and with data in other cloud-based applications you may be using.

Your organization needs to remain in control of your data assets and able to access, integrate, and trust them—wherever they are. Data integration issues can impede your company’s success with cloud solutions. They can delay and inhibit user adoption.

Basic Requirements of SaaS Data

Regardless of the size, every SaaS (Workday HR or Salesforce SFA/CRM ) deployment requires:

• Loading data. Legacy backend systems (e.g., ERP, CRM or HR) usually contain all the operational and historical data needed by end users. Access to relevant, timely, trusted information from these systems significantly improves SaaS adoption and value.

• Securely Synchronizing data. With more data moving to the cloud, companies need to maintain current, accurate, and real-time synchronization with backend systems and in-house databases to ensure secure, reliable visibility into critical information at all times. This is especially critical with data that changes every day, throughout the day, such as customer, sales, inventory, and product information.

• Extracting data. Whether it’s a backup and compliance requirement or a BI/analytics system designed for historical reporting on data from multiple systems, companies need to extract, transfer, and replicate cloud-based data to other systems.

• Visualization Mashups — Gather information from disparate systems and display this within the native user interface of a single application.  It is possible to mashup multi-source data to present a single unified view, without leaving the current application. For example, you can mashup data from multiple service providers into one performance dashboard or scorecard.

Real-world Integration with Salesforce.com

So for IT managers looking to quickly migrate, synchronize or replicate their Salesforce or Force.com data with on-premise systems, five data integration services are required:

1. Data Loader Services ( Batch and Realtime) – Streamline the loading and extraction of data between Salesforce SFA, flat files, and relational databases.
2. Data Synchronization Services. Delivers data loading and extraction capabilities. It also automates everything from simple to complex multistep integrations between various cloud and on-premise systems.
3. Data Replication Services. Automates the replication or archiving of data from Salesforce SFA or an on-premise database to CSV, flat files, or databases.
4. Data Profiling Services. Measures and monitors data quality in the Lead, Opportunity, Contact, and Account objects within Salesforce SFA.
5. Visualization Services: Ability to create intricate scorecards, dashboards and reports using mashup techniques and data from multiple services.

These services are required to meet the data integration needs of Salesforce.com users. Similar capabilities are need for Workday HR also.

Approaches to SaaS Data Integration  

The three most common cloud data integration approaches for SaaS are:

• Custom coding. Whether done in-house or by consultants, custom coding a data integration solution may seem a quick fix at first. But custom coding is a one-off activity and can quickly become time consuming, error prone, and very expensive to maintain.
• On-premise integration platforms and tools. These “configuration, not coding”  solutions are installed inside a company’s firewall on a server or an appliance provided by a vendor, then maintained by IT. Companies that have a skilled data integration team often choose this approach. Both Salesforce.com and Workday provides customers with a Web service-based API for data integration.
• Data integration as a service. This type of data integration takes place in the cloud and is maintained by the integration platform like Informatica PowerExchange. Data integration as a service is ideal for organizations with limited IT resources and/or those relying on a Salesforce CRM administrator or a business analyst to perform and manage the integration.

What is the Right Data Integration Approach for you?

Picking the right approach requires careful analysis of current and future state.  You have to map out various factors as the near-term vs. long-term requirements, size of the company, the availability and sophistication of in-house IT resources, the size and complexity of the SaaS deployment, and the data volumes they need to process.

Issues you need to consider in developing the SaaS data integration strategy:

1)     Are integration requirements documented or even understood? Most SaaS implementations are departmental solutions that are managed by nontechnical line-of-business (LoB) users in sales, marketing, human resources, and customer support organizations in small and midsized companies. This means that you have to work closely with end users to articulate what requirements need to be addressed. What is the scope of the integration effort?

2)     How much internal or outsourced IT capability is available?  Most SaaS customers lack sufficient IT resources to respond to every business request. The reason why LoB and business users went the SaaS route is probably because IT was not as responsive as they liked.  Often it can take a typical IT organization weeks or even months to hand code a data integration solution or 12-18 months to fully implement an on-premise data integration solution. If IT is outsourced to a third party, it can be even more challenging as change orders need to be generated.

3)     Are the business processes and workflows well documented?  Is the data integration a once-a-month need, once-a-week, once-a-day, or a continuous bi-directional workflow.  How many applications are consumers of the data? What decisions are being made or impacted by the SaaS data flows?

4)     Which vendor should you select?  Vendor selection via a coordinate RFI/RFP process is a major effort that needs to be part of the consideration in your data integration journey.  Some traditional data integration vendors like Informatica and IBM Cast Iron have “out-of-the-box” platform/frameworks. They attempt to minimize the complexity that SaaS solutions like Workday and Salesforce require to allow smooth, trouble-free operations across corporate firewalls.

Bottom-line

Data integration has been a well understood process for years, but Cloud SaaS is bringing on new challenges.

With the cloud, more and more companies are turning to a wider variety of best-of-breed technology solutions to build a “collection of services” that’s custom-fit for their business, it’s clear that the integration challenge is mounting.

The problem is similar to the Trojan horse (a simplistic trick that causes a target to invite a foe into a securely protected bastion or space).  Many SaaS projects like Workday or Salesforce.com start at a departmental level with simple integration requirements in their initial phase of deployment, but it’s common for the integration complexity to grow exponentially as end-user adoption increases and the SaaS application becomes mission-critical and part of enterprise architecture.

But it’s not all doom and gloom.  Integration problems can be easily addressed with the right management focus and team.  It is time for IT leadership to realize that automating the movement, monitoring, securing and synchronization of data is a “must-have” capability.

Additional Notes, Resources and Links

1. Data quality challenges will continue to rise with the cloud. With more data and more types of data, comes more work in cleansing and parsing that information.
2. We are very impressed with Informatica and their expertise in the SaaS data integration.  They are selling “Data Integration as a Service” which purports to have three advantages: (1) Pay as you go and pay for only what you use—pricing is flexible and affordable; (2) Get up and running quickly to deliver value to the business faster; and (3) Eliminate costs associated with buying and maintaining hardware and software.
3. In 2011, we are seeing a rapid increase in Workday HR adoption.  Most firms have old versions of Peoplesoft and other solutions that are difficult to upgrade and simply not adequate to meet the needs of 2011. As core HR – payroll, transaction management and benefits (health and retirement) –  evolves into Talent, Succession, Performance and Compensation Management,  HR Analytics and other areas, more and more companies are looking at Workday.
4. Snaplogic released 2011 Application Connection Priorities - results of a executive survey on SaaS Data Integration with the following conclusions:

• The number of responders who will implement at least four SaaS/Cloud  applications will double (17% versus 33%) over the next 2 years.
• The major benefit from easily connecting applications and data would be the increase in time for staff to work on other business initiatives (56%).
• Implementation costs (69%) and reusability/adaptability of solutions for future integration projects (54%) are the two most important requirements for integrating applications and data (CoE Model?)