What is operational data sharing?

Operational data is information that supports the processes and systems that run a business. Operational data sharing is gaining traction, but some organizations are anxious and nervous about it. There’s a "don't share unless" attitude towards data sharing within and across organizations. If you’re part of an organization that still wants to hang on to old beliefs by not sharing operational data with other departments and affiliates, you are missing out on money to be made. You’re also failing to leverage time-saving opportunities, and your team is ignoring a powerful concept that can transform business intelligence into intelligent business.

In simple, metaphorical terms, operational data sharing is what organizations need for breakfast, lunch, and dinner. It provides necessary calories and vitamins to run your business. Both the quality and quantity of data sharing account for seamless daily operations. (Your data warehouse and data lake are more like a metaphorical 10-course meal on a special occasion because they require more planning and budget.)

Operational data sharing defined

Operational data is produced by and records your organization's day-to-day activities and transactions. Examples include information and processes related to your workforce, customers, vendors, inventory, and financial data. Due to security and trust concerns, most operational data is housed in enterprise IT's walled gardens and ends up in an operational data silo. As organizations grow, data silos emerge naturally, and factors such as technology, company culture, and organizational processes limit or discourage information sharing. Data silos isolate data from users who do not have access to it. As a result, business strategies and decisions are not based on all available data, which can lead to poorly informed decisions.

A data silo (or information silo) is a repository of information in a department or an application that is not easily or fully accessible by other departments or applications. Marketing, sales, HR, and other departments rely on specific information to function, and those collections of often overlapping-but-inconsistent data are in separate silos. This can result in duplicate and inconsistent records, and it’s a major drag on efficiency.

Imagine, for instance, that a marketing department runs its automated marketing campaigns through Marketo while the sales department maintains its customer data in Salesforce. Without proper integration, the marketing outreach data is isolated within the marketing department, and the sales information is also siloed. This isolation makes it difficult or impossible to attribute sales to a particular marketing campaign since neither department can see where leads come from, accurately measure campaign performance, or develop strategies for increasing lead flow and conversions from ideal prospects.

The direct costs of silos are not visible, but they waste company resources and reduce employee efficiency.

Silos harm organizations in five ways:

1. Silos slow data-driven decision-making, which can impair the ability to compete.
2. Silos inhibit trust and constrain collaboration among company teams.
3. They come with high costs as a result of redundant IT and application infrastructure.
4. Silos reduce data quality, limiting the ability to leverage data analytics.
5. They cage the customer experience, diminishing quality through the customer journey.

Finally, the problems caused by data silos are exacerbated as people either accept work in isolation or devise workarounds that become complex, difficult to maintain, and likely to yield incorrect information. This conduct results in a negative feedback loop where processes deteriorate over time.

If you are unsure whether you have a data silo problem, consider the following five questions:

1. Can you see all aspects of your operation?
2. How much of your business is influenced by subjective decisions or gut feelings?
3. Does your team use rules of thumb in important activities?
4. Are you still using paper and manual processes in your business?
5. Are you prepared to handle rapid business growth?

You have a data sharing problem in your organization if your answers to these questions match the answer key sequence: Yes, Some, Yes, Yes, Not Sure.

Use cases for operational data sharing

Operational data sharing enables data-driven decision-making for tactical decisions, which is what organizational managers eat for breakfast, lunch, and dinner. For some folks, it may seem to be small stuff. But, sometimes it pays to sweat the small stuff.

Here are a few examples to describe what is possible with operational data sharing:

Airline on-time performance (OTP)

In the airline industry, one delayed plane in the morning can cause more than 70 other planes to be delayed in the afternoon. One reason US domestic flights are about 80% on time at 6:00 a.m. and only about 50% on time at 6:00 p.m. is that minutes delayed typically double by the day's end. OTP is affected by decisions made in multiple areas, including fleet size and composition, flight crew composition, and flight turnaround times.

Maintaining good OTP is an uphill battle. Many people, processes, functions, and technologies need to work together with a high degree of integration day and night. . For example, scheduling and operations personnel must be in constant communication since the activities and decisions of one directly affect the other. Poor scheduling often creates problems for operations, and when different functional groups fail to execute as one team, scheduling may need to be adjusted on the fly. This high degree of cross-functional integration on the day of execution is only possible if departments and affiliates in flight operations collaborate and share data with a "share unless" attitude.

Responsibly expanding access to credit

If properly designed, leveraging operational data shared by multiple sources, banking products can expand access to credit that helps, rather than harms, consumers. In the United Kingdom, Mojo Mortgages combines banking data with more widely used scoring methods to accurately assess what a customer can afford. Canopy uses consumer rent payments to improve credit scores, a metric previously excluded by most large credit bureaus.

Business automation in investment banking

According to a London School of Economics study, at least two-thirds of staff time was spent in a parallel world outside core IT systems, running on email and Excel, and generating enormous communications complexity. There is an opportunity to automate human tasks with a combination of AI and "data-aware" APIs.

People transportation businesses

Organizations in people transportation have improved their services and effectiveness by incorporating operational data sharing into their route planning. Instead of planning routes from point A to point B, they use technology to turn planning into a science. They can now make smarter business plays and decisions (like route planning) by taking into account shared operational data such as ride times, time windows, cancellations, returns, delays, and more.

The opaque supply chain

Opaque supply chains are another big area full of issues due to a lack of operational data sharing amongst partners. If cargo ships become stuck in the Suez Canal, factories in Guangzhou close due to stay-at-home orders, or transport routes are disrupted by war, it is difficult for organizations to determine how much the total worth of goods is stuck in transit or how the delays in parts and equipment supplies will impact production. The shortage of parts and equipment leads to a flow of counterfeit in the market to fulfill demand. Now you have a vicious circle that gets worse every day. This is not only a revenue loss, but it tarnishes the image of your brand. Organizations invested in data sharing prior to pandemic have fared better during pandemic supply chain crises than others that haven't. The pandemic tested the adaptability of an organization's supply chain. Organizations with good visibility into the end-to-end supply chain can quickly adapt and proactively react to global and local crises as soon as signals become visible.

Operational vs. analytical data sharing

The difference between operational and analytical data sharing is the difference between "observe your business" and "run your business." Operational data is about operational decisions on the day of execution. Analytical data is about near-term and long-term planning and can only be enhanced when paired with operational data. The operational data sharing not only improves the quality of day to day decisions but also informs planning and strategy in a qualitative way.

Differences between operational and analytical data sharing

With operational data sharing, you create visibility and transparency that allows you to factor data into your tactical decision-making to run your business daily, hourly, or weekly. The data is captured and made available to relevant stakeholders consistently (as soon as it happens).

There are two popular ways to make this data available for tactical decision-making on the day of execution: Peer-to-Peer (P2P) interfaces and the data-centric approach.

You can build Peer-to-Peer (P2P) interfaces between operational stores to bring cross-functional data into real-time applications and analytics. This approach has several disadvantages though, which makes it a complex IT operation. The data supply chain is pull-based; in the event of a higher demand for products and services or customer interactions with the brand, it puts multiple systems and networks under stress. This approach is app2app integration and doesn't make your data agile and interoperable within the organization.

In the data-centric approach, the data is pushed to a centralized repository in an intermediate data format (i.e., common data format or canonical data format) that is based on the language you speak with your customers and partners. This has two benefits. First, your data is interoperable; second, you can scale the intermediate datastore independently of siloed legacy systems.

Compare the data-centric approach with analytical data sharing, which is about finding patterns in historical data (months and years old) and using the patterns to define a plan. Analytical data is best stored in a data system designed for heavy aggregation, data mining, and ad hoc queries; it’s called an Online Analytical Processing system, OLAP, or a data warehouse. In data warehouses, heavy extract, transform, and load (ETL) usage is involved. Maintaining uptime for data pipelines is essential. The amount of data transformation involved introduces latency and makes it unfit for real-time operational use. So, you can only use it to observe and report your KPIs to senior leaders in your organization and for short-term and long-term planning.

To recap, operational data systems, consisting largely of transactional data, are built for quicker updates and tactical ease of execution decisions. Analytical data systems, which are intended for long-term and short-term decision-making, are built for more efficient analysis.

The difficulty with implementing operational data sharing

The threshold is often high to start sharing data, commonly due to an overestimation of the value of existing data. (There are market values put on companies that own a lot of data, but frankly, those are guesses.) Sometimes the value of data is its price on the market. Other times, data is valued by the cost of its production. But, even then, value is not always clear. For example, the data generated in internet shopping is often a byproduct of the transaction.

The value provided by the integration of different data sources is often underestimated. Simple datasets just won’t cut it anymore. To get truly powerful insights, you need to pull in data from multiple sources. The more complex and diverse your datasets are, the more surprising and potent the insights they’ll produce. Additional data sources increase your chances to inform actions, fueling top-line and bottom-line growth.

In other cases, data availability and data quality are common problems for operational data sharing because some businesses (or some departments within businesses) are still not digitized. Information is first recorded on paper and then entered into the IT systems. In some cases, real-time data is not available. For example, most people still buy their insurance through small shop agencies that operate with inadequate IT infrastructure. Most simply use Excel to capture and report data leading to structural, semantic, and naming convention-related inconsistencies, leading further to underwriting and claims processing friction. Imagine if there were a way to harmonize the data’s naming conventions, structure, and semantics in a consistent format. This auto reconciliation of data could have a big impact on data agility. Your siloed data will start to flow in day to day decisions more regularly. It will drive business agility.

Besides issues related to methodologies for assessing the true value of current data and data quality issues (due to lack of digitization and manual entry of data into IT systems with delays and errors), there are other issues further fragmenting the operational data landscape.

So far, we've only heard of data sitting in silos because an organization uses multiple applications for different business functions, such as Marketo for marketing campaigns and Salesforce for sales and service. However, a new category of challenges is emerging because companies have started to adopt multiple clouds and use multiple cloud regions.

Using multiple clouds is not always a decision in an organization’s control. Organizations know it’s hard to do multi-cloud, but because of mergers and acquisitions, it’s a problem they cannot avoid. The acquired company could use a different cloud than the acquiring organization prefers. They also don't want to move everything to a single cloud every time after an acquisition because the reverse trend, such as disinvestment, is also possible. Why would an organization migrate to a single cloud when they may have to disinvest in that business later? The side effect of this decision is the organization data is now fragmented on two different clouds.

Organizations cannot avoid operating in multiple regions, either. Data privacy regulations such as the California Privacy Act, the EU General Data Protection Regulation (GDPR), and the Canadian Consumer Privacy Protection Act are all good reasons to keep data in separate cloud regions.

How Vendia can help

Data is on everyone's mind, and even with the plethora of tools on the market, there hasn’t been a good solution for solving data problems. We need a new discipline, a "data-aware" discipline to start thinking about data in a more specialized way to solve data problems.

There will never be one data solution for all problems:

• You’ll still require a data warehouse to conduct high-quality enterprise reporting and to widely distribute standard reports to observe your business.
• You will have data lakes to dump data into low-cost storage and then bring tons of tools to discover something out of it.
• You’ll still have a fresh need to connect operational data for real-time business operations to differentiate your business and serve your consideration better while improving your topline.
• And you cannot solve those challenges with P2P interfaces.

You’ll need an intermediate (interoperable) data store consistent with your siloed operational systems and its infrastructure to be cost-efficient and easy to maintain to build data-aware APIs for operational data sharing.

This is where Vendia can help. As a "data-aware" solution, we make your data agile. This agility further improves data integrity, scalability, and performance so you can achieve new successes you once thought were impossible:

1. Monetize the data value you’ve left on the table
2. Eliminate inefficiencies for which you created workarounds
3. Form partnerships that no longer make you feel anxious due to trust issues

You can use Vendia to build the following architectural patterns for operational data sharing:

Ledgered data hub

You can aggregate data from different sources to create a single source of truth that you can use for audit, compliance, provenance, licensing, entitlements, and authenticity. You can also use this architectural pattern to create an event graph of interaction with the dimensions of time for different business objects: Customers, audiences, or bills of materials for parts, cars, equipment, materials, etc. You can also use it to build high-quality, cross-departmental (cross-domain) data stores with automatically generated data that you can consume from your packaged applications (SaaS and legacy).

Secure data exchange

You can use this pattern to exchange data with your affiliates: suppliers, B2B customers, data monetization, supply chain data, sustainability data, intellectual property data, etc. Every party involved in data sharing gets a node (like a secure inbox with incoming and outgoing messages tracking enabled) that allows you to notarize data shared and received by each party and brings trust and transparency into data sharing.

Secure data aggregation

The data you need to develop predictive models lives with multiple partner organizations or customers. How can you integrate this data with strong governance in a privacy-preserving way? You can use the data from partners, transfer it, and use it to build high-quality data stores (data lake and data warehouse) for your analytics.

After all, Andrew Ng thinks that better AI needs good data, not big data. The naming convention-wise, structurally, and semantically consistent intermediate data stores can power quality analytics and AI in your organization. Secure data aggregation using Vendia is the answer to your data-centric AI. Vendia can provide real-time, fresh, and harmonized data for your AI algorithm from multiple diverse sources in a privacy-preserving manner. .

Auto reconciliation

You can use this pattern for loyalty settlements, earning and settling rewards, telecom roaming charges settlements, invoice payments based on service usage, or royalty on IP use. You don't have to reconcile microtransactions from multiple partners; they are automatically reconciled. You can even flag and trigger workflows for exception handling. Every party has access to microtransactions independently through their own nodes. Using Vendia auto reconciliation, you can say "No" to cumbersome manual reconciliation efforts at the end of the billing cycle due to errors and inconsistencies in data reported by two different parties. Vendia provides a single shared view of data for the same transactions to all involved parties.

Data escrow

If you or your partner notices data overlap that can be used to create new synergies but cannot share it directly, Vendia has tools that can assist you with data statistics without exposing the data or IP of other partners.

Take action: Bridge your silos

Vendia provides distributed ledger technology with a data-centric API (bring your data model, and Vendia automatically generates API based on your data model) that will allow you to share data in a secure, compliant manner with transparency and trust between departments, clouds, and partners. The architectural patterns we discussed above don't exist in isolation. You can mix and match them to implement the more complex operational data-sharing architecture in your organizations.

However, it is your responsibility to foster a data-sharing culture, rather than a data-ownership culture, by identifying and addressing the emotional impacts and inherent biases that impede data sharing.

Set up a meeting to learn more about how to bridge those silos, or request your proof of concept.