HighByte Blog
Read company updates and our technology viewpoints here.
|
Read company updates and our technology viewpoints here.
|
Elevate your industrial interoperability: A primer on data Pipelines in the Intelligence Hub3/8/2024
Time to read: 15 minutes In HighByte Intelligence Hub, the Pipelines feature was created to make modeled data consumable by a diverse range of applications and services. With the last few releases of the Intelligence Hub, Pipelines has undergone big changes to further that goal and more. From adding new functionality to refining the UX, Pipelines has swiftly evolved beyond its initial focus on “post-processing” payloads for advanced use cases. It has become a core data engineering capability to solve industrial interoperability problems within the Intelligence Hub. Time to read: 9 minutes I consistently hear that many manufacturers are drowning in data and struggling to make it useful. Why is that? A modern industrial facility can easily produce more than a terabyte of data each day. With a wave of new technologies for artificial intelligence and machine learning coupled with real-time dashboards and prescriptive insights, industrial companies should be seeing huge gains in productivity. Unplanned asset and production line maintenance should be a thing of the past. But we know that is not the case. Access to data does not make it useful. Industrial data is raw and must be made fit for purpose to extract its true value. Furthermore, the tools used to make the data fit for purpose must operate at the scale of an industrial enterprise. For many industrial companies, this is a daunting task requiring alignment of people, process, and technology across a global footprint and supply chain. At HighByte, we’re putting our best foot forward to solve this data architecture and contextualization problem from a technology perspective. But what about people and process? To pull it all together, we recently published a new guide, “Think Big, Start Small, Scale Fast: The Data Engineering Workbook.” The guide provides 10 steps to achieving a scalable data architecture based on the best practices we’ve learned from our customers over the last several years. Time to read: 6 minutes Ever since the first release of HighByte Intelligence Hub in 2020, HighByte has developed the solution to meet the industrial data integration needs of today’s industrial customers and tomorrow’s market requirements. The first version of the Intelligence Hub was a client-based application that collected and published contextualized data to any consuming system. As Industry 4.0 leaders began to integrate more systems and assets into their ecosystems, data consumers needed better data visibility and access. The consumers wanted to be able to see all available information, so they could access exactly what they needed. To deliver this visibility and access, we embedded an MQTT broker into the Intelligence Hub, giving administrators the necessary tools to build a Unified Namespace (UNS) that would allow consumers to easily subscribe to the information they desired. With changing needs in mind, in May 2023, we took the next step in the evolution of the Intelligence Hub, adding the ability to request data on-demand through the Intelligence Hub with a built-in REST Data Server. This addition allows users to request time series, transactional, or master data from systems through a single, simple API. Time to read: 7 minutes You don’t have to agree with environmental policies to know that sustainability is a part of business and life today. Supply chain partners, regulators, customers, and investors are demanding more environmental accountability from manufacturers—and with good cause. According to the International Energy Agency, the manufacturing and power sectors account for 63% of energy-related CO2 emissions worldwide. Progress depends largely upon their success. Thankfully, manufacturing has come a long way since the third industrial revolution that saw a rise in automation and productivity without much consideration for environmental impact. The fourth industrial revolution, or Industry 4.0, has given manufacturers more insight into their operational efficiencies. Network-connected assets provide a real-time lens into performance metrics that go hand-in-hand with more sustainable production. Still, this level of connectedness presents a new challenge: How to manage data more efficiently. Time to read: 7 minutes For the past several months, 55 beta testers in 13 countries have been kicking the tires on HighByte Intelligence Hub version 3.0 and generously providing their feedback. Today, I’m excited to announce this major release is now available. Version 3.0 is a step change for the Intelligence Hub and for the Industrial DataOps market. It raises the bar for what a DataOps solution can be at Enterprise scale. It introduces a powerful new Pipelines builder to curate complex data pipelines. It makes the often-vague concept of the Unified Namespace (UNS) tangible and achievable with an embedded MQTT broker—reducing additional software, cost, and administration overhead for our customers. I sat down with HighByte Chief Product Officer John Harrington to talk about some of these advancements available in Version 3.0, including Pipelines. His thoughts are below. I also provide insights from our partner Goodtech, a deep dive on the embedded broker, a review of new project management capabilities, and more. Time to read: 6 minutes Have you ever watched a press conference when a room full of reporters bark questions at the same time? Typically, the media event host will call on a particular reporter to repeat the question and then move on to the next person in the room. Without some ground rules, an actual conversation couldn’t take place. No one could hear the questions being asked, and few would get any answers. Unfortunately, this same scenario often occurs with industrial data. With so much operational technology (OT) data generated on any given day, manufacturers risk losing critical information in the sea of “data noise” coming from their systems or having to expend vast resources to clean that data in the cloud. Time to read: 8 minutes All eyes are on manufacturing these days. Global leaders see manufacturing as the engine powering a wide range of initiatives—from infrastructure development to energy efficiency. Their focus on industrial growth and sustainability shouldn’t be surprising when you consider that manufacturing accounts for roughly 17% of the global GDP and 23% of direct carbon emissions. The reprioritization of industrial investments around the world is good news for manufacturers. Are you ready for the bad news? Manufacturers lag other sectors by a significant margin when it comes to data management. Enterprise Strategy Group (ESG), a division of TechTarget, surveyed 403 technical and business data professionals at organizations in North America to assess the state of DataOps in 2022. They defined DataOps as “improving the quality, delivery, and management of data and analytics at scale.” The study looked at market maturity, challenges, factors influencing buying and planning decisions, and business benefits among those surveyed. The findings were telling. Time to read: 6 minutes When it comes to data collection, who are you really serving? That objective often gets lost amid the OT/IT alignment discussions. Anyone who has embarked on a digital transformation project is likely familiar with the data silos that exist between their OT and IT departments. But we don’t spend enough time talking about how to make that data usable for the line of business. Our line of business colleagues (and their systems of record) are the ultimate customer. The use of IoT-enabled devices is increasing the availability of operational data. IDC has projected there will be 41.6 billion IoT devices in the field generating 79.4 zettabytes of data by 2025. These devices include machines, sensors, and cameras as well as industrial tools. To truly make that data usable, we need to merge this data with information from other systems and provide context for line of business users. In an industrial environment, these users include quality, maintenance, engineering, R&D, regulatory, and product management. Time to read: 7 minutes The efforts of standards organizations like OPC Foundation, Eclipse Foundation (Sparkplug), ISA, CESMII, and MTConnect represent a significant step forward for the advancement of Industry 4.0 in manufacturing. But industry standards only go so far. Businesses need data to tell the story of what is happening, why it is happening, and how to fix it. Multiple pieces of information must be assembled with other pieces of information from other sources to tell the use case story—just like words must be combined into sentences and sentences combined to form stories. Data standards can’t tell the use case story—they can only provide a dictionary. Standardizing the device-level data into structures is key, but only the beginning. Data standards alone will not solve your interoperability problems because they don’t provide the use case related context you need to make strategic decisions. Here are four key reasons why you still need an Industrial DataOps solution like the Intelligence Hub—even with the introduction or evolution of new standards. Time to read: 10 minutes The promise of Industry 4.0 has many manufacturing leaders thinking big. They envision a future in which real-time access to data opens the door to unprecedented levels of operational flexibility, predictability, and business improvement. For many, early-stage wins often lead to larger projects that stall or fail to scale because their data infrastructure couldn’t support the increasing project complexity. Enter Industrial DataOps. DataOps (data operations) is the orchestration of people, processes, and technology to securely deliver trusted, ready-to-use data to all the systems and people who require it. The first known mention of the term “DataOps” came from technology consultant and InformationWeek contributing editor Lenny Liebmann in a 2014 blog post titled, “DataOps: Why Big Data Infrastructure Matters.” According to Leibmann: “You can’t simply throw data science over the wall and expect operations to deliver the performance you need in the production environment—any more than you can do the same with application code. That’s why DataOps—the discipline that ensures alignment between data science and infrastructure—is as important to Big Data success as DevOps is to application success.” Time to read: 7 minutes I love the chaos of an early market like DataOps for Manufacturing. It’s clear that things are changing, but what technologies and approaches will win out is less obvious. In these types of markets, as a solution provider, it’s equally fun to watch them mature. One sign of a maturing market is the type of questions early customers ask about a solution. At first, the questions are different variations of “Does it work?” or “How is it different than a, b, or c?” as customers try and understand the solution and how it solves their problem. As the market matures, the questions shift focus to technical requirements like “What’s the performance with 10,000x?” or “Does it support high availability?” Here at HighByte we’re seeing more scale and reliability questions in early engagements, a sign that both the market and the product are maturing. That’s why I’m excited to announce some key features in version 2.1 that make HighByte Intelligence Hub more scalable and reliable to fit the needs of your production environment. Time to read: 7 minutes Manufacturers and other industrial companies adopting Industry 4.0 want to make industrial data available at scale across the enterprise to drive business decisions. Yet as these companies connect more processes, systems, and machines, their data modeling and integration needs have become more complex. Industrial DataOps solutions like HighByte Intelligence Hub provide an answer to this complexity. The software provides a dedicated data modeling management and abstraction layer that helps users streamline their data architecture and reduce time to deploy new systems. In fact, as companies have expanded their usage of HighByte Intelligence Hub, they’ve begun to implement deployment architectures beyond a single hub. In a recent poll of HighByte Intelligence Hub users, we asked how many instances they plan to run at a single site. The results validated the demand for a multi-hub architecture: Half of the respondents expect to deploy two to five hubs per site; nearly one-quarter said they plan to use six to 10 hubs per location. Time to read: 6 minutes In my last post, “An intro to industrial data modeling”, I shared my definition of a data model and why data modeling is important for Industry 4.0. I’d like to take that a step further in this post by explaining why you need a dedicated abstraction layer for data modeling to achieve a data infrastructure that can really scale. Time to read: 7 minutes Based on my conversations with more than 500 manufacturing companies and integrators over the past five years, I believe the Industrial Internet of Things (IIoT) will continue to be a paramount part of the manufacturing landscape in 2021. The new year will bring a continued increase in digitalization across enterprises. While we have seen an increase in “digital transformation” initiatives among manufacturing companies for several years, the COVID-19 pandemic and the challenges it created for production, safety, remote access, and supply chain have accelerated the urgency to make digitalization a reality. I also believe IIoT projects will continue to scale because of changes we are seeing in people, processes, and technology. Here are five predictions for 2021.
Time to read: 10 minutes
Most manufacturing companies realize the benefits of leveraging industrial data to improve production and save costs, but they remain challenged as to how to scale-up their pilots and small-scale tests to the plant-wide, multi-plant, or enterprise level. There are many reasons for this including the time and cost of integration projects, the fear of exposing operational systems to cyber-threats, and a lack of skilled human resources.
At the root of all of these problems is the difficulty of integrating data streams across applications in a multi-system and multi-vendor environment, which has required some degree of custom coding and scripting. Standardizing data models, flows, and networks is hard work. Unlike an office environment with its handful of systems and databases, a typical factory can have hundreds of data sources distributed across machine controls, PLCs, sensors, servers, databases, SCADA systems, and historians—just to name a few. Industrial DataOps provides a new approach to data integration and management. It provides a software environment for data documentation, governance, and security from the most granular level of a machine in a factory, up to the line, plant, or enterprise level. Industrial DataOps offers a separate data abstraction layer, or hub, to securely collect data in standard data models for distribution across on-premises and cloud-based applications. These four use cases illustrate how Industrial DataOps can integrate your role-based operational systems with your business IT systems as well as those of outside vendors such as machine builders and service providers.
Time to read: 7 minutes
An executive for an industrial products company once told me even though his factories are full of similar equipment, he still struggled to access meaningful data from the machines. Each one of the plastic injection molding machines had a different way of presenting the data. That meant the company needed to customize coding for every piece of equipment to obtain meaningful insights.
It’s a common scenario in many industrial environments, where plants may have hundreds of PLCs and machine controllers on disparate machines generating operational data that is unintelligible to the data scientists who must make sense of it. This is where Industrial DataOps comes in. It provides a way to standardize data using common models, or object-oriented approaches, to integrate and manage information coming from multiple sources. Here’s a closer look at the top six signs it’s time to consider an Industrial DataOps architecture for your company.
Time to read: 14 minutes
If you know me well, then you’ve probably heard me say words matter. A shared vocabulary—and a shared understanding of a word’s meaning—is a simple but powerful tool when two bodies approach a problem from different perspectives.
Two bodies that often approach problems, projects, and process from different perspectives are IT and Operations Technology (OT). While the industrial automation community has been writing and discussing the necessity of IT-OT convergence for nearly a decade, this functional collaboration still remains a stumbling block for many industrial companies on their Industry 4.0 journeys. The good news is that the emerging concept of Industrial DataOps can provide some common ground. DataOps is a new approach to data integration and security that aims to improve data quality and reduce time spent preparing data for use throughout the enterprise. Industrial DataOps provides a toolset—and a mindset—for OT to establish “data contracts” with IT. By using an Industrial DataOps solution, OT is empowered to model, transform, and share plant floor data with IT systems without the integration and security concerns that have long vexed the collaboration. If we see the value in IT-OT collaboration, the first step is getting these functions to speak the same language. This post aims to document key terms surrounding Industrial DataOps and provide IT and OT with a common dictionary. Some of these definitions are more technical in nature and others are more business oriented. Let’s dive in.
Time to read: 4 minutes
In my last post, “Seven steps to making your industrial data fit for purpose”, I briefly covered seven steps that are critical for manufacturers looking to scale their IIoT projects and wrangle data governance. I’d like to use this post to dive deeper on step 4, selecting your integration architecture, which requires diligence during IIoT planning.
Integration architectures fall in two camps: direct application programming interface (API) connections (application-to-application) or integration hubs (DataOps solutions). Time to read: 8 minutes The manufacturing industry is experiencing a change so significant it has earned the title of Fourth Industrial Revolution. This transformation was kick-started by the need to become more data driven and then fueled by a number of recent technological advances. Early adopters in factories around the world recognize that industrial data—operations data coming from machines, processes, products, and systems on the plant floor—is gold. More users and systems want access to this data in real time to convert it into valuable information they can act on to predict machine failure, prevent downtime, and improve product quality. In fact, IDC recently projected that there will be 41.6 billion IoT devices in the field generating 79.4 zettabytes of data by 2025. These devices include machines, sensors, and cameras as well as industrial tools. It’s an immense, even overwhelming, volume of data. How can companies leverage it effectively? |
|