HighByte Intelligence Hub version 4.1 brings powerful new features to the Industrial DataOps space. The latest release enables industrial companies to reach deeper into their operations and create data services. It focuses on engineering request-based data pipelines, data virtualization, and advanced file processing.
Version 4.1 advances Pipelines with the introduction of “Callable Pipelines” along with new observability capabilities that feature visual before-and-after payload inspection for debugging configuration. Callable Pipelines enable you to compose data processing that is invoked within the Intelligence Hub or through an API. When used alongside the REST Data Server, Callable Pipelines functionally serve as an “Industrial Data API Builder.” This allows you to define custom API endpoints to interact with industrial data.
In addition to Pipelines, version 4.1 delivers new native integrations with Aspen InfoPlus.21 and Amazon S3 Tables, granular audit logging for regulated industries, and seamless event capture from file-based data sources. Let’s explore these new capabilities in more detail.
Creating Data Services
When an application, service, or team needs data, you must consider not just how data is formed but also how it is delivered and accessed. If it’s for an ERP application, data is often delivered by transacting with a set of APIs. If it’s for analytics, data is delivered to a data warehouse or data lake that is queried by its users. If it’s for event-driven services, data is published to a message broker that consuming nodes subscribe to. So far, all these scenarios involve pushing data to a destination and accessing data there.
Conversely, there are integration patterns where pushing data is not preferable. Some use cases call for more dynamic or ad-hoc data access. This could involve applications that don’t know what data they need in advance. It can also be impractical to move data to another system if the datasets are large or networks are constrained. Without a defined access pattern, collecting data locally can be more efficient and agile. In these scenarios, it makes sense for the data consumer to pull data. Examples include a QMS that requires inspection data at a specific phase in a batch or an MES that requires certain external conditions to be met before proceeding with a process.
In HighByte Intelligence Hub version 4.1, you can create data services with the introduction of “Callable Pipelines."
Industrial Data API Builder
Pipelines in the Intelligence Hub were initially dependent on Flows to trigger them. In version 4.0, Pipelines became self-triggering with the ability to trigger based on a time interval or an event. In version 4.1, Pipelines are now “callable”. This encapsulates and abstracts data processing within a Pipeline as a simple request and response service.
Within the Intelligence Hub, Pipelines can call other Pipelines to enable re-use and sharing of data processing. Externally, Pipelines can be exposed to the REST Data Server, enabling highly customized data virtualization and on-demand data processing. External data consumers can securely issue requests on the Intelligence Hub to push or pull data through a Pipeline. With this powerful new feature, you can build custom APIs for your operations and industrial data sources.
Figure 1: Create data services with Pipelines and expose them with the REST Data Server in HighByte Intelligence Hub version 4.1.
With the API builder for industrial data, users can:
- Build bespoke APIs for their factories and business processes
- Provide programmatic access to ERP or other applications without compromising on security or performance objectives
- Create simplified interfaces for line-of-business users and business intelligence tools to extract data from sophisticated historians like AVEVA PI System, Aspen IP.21, and Honeywell PHD
- Create compatible interfaces for assets and instrumentation for MES solutions such as Körber PAS-X
- Create Unified Namespace services for providing historized data and facilitating transactions
See the Industrial Data API Builder in action in this short demo video.
Pipeline Development Environment
As Pipelines continue to advance in capability and play a central role in the Intelligence Hub, building observability into the Pipeline Graph to accelerate development and debugging has become essential. 
Figure 2: A view, in dark mode, of the new Pipeline Debug capabilities in HighByte Intelligence Hub version 4.1.
Complementing existing observability features such as Pipeline Replay, Intelligence Hub version 4.1 introduces Pipeline Debug, a new capability to perform individual Pipeline Stages testing. Pipeline Debug allows you to test the impact of different configurations and incoming event payloads within a Pipeline. You can visually interrogate and compare the before-and-after of a payload at each Pipeline Stage, streamlining pipeline development. Taking advantage of this new feature as part of your pipeline creation and management workflow makes Pipelines even easier to configure and troubleshoot.
Watch a demo of Pipeline Debug Mode.
Connectivity to Call Out
File connection enhancements
In version 4.1 we’ve improved handling of text-based files and related use cases for the File connection. Text files can now be directly read and decoded at the input level. With browsing, you can remotely see file contents and create inputs/outputs without the need to log into a data source. Inputs can be set to read directories and file metadata, allowing you to build Pipelines to extract and process files based on recent changes, size, date created, and more. Version 4.1 also introduces support for Secure File Transport Protocol (SFTP) to exchange data with SSH endpoints.
Direct integration with Amazon S3 Tables
Amazon S3 Tables is a new feature within S3 that optimizes the management, performance, and storage of tabular data for analytical workloads, offered as a managed data lake service based on the Apache Iceberg and Parquet formats.
HighByte Intelligence Hub version 4.1 includes direct integration with Amazon S3 Tables. This makes building an open, transactional data lake for industrial data easy, allowing the Intelligence Hub to interact with S3 Tables directly—without any intermediate processing or staging tools. It can create tables, evolve schema, and insert records, making it a helpful addition for joint HighByte and AWS customers.
Native connectivity to IP.21
Aspen InfoPlus.21 (IP.21) is a popular historian in the energy, life sciences, and food & beverage verticals. Version 4.1 of the Intelligence Hub introduces a native integration with IP.21, capable of browsing schema, such as tags and definitions, to drag-and-drop them into queries. Now you can build sophisticated inputs with dynamic parameters for the IP.21 SQLPlus query engine.
Granular Audit Logging
HighByte Intelligence Hub version 4.1 offers enterprise administration capabilities that meet the growing demand for scalable, secure, and compliant data infrastructure in highly regulated industries.
“The latest release of HighByte Intelligence Hub directly addresses these critical needs, reinforcing its role as a key enabler of manufacturing intelligence, real time data driven decision making, and the enhancement of existing data systems,” said Paul Power, Chief Executive Officer at Deep Digital Solutions Group, a HighByte integration partner.
Common among these industries are formal practices around change management, traceability, and accountability. Auditing plays a key role in these practices, demanding the use of systems that can capture granular event detail.
To support these critical environments, we’ve overhauled audit logging in the Intelligence Hub. When configuration elements are updated through the user interface or programmatically, audit logging plays an integral role in the process.
Not only are more changes audited, but the events are logged with greater detail. You can see when a change was made, what user made the change, what was changed, the before-and-after state, and any cascading changes on related configurations. For example, if a user renames a Connection, this as well as its impact on other configuration elements, such as a Pipeline using the connection, would be logged.
Figure 3: A view of the new Audit Log in HighByte Intelligence Hub version 4.1.
To ensure the integrity of the audit log, its contents are stored in a dedicated encrypted file. The audit log can be viewed and queried, with or without runtime events, from the Intelligence Hub UI and can also be monitored by external observability platforms.
Wrap Up
With a commitment to innovation, version 4.1 of the Intelligence Hub continues to push the boundaries of what is possible and should be expected of an enterprise Industrial DataOps solution. New engineering capabilities and improved features for heavily regulated industries make version 4.1 an important and valuable step up for Intelligence Hub users.
To learn more, please check out these additional resources:
- Read the product announcement
- Explore the release notes for details on all new features and fixes
- See the Industrial Data API Builder in action
- Watch a demo of Pipeline Debug Mode
- Request a free trial or log in to your existing account to test and deploy the software in your unique environment
