Data Development

An agile, one-stop big data development platform built on batch–stream and lake–house architecture. Unifies real-time and offline data with one-stop development and O&M, full-process visualization, high performance, and high availability for warehousing and large-scale analytics.

An agile, one-stop big data development platform built on batch–stream and lake–house architecture. Unifies real-time and offline data with end-to-end visualization, high performance, and high availability for warehousing and analytics workloads.

Product Advantages

Consult Now

Full Process Visualization Intelligent Development

One platform covers the entire data development process, and the whole process is visualized with syntax hints to let users get started quickly. Self-developed scheduling system, which can support daily instance scheduling of tens of millions of instances.

Open and Compatible Development Environment

Supports multiple types and versions of big data docks, including cloud data warehouses (AWS Redshift, etc.), traditional data warehouses (Oracle, etc.), MPP data warehouses (OceanBase, StarRocks, Doris, etc.), Hadoop (CDH, TDH, Mrs, etc.), and hybrid.

Seamless Integration of Lake Warehouse Capabilities

Unify the flow and batch table from the table format layer, integrate various engine capabilities, and realize the integration of data into the lake, and build a warehouse in the lake to integrate the batch flow of the entire data link.

Core Capabilities

Data Integration

Provides a reliable, secure, low-cost, elastically scalable data synchronization platform across heterogeneous data storage system capabilities. It also provides data synchronization channels under different network environments, supports configuration, debugging, logging, version management and other functions, and achieves rapid orchestration of data integration processes through visual configuration. It has real-time synchronization, offline synchronization, and full library synchronization capabilities, while achieving full and incremental synchronization.

Data Development IDE

Set batch stream development as a whole, provide visual code editor, support rapid code development, task orchestration, run query, error handling and performance optimization and other complex operations, preset SQL, Spark, Flink, Shell, Python, machine learning and other task types, form workflow development by drag and drop, complete complex data analysis and processing.

Task Scheduling

Provide single scheduling and cycle scheduling. Cycle scheduling supports visual configuration of multiple scheduling cycles in minutes, hours, days, weeks, months, and years, and supports same-cycle dependency and cross-cycle dependency. And a variety of system scheduling parameters are built into the development kit, which can dynamically adapt to changes in the environment, meet the task configuration needs of most scenarios, and support flexible custom parameter configuration. The self-developed task scheduling engine supports up to 10 million tasks per virtual machine, making data processing smoother.

Intelligent O&M

Fully managed operation and maintenance monitoring capabilities, providing functions such as daily operation and maintenance of tasks, task operation monitoring, and abnormal alarms. It also provides baseline early warning, baseline diagnosis and other capabilities to help operation and maintenance personnel perceive the risk of instance running timeout in advance, and quickly locate problems through the automatic diagnosis function. Provide different operation and maintenance management capabilities based on the different execution characteristics of offline and real-time tasks.

Core Capabilities

Provides a reliable, secure, low-cost, elastically scalable data synchronization platform across heterogeneous data storage system capabilities. It also provides data synchronization channels under different network environments, supports configuration, debugging, logging, version management and other functions, and achieves rapid orchestration of data integration processes through visual configuration. It has real-time synchronization, offline synchronization, and full library synchronization capabilities, while achieving full and incremental synchronization.

Application Scenarios

Consult Now

Enterprise Data Warehouse Build

Business Pain Points

The data is scattered, the system is isolated to a high degree, and a variety of permissions are controlled, resulting in a high threshold for data use and a low degree of application.

There are problems such as complex data types, inconsistent structures, and inconsistent temporal granularity, and there are obstacles to data fusion.

Business Value

It has one-stop construction data integration, development, operation and maintenance, security and other capabilities, providing infrastructure for enterprises to build global data warehouses, and using global data to drive business, forming a closed loop for business and data connectivity, so that data can play a greater value.