Qlik Compose Data Lakes

What is Qlik Compose Data Lakes?

Automate analytics-ready data pipelines

Qlik Compose for Data Lakes (formerly Attunity Compose) automates your data pipelines to create analysis-ready datasets. By automating data ingestion, schema creation, and continuous updates, organizations get a faster appreciation time for existing data lake investments.

Universal data ingestion

With support for one of the widest ranges of data sources, Qlik Compose for Data Lakes ingests data in your data lake, whether on-premises, in the cloud, or in a hybrid environment. Sources include:

• RDBMS: DB2, MySQL, Oracle, PostgreSQL, SQL Server, Sybase
• Data warehouses: Exadata, IBM Netezza, Pivotal, Teradata, Vertica
• Hadoop: Hadoop Apache, Cloudera, Hortonworks, MapR
• Cloud: Amazon Web Services, Microsoft Azure, Google Cloud
• Messaging systems: Apache Kafka
• Enterprise applications: SAP
• Legacy systems: DB2 z / OS, IMS / DB, RMS, VSAM

Easy structuring and transformation of data

An intuitive, guided user interface helps you create, model, and run data lake pipelines.

• Automatically generate Hive Catalog schemas and frameworks for operational data storage (ODS) and historical data storages (HDS) without manual coding.

Continuous updates

Make sure your ODS and HDS accurately represent your source systems.

• Use changed data capture (CDC) to enable real-time analytics with less administrative and processing overhead.
• Take advantage of time-based partitioning with transactional consistency to ensure that only transactions completed within a specified time are processed.

Take advantage of the latest technology

Take advantage of the advances of Hive SQL and Apache Spark, including:

• The latest advances in Hive SQL, including the ACID MERGE operation, efficiently process data inserts, updates, and deletions, ensuring data integrity.
• Pushdown processing for Hadoop or Spark engines. The automatically generated transformation logic is transferred to Hadoop or Spark for processing as the data flows through the pipeline.

Historical data storage

Derive analysis-specific datasets from a complete historical data store (HDS).

• New rows are automatically attached to the HDS when data updates arrive from the source systems.
• New HDS records are automatically marked over time, allowing the creation of trend analysis and other time-oriented analytical data marts.

System requirements:

CPU: 1.8 GHz dual core processor

RAM: 2 GB

HDD / SSD: 50 GB of free space on the main drive

Operating system: Microsoft Windows 8 32-bit, Microsoft Windows 7 32-bit and Microsoft Windows Vista 32-bit

Resolution: 1024×768