tcVISION in the Cloud – The right data at the right time – in the cloud as well.
The Challenge
In the digital age BigData and Analytics are widely being talked about. The efficient integration of mainframe data has multiple reasons: BigData, workload relief of the mainframe to reduce costs, application modernization together with data and application migration without a “Big Bang”. Declining mainframe knowhow, databases and their complexities grown over decades – these are big challenges when it comes to data exchange in a mainframe environment.
Many corporations used FTP or other file transfer solutions to resolve the data exchange problem. This is extremely CPU intensive, insecure and creates high costs for the distribution of mass data. Nowadays these solutions are neither acceptable nor practicable anymore. Not only because of the costs, but also because of limited batch windows.
The Solution
There is a better solution to manage and master this complex and ambitious task in an easy, fast, reliable and efficient way: tcVISION for the timely, bidirectional data synchronization and replication based on changed data. tcVISION turns the data exchange into a single-step operation. No middleware or message queueing is required. The data is exchanged in raw format, compressed and reduced to the processing of changed data.
Benefits
- The focus is on changed data (Changed Data Capture), so the data transfer volume is reduced to a minimum.
- Highest topicality through continuous real-time processing
- Cost reduction through relocation of data exchange processes from the mainframe to cost efficient platforms (e.g. MS Azure, amazon cloud, others)
- Cost reduction through compressed data transfers
- High integration potential of the tcVISION solution: Multiple Change Data Capture technologies can be used depending on change frequencies and latency times
- Comfortable data mapping
- Prevention of mainframe costs: Integrated data repository creates transparency for all available data
- No additional middleware required – elimination of costs and implementation efforts – efficient transport layer
- Elimination of programming efforts for data transfers
- Comprehensive conversion of historically developed data structures
- Integrated pooling/streaming processes avoid programming efforts and message queueing to prevent data loss because of unavailability of the target system or delays
- Processes which have proven to work in practice are available to restart a replication after system failures (database errors, transmission errors, etc.)
Application Examples
- Synchronization of data in a heterogeneous system environment consisting of a mainframe and mobile applications which make use of data from cloud solutions
- Gradual migration of data and applications in in cloud based solutions
- Transfer of database data to various databases hosted by the cloud solution
- Provision of mainframe data in cloud systems for Data Warehousing, ETL, Business Intelligence, Analytics & BigData
Intuitive mapping offers comprehensive possibilities for data modeling and a variety of integrated functions for the transformation of data types between mainframes and databases in – in this example here: MS Azure – including for example packed fields on the mainframe. The complete change of the data model is no problem for tcVISION – even in a bidirectional replication.
tcVISION for Cloud Solutions – Facts
- Available for Windows, Linux and UNIX, depending on the cloud system
- Databases supported in cloud systems: IBM DB2 LUW, IBM BLU Acceleration, IBM Informix, IBM NETEZZA, Oracle, Sybase, Microsoft SQL Server, Software AG Adabas LUW, PostgreSQL, Teradata, MongoDB, Flat File Integration, SAP Hana, EXASOL, MySQL / MariaDB, JSON / Avro, ODBC, Kafka, Hadoop Data Lakes, HDFS, CSV
- Near real-time processing of the active or archived log files to capture the changed data
- Change capture processing uses standardized interfaces
- Support of all Change Data Capture methods:
- DBMS Extension
- Log Processing
- Batch Compare
- Loads
- Comprehensive mapping functions for the creation of structure information for all supported databases running in the cloud solution
- Central, relational repository for storage of the metadata, linkage and processing rules
- No additional middleware required
- No message queueing required
- Compressed and efficient data transfer
- Restart/Recovery guaranteed after system failures
- Support of DBMS backup formats (archive log files) for the efficient exchange of bulk data without the necessity of accessing production data
- Support of unidirectional, bidirectional and 1:1, 1:n, n:1 and n:n replications
- Convenient single-step-operations. Capturing of change data – transformation of data – application to target systems
- Integrated workload balancing to shift tasks like processing and conversion to more cost-effective systems (e.g. from mainframe to cloud system).
- Parallelization of load processes in order to realize real low latency synchronization solutions
- Integrated features for direct application of data into target systems
- Powerful load functions to transfer and apply bulk data
- Integrated pooling and streaming processes prevent data loss if the target cannot receive data or receives slower than the source is sending.
- Integrated loopback prevention for bidirectional data exchange
- Built-in key management for non-indexed data
- Extensive monitoring, logging and integrated alert notification