Netezza Twinfin is the advanced analytics and warehousing solution provided by IBM. It currently has been rebranded as IBM Puredata for analytics (PDA).

• It was an offering from a company known as Netezza launched in 1999 and then got acquired by IBM in the year 2010. Ever since it has been developed as a subsidiary of IBM.
• It is based on the AMPP (asymmetric massively parallel processing) architecture which has an SMP frontend to get the queries from the client and communicate with the MPP backend to do the processing
• IBM Netezza Analytics’ advanced technology supports data warehousing and in-database analytics into a scalable, high-performance, massively parallel advanced analytic platform that is designed to work with petascale data volumes.

Netezza utilizes a restrictive design called Asymmetric Massively Parallel Processing (AMPP) which joins the enormous information processing proficiency of Massively Parallel Processing (MPP) where nothing (CPU, memory, stockpiling) is shared and symmetric multiprocessing to arrange the equal processing. The MPP is accomplished through an array of S-Blades which are workers on its own running its own working frameworks associated with plates. While there might be different items which follow comparable design, one extraordinary equipment part utilized by Netezza called the Database Accelerator card which is joined to the S-Blades. These quickening agent cards can play out a portion of the question processing stages while information is being perused from the circle rather than the processing being done in the CPU. Moving huge measure of information from the circle to the CPU and playing out all the phases of question processing in the CPU is one of the significant bottlenecks in the huge numbers of the data set administration frameworks utilized for information warehousing and investigation use cases.

The fundamental equipment segments of the Netezza machine are a host which is a Linux worker, which can convey to an array of S-Blades every one of which has 8 processor centers and 16 GB of RAM running Linux working framework. Every processor in the S-Blade is associated with plates in a circle array through a Database Accelerator card which utilizes FPGA innovation. Host is additionally liable for all the customer collaborations to the apparatus like dealing with information base questions, meetings and so on alongside dealing with the meta-information about the items like data set, tables and so on put away in the apparatus. The S-Blades among themselves and to the host can convey through an exclusively fabricated IP based superior organization.

Netezza S-Blades

The S-Blades are likewise alluded as Snippet Processing Array or SPA in short and every CPU in the S-Blades joined with the Database Accelerator card appended to the CPU is alluded as a Snippet Processor.

Let us use the example of a Data Warehouse for a huge retail firm and one of the tables store the insights concerning the entirety of its 10 million clients. Likewise expect that there are 25 columns in the tables and the absolute length of each table column is 250 bytes. In Netezza the 10 million client records will be stored fairly equally across all the disks available in the disk arrays connected to the snippet processors in the S-Blades in a compressed form. At the point when an user queries for state Customer Id, Name and State who joined the retail firm in a specific period arranged by state and name, the below is how the processing will occur:

• The host gets the query, parses and confirms the question, makes the code to be executed to by the snippet processors in the S-Blades and passes the code for the S-Blades
• The snippet processors execute the code and as a feature of the execution, the information block which stores the information needed to fulfill the inquiry in a compacted structure from the circle connected to the snippet processor will be added something extra to memory. The Database Accelerator card in the snippet processor will un-pack the information which will remember all the sections for the table, at that point it will eliminate the undesirable segments from the information which in the event that will be 22 segments for example 220 bytes out of the 250 bytes, applies the where proviso which will eliminate the undesirable columns from the information and passes the modest quantity of the information to the CPU in the snippet processor. In customary information bases every one of these means are acted in the CPU.
• The CPU in the snippet processor performs assignments like accumulation, whole, sort and so forth on the information from the information base quickening agent card and parses the outcome to the host through the organization.
• The host combines the outcomes from all the S-Blades and plays out extra advances like arranging or accumulation on the information prior to imparting back the end-product to the customer.

Conclusion

• The Netezza can deal with huge volume of information in equal and the key is to ensure that the information is disseminated properly to use the enormous equal processing.
• Execute plans such that the vast majority of the processing occurs in the snippet processors; limit correspondence between snippet processors and insignificant information correspondence to the host.

Meet Ananth Tirumanur. Hi there 👋

I work on projects in data science, big data, data engineering, data modeling, software engineering, and system design.

• 👨‍💻 All of my projects are available at https://github.com/akrish1982
• 💬 Ask me about Data engineering, SQL, Databases, Data pipelines, Data infrastructure
• 📄 My work history: https://www.linkedin.com/in/ananthtirumanur/
• ⚡ Fun fact: Marathoner & Casual Birding enthusiast

Connect with me:

• Twitter: @akrish82
• LinkedIn: https://linkedin.com/in/ananthtirumanur/

My Resources:

• LinkedIn Newsletter: Data Engineering with AWS
• Udemy Course: AWS Certified Data Engineer Associate Practice Test
• Python Crash Course: Python Crash Course on Gumroad

Languages and Tools:

AWS, Bash, Docker, Elasticsearch, Git, Grafana, Hadoop, Hive, EMR, Glue, Athena, Lambda, Step Functions, Airflow/MWAA, DynamoDB, Kafka, Kubernetes, Linux, MariaDB, MySQL, Pandas, PostgreSQL, Python, Redis, Scala, SQLite