Sunday, March 9, 2025

Hadoop/Hive

The term bigdata is used to refer large volumes of data. Using the traditional way to handle this data is very difficult. So, the Apache software has created a framework called hadoop to handle these problems.

Hadoop is the open-source framework to stored and process bigdata. It contains two modules Hadoop Distributed file system (HDFS) and MapReduce  

MapReduce: It is used to process the large volumes or structured, semi-structured and unstructured data.

HDFS : It is used to store the data.

There are certain tools like Sqoop, Pig, and Hive helps hadoop modules.

Hive is not a Relational data base. It is query engine.

Import Command is used to import data from DBMS to hive and Exports command is used to export data into specific location.



at
Labels:

No comments:

Post a Comment

Subscribe to: Post Comments (Atom)

Partitioning

Hive organizes the tables into partitions, and partitions are subdivided into buckets. ALTER TABLE table_name ADD [IF NOT EXISTS] PARTITION ...

  • Update Strategy Transformation
    Update Strategy Transformation is Connected and Active transformation. It is used to insert, update, delete and reject records from a file ...
  • Real Time Scenarios
    1) A flat file is arrived on the server and workflow will be launched using 'PMCMD' command. After some time another flat file arri...
  • Session parameters & Session variables
    Session Parameter :- ---------------------- Session parameters, like mapping parameters, represent values you might want to change betwee...