Colmena de Big Data

Uno. Conceptos básicos de Hive

1.What is Hive
  1.Hive is a data warehouse based on Hadoop
  2.A large-scale data that can be stored, queried, and analyzed in Hadoop
  3.Hive defines a simple SQL-like query language called HQL,It allows users familiar with sql to query data
  4.Allow developers familiar with MapReduce to develop custom mappers and reducers to handle complex analysis tasks that the built-in mappers and reducers cannot complete
  5.No special data format
   Essentially, it converts HQL into a MapReduce program

2.Hive data storage
       Hive is built on Hadoop, and all data is stored on HDFS
       The database can save the data on the block device or in the local file system

Dos. Instalación y configuración de Colmenas

1.Install the dependencies required to start Hive
      a.jdk1.6the above 
      b.Hadoop to start
      
2.Required jar package
      a.wget http://apache.cs.utah.edu/hive/hive-storage-2.7.2/hive-storage-2.7.2.tar.gz    http://archive.apache.org/dist/hive/        
      b.mysql-libs.zip
      
3.Install   
   a.Unzip
   b.Configure environment variables      
     1.Command mode
        export HIVE_HOME= /home/jayliu/hive-0.11.0
        export PATH = $HIVE_HOME/bin:$PATH
        source /etc/profile              
     2.How to edit and save files  /etc/profile file  
   c.Startup and test

4.Hive configuration mysql
    a.Edit hive-site.xml file,Four information: url/drive/account number/Password 
    b.hive/There is a mysql-driven jar package in the lib folder
    
  <configuration>
     <property>
          <name>javax.jdo.option.ConnectionURL</name>   
          <value>jdbc:mysql://16.187.94.183:3306/hive?characterEncoding=UTF-8</value>
     </property>
     <property>
         <name>javax.jdo.option.ConnectionDriverName</name>
         <value>com.mysql.jdbc.Driver</value>
     </property>
     <property>
        <name>javax.jdo.option.ConnectionUserName</name>
        <value>root</value>
     </property>
     <property>
         <name>javax.jdo.option.ConnectionPassword</name>
         <value>mysql</value>
     </property>

Tres. Tipo de datos Hie

a.Basic data type (4 2 1 1）
    4：  TINYINT   SMALLINT   INT    BIGINT
    2:   FLOAT     DOUBLE
    1:   BOOLEAN
    1    STRING
    
b.Compound type
    ARRAY:  Ordered, the field types must be the same  Array(1,2)
    MAP:      Unordered, key-value pairs           MAP('a',1,'b',2)
    STRUCT:The field type can be different     Struct('a', 1, 1.2)

Cuatro. Definición de datos DDL

Cincos. Manipulación de datos DML

Seis. Preguntar

Siete. Función

Ocho. Compresión y almacenamiento

Nueve. Ajuste a nivel empresarial

Diez. El combate real del audio y el vídeo de granos

Once. Errores y soluciones comunes