SQOOP IMPORTS AND EXPORTS

Home Forums Technical Discussions Bigdata SQOOP IMPORTS AND EXPORTS

This topic contains 0 replies, has 1 voice, and was last updated by  Vinoth Sekar 5 years, 1 month ago.

  • Author
    Posts
  • #336

    Vinoth Sekar
    Keymaster

    sqoop gets the metadata information from the RDBMS.

    USing that metadata information it generates the javafile and class file and jdk compiles converts into jar files.

    RDBMS to HDFS :

    1.sqoop import –connect jdbc:mysql://localhost/test –table employee -m 1

    setting target-dir:

    2.sqoop import –connect jdbc:mysql://localhost/test –table employee –target-dir /practice -m 1

    setting filter :

    3.sqoop import –connect jdbc:mysql://localhost/test –table employee –target-dir /practice_where –where “eid between 2 and 3″ -m 1

    warehouse-dir

    4.sqoop import –connect jdbc:mysql://localhost/test –table employee –warehouse-dir
    /practice -m 1

    incremental append :

    5.sqoop import –connect jdbc:mysql://localhost/test –table employee –warehouse-dir /practice_where –check-column eid –last-value 3 –incremental append -m 1

    Running query:

    6.sqoop import –connect jdbc:mysql://localhost/test –query ‘select * from customer where $CONDITIONS’ –target-dir /user/cloudera/customer_bkp -m 1

    query must include the token $CONDITIONS which each Sqoop process will replace with a unique condition expression.

    Importing data into hive :

    create-hive-table

    7.sqoop create-hive-table –connect jdbc:mysql://localhost/test –table employee –hive-table emp_bkp

    hive-import

    8.sqoop import –connect –table employee –hive-table emps –hive-import

    Exporting of data from HDFS to MYSQL :

    Inserting data into a table

    9. sqoop export –connect jdbc:mysql://localhost/test –table employee -m 1 –export-dir ‘/hdfsfilename’

    updating data into a table

    10. sqoop export –connect jdbc:mysql://localhost/test –table employee –export-dir ‘/hdfsfilename’ –update-key cust_id

    Sqoop eval command

    11. sqoop eval –connect jdbc:mysql://localhost/test –query “select * from employee”;
    12. sqoop eval –connect jdbc:mysql://localhost/test –query “select * from employee limit 1″;
    13. sqoop eval –connect jdbc:mysql://localhost/test –query “insert into employee values(3,’lak’)”;

    sqoop list tables or listing tables using sqoop

    14. sqoop list-tables –connect jdbc:mysql://localhost/test

    sqoop list databases

    15. sqoop list-databases –connect jdbc:mysql://localhost/test

You must be logged in to reply to this topic.