Thursday, May 26, 2016

PostgresSql configuration for SQOOP2




Please follow below steps to configure the PostgresSql for Sqoop2.
(Sqoop2 will store it's metastore in PostgresSql.)
This one is specific to MapR-cluster  environment.

Required Steps:
Step 1:
Install the postgresql using below command
$ yum install postgresql-server

Step 2:
Start the postgresql service using below command
$ service postgresql initdb

Step 3: Change the parameter in the below specified file
$ vim /var/lib/pgsql/data/postgresql.conf
listen_addresses = <10.10.71.19 >
#Note : add IP of postgresql IP where it has installed.

Step 4:
Add parameters to below specified file
$ vim /var/lib/pgsql/data/pg_hba.conf

# "local" is for Unix domain socket connections only
#local all all ident
local all all trust
# IPv4 local connections:
#host all all 127.0.0.1/32 trust
host all all 10.10.72.78/32 trust

# IPv6 local connections:
host all all ::1/128 ident

Step 5:
Comment existing below parameters in the below and add new parameters values into the specified file.
$ vi /opt/mapr/sqoop/sqoop-2.0.0/server/conf/sqoop.properties

org.apache.sqoop.repository.jdbc.handler=org.apache.sqoop.repository.postgresql.PostgresqlRepositoryHandler
org.apache.sqoop.repository.jdbc.transaction.isolation=READ_COMMITTED
org.apache.sqoop.repository.jdbc.maximum.connections=10
org.apache.sqoop.repository.jdbc.url=jdbc:postgresql://10.10.72.110:5432/sqoop
org.apache.sqoop.repository.jdbc.driver=org.postgresql.Driver
org.apache.sqoop.repository.jdbc.user=sqoop
org.apache.sqoop.repository.jdbc.password=sqoop
#org.apache.sqoop.repository.jdbc.properties.property=value


Step 6:
Download Jar and place it into below path.
Downloadablw link:
Place the downloaded jar into this location.
/opt/mapr/sqoop/sqoop-2.0.0/lib

Step 7: Ecxecute below command.
$ chkconfig postgresql on

Step 8:
Start the postgresql shell using below command
$ psql -U postgres

Step 9:
Create a table using below command.

$ CREATE ROLE sqoop LOGIN ENCRYPTED PASSWORD 'sqoop'
NOSUPERUSER INHERIT CREATEDB NOCREATEROLE;

$ CREATE DATABASE "sqoop" WITH OWNER = sqoop TABLESPACE = pg_default;

Step 10:
login postgres
$/usr/bin/pg_ctl -D /var/lib/pgsql/data -l logfile start



No comments:

Post a Comment