serverToHdfs

Definition

์„œ๋ฒ„์— ์œ„์น˜ํ•œ ํŒŒ์ผ์„ HDFS๋กœ ๋ณต์‚ฌํ•˜๋Š” ๋…ธ๋“œ์ž…๋‹ˆ๋‹ค.
์ขŒ์ธก [Flow๊ตฌ์„ฑ]๋…ธ๋“œ ์ค‘ [serverToHdfs]๋…ธ๋“œ๋ฅผ drag & drop ํ•œ ํ›„ Property ํ•ญ๋ชฉ์„ ์ž…๋ ฅํ•ฉ๋‹ˆ๋‹ค.

Set

[setting], [scheduler], [parameter] ์„ค์ •์€ [์›Œํฌํ”Œ๋กœ์šฐ] > [์ƒ์„ฑ] > [๊ธฐ๋ณธ๊ตฌ์„ฑ]์„ ์ฐธ๊ณ ํ•ฉ๋‹ˆ๋‹ค.

property

[Node Description] ์ž‘์„ฑ ์ค‘์ธ ๋…ธ๋“œ๋ช… ์ž…๋ ฅ

servertohdfs01

property ์ค‘ 1,4,5 ํ•ญ๋ชฉ์€ [์—ฐ๊ฒฐ๊ด€๋ฆฌ]์— ๋“ฑ๋ก๋œ ์›๊ฒฉ์„œ๋ฒ„ ์ •๋ณด ์„ ํƒ ์‹œ ์ž๋™์œผ๋กœ ์ž…๋ ฅ๋˜๋ฉฐ, ์›ํ•˜๋Š” ์„œ๋ฒ„๋ชฉ๋ก์ด ์—†์„ ๊ฒฝ์šฐ [์„ค์ •] > [์›๊ฒฉ ์„œ๋ฒ„]์—์„œ ํ•„์š”์„œ๋ฒ„๋ฅผ ๋จผ์ € ๋“ฑ๋กํ•ฉ๋‹ˆ๋‹ค.

  1. ์›๊ฒฉ ์„œ๋ฒ„ IP : [์„ค์ •] > [์›๊ฒฉ ์„œ๋ฒ„]์— ์ €์žฅ๋œ ์„œ๋ฒ„ IP, PORT
  2. sourceFolder : source ํด๋” ๊ฒฝ๋กœ ex) /home/dpcore
  3. sourceFile : source ํŒŒ์ผ ๋ช… ex) target.csv
  4. user : [์„ค์ •] > [์›๊ฒฉ ์„œ๋ฒ„]์— ์ €์žฅ๋œ ์„œ๋ฒ„ SSH User ID
  5. password : [์„ค์ •] > [์›๊ฒฉ ์„œ๋ฒ„]์— ์ €์žฅ๋œ ์„œ๋ฒ„ SSH User Password
  6. hdfsFilePath : ์ €์žฅํ•  HDFS ๊ฒฝ๋กœ ex) /tmp

Example

์›๊ฒฉ์„œ๋ฒ„ /home/dpcore ๊ฒฝ๋กœ์— ์œ„์น˜ํ•œ servertohdfs.csvํŒŒ์ผ์„ HDFS์— ์ ์žฌํ•˜๊ณ ์ž ํ•ฉ๋‹ˆ๋‹ค.

servertohdfs02

  1. ๋ชฉ๋ก์—ด๊ธฐ ๋ฒ„ํŠผ์„ ํ™œ์šฉํ•˜์—ฌ ์›๊ฒฉ์„œ๋ฒ„ ์ •๋ณด๋ฅผ ์„ ํƒ, ์„œ๋ฒ„์ •๋ณด๋ฅผ ์„ ํƒํ•˜๋ฉด [1.์›๊ฒฉ ์„œ๋ฒ„ IP], [4.user], [5.password] ํ•ญ๋ชฉ์ด ์ž๋™ ์ž…๋ ฅ
  2. [2.sourceFolder]์— "/home/dpcore"๋ฅผ [3.sourceFile]์— "servertohdfs.csv"๋ฅผ ์ž…๋ ฅ
  3. [6.hdfsFilePath] ์ •๋ณด ์ž…๋ ฅ
    servertohdfs03
  4. ์›Œํฌํ”Œ๋กœ์šฐ ์‹คํ–‰๊ฒฐ๊ณผ
    servertohdfs04