Hadoop

I recently bought Hadoop: The Definitive Guide Second Edition from O’Reilly.

The book is pretty good and gives a good overview of Hadoop and its family of projects

  • MapReduce (a distributed data processing model and execution environment that runs on large clusters of commodity machines)
  • HDFS (is an abstraction layer for multiple filesystems and a distributed filesystem that runs on large clusters of commodity machines)
  • Avro (an efficient serialization system for cross-language RPC, and persistent data storage)
  • Pig (a data flow language and execution environment for exploring very large datasets)
  • Hive (SQL like language which is translated to MapReduce jobs)
  • HBase (a distributed column-oriented(family) database which uses HDFS)
  • Zookeeper (a distributed, highly available coordination service)
  • Sqoop (a tool for efficiently moving data between relational databases and HDFS)

I can see Hadoop being useful for certain scenarios but the datasets would have to be huge and growing quickly. A commodity server, running an RDBMS, these days can be 48 cores and 1TB of RAM. That’s pretty powerful. It also depends on how the data is to be used. If the data can be summarised / aggregated daily then this can be efficiently stored within an RDBMS.

The book does include some good case studies. If you are looking to learn about Hadoop and its family of projects then this is a good book.

Advertisements

4 Responses to “Hadoop”

  1. marc Says:

    Also check the old videos on http://www.cloudera.com/resources/Video/
    Lots of sales and conference stuff, but watch the last two videos in the list named “Hadoop Training”

  2. Kiran My Says:

    Thanks for providing book infomatio on Hadoop. hadoop can also be used olap and oltp processing.

    Please click here to know more on basics Hadoop
    why Hadoop is introduced

  3. When should I use Hadoop | The diary of a CTO Says:

    […] although I am not sure even data around 5TB warrants using Hadoop. I wrote a short blog post on Hadoop a while […]

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s


%d bloggers like this: