Posts

  ‘WHAT’ we are doing remains the same but ‘HOW’ we are doing is different now   Sai: Um.. Me : See, we are reading books, watching movies but the way we are doing is different. S: But the technical landscape behind that is different now. M: Both Yes & NO. There also, same rule applies…’WHAT’ remains same but ‘HOW’ is different.  S: ? M: Take big data. How do you think, any data system should work. S: It should read the input, store the input, process the input & provide output. M: Same is happening in BIG Data. We call this Ingestion/ pulling in raw data( by Pig/Hive) , storage (in HDFS) , analysis ( by Spark/MapReduce), output presentation ( by  Tableau/Plotly )   Only thing is that now each component is doing say only 1 job but doing that job very effectively.   So, while earlier, all that work was getting done by a big program, now different components of that program are available & overall combination of those components is the final output.  Each component is really