When To Use Hadoop vs In-Memory vs MPP

-

When presented with more options, choosing among big data tools can seem confusing. This blog explains the benefits and uses of the various option paths available to help guide choices for your next big data project.

Spark, Storm, or Kafka: It is an in-memory data grid that provides real-time data access to applications that are critical to the revenue stream of the business.

MangoDB: It’s massively parallel processing style of data management makes it an excellent choice for analytics.

 Hadoop:  is your research and development arm. As the landing spot for all data, and powered by a powerful SQL query engine, you can explore all data to identify new insights and opportunities you can later operationalize with MPP or in-memory.

Questions you need to consider when choosing:
  • When do I need it? Now? Later?
  • What do I want to do with it? Singular event processing? (includes some analytics), Transactions? Exploratory analytics?
  • How will I query and search? Structured, regular? Using an Alternate index(other sources)? Unstructured/unknown?
  • How do I need to store it? Temporarily?  I do, but not required? I must and I’m required to?
  • Where is it coming from? Events/Stream? File? ETL?


Comments

Post a Comment

Popular posts from this blog

Book Review: The Catalyst: How to Change Anyone's Mind

-
This week, I was reading a  book  by Jonah Burger (The Catalyst: How to Change Anyone's Mind) which is one of his great collections of social influence topics. In the book, he mentions that everyone has something they want to change. Employees want to change their bosses' minds, and leaders want to transform organizations. Salespeople want to win new clients, and startups want to revolutionize industries. Parents want to change their children's behavior, and political canvassers want to sway voters. But change is hard. We pressure and coax and cajole, and often nothing moves. Could there be a better way? Whether you are trying to convince a client, change an organization, disrupt a whole industry or just get someone to adopt a puppy, the same rules apply: Reduce reactance: Allow an agency, make them feel they made the decision. Ease endowment:  Bring the cost of inaction to the service, if you don't upgrade, we won't support Shrink distance:  Uber to rid...
Read more

Math for Data Science

-
Here's an exhaustive list of learning resources (in no particular order): ==================================== 1) Linear Algebra: (used in machine learning (& deep learning) to understand how algorithms work under the hood. Basically, it's all about vector/matrix/tensor operations, no black magic is involved!) - Khan Academy Linear Algebra series (beginner friendly) (https://www.khanacademy.org/math/linear-algebra) - Coding the Matrix course (and book) (http://codingthematrix.com/, http://academictorrents.com/details/54cd86f3038dfd446b037891406ba4e0b1200d5a) - 3Blue1Brown Linear Algebra series (https://www.youtube.com/playlist?list=PLZHQObOWTQDPD3MizzM2xVFitgF8hE_ab) - fastai Linear Algebra for coders course, highly related to modern ML workflow! (https://github.com/fastai/numerical-linear-algebra/blob/master/README.md) - First course in Coursera Mathematics for Machine Learning specialization (https://www.coursera.org/specializations/mathematics-machine-lear...
Read more