World Library  
Flag as Inappropriate
Email this Article

Dynamo (storage system)

Article Id: WHEBN0016872669
Reproduction Date:

Title: Dynamo (storage system)  
Author: World Heritage Encyclopedia
Language: English
Subject: NoSQL, Amazon DynamoDB, Apache Cassandra, Werner Vogels, Amazon.com
Collection: Amazon.Com, Distributed Data Stores, Nosql
Publisher: World Heritage Encyclopedia
Publication
Date:
 

Dynamo (storage system)

Dynamo is the name given to a set of techniques that when taken together can form a highly available key-value structured storage system[1] or a distributed data store.[2] It has properties of both databases and distributed hash tables (DHTs). Amazon DynamoDB is "built on the principles of Dynamo" [3] (apparently with some improvements) and is a hosted solution within the AWS infrastructure. It was created to help address some scalability issues that Amazon.com's website experienced during the holiday season of 2004.[4] Today it is "used to power parts of our Amazon Web Services, such as S3." [5]

Contents

  • Principles of Dynamo Systems 1
  • Techniques Used in Dynamo Systems 2
  • Implementation History 3
  • See also 4
  • References 5
  • External links 6

Principles of Dynamo Systems

  • Incremental scalability: Dynamo should be able to scale out one storage host (henceforth, referred to as “node”) at a time, with minimal impact on both operators of the system and the system itself.
  • Symmetry: Every node in Dynamo should have the same set of responsibilities as its peers; there should be no distinguished node or nodes that take special roles or extra set of responsibilities.
  • Decentralization: An extension of symmetry, the design should favor decentralized peer-to-peer techniques over centralized control.
  • Heterogeneity: The system needs to be able to exploit heterogeneity in the infrastructure it runs on. e.g. the work distribution must be proportional to the capabilities of the individual servers. This is essential in adding new nodes with higher capacity without having to upgrade all hosts at once.

Techniques Used in Dynamo Systems

Problem Technique Advantage
dataset partitioning Consistent Hashing Incremental, possibly linear scalability in proportion to the number of collaborating nodes.
highly available writes Vector Clock or Dotted-Version-Vector Sets, reconciliation during reads Version size is decoupled from update rates.
handling temporary failures Sloppy Quorum and Hinted Handoff Provides high availability and durability guarantee when some of the replicas are not available.
recovering from permanent failures anti-entropy using Merkle tree Can be used to identify differences between replica owners and synchronize divergent replicas pro-actively.
membership and failure detection gossip-based membership protocol and failure detection Avoids having a centralized registry for storing membership and node liveness information, preserving symmetry.

Implementation History

Amazon published the paper on Dynamo, but never released their implementation. The index layer of Amazon S3 implements and extends many of the core features of a Dynamo. Since then, several implementations have been created based on the paper. Apparently, the paper also inspired many other NoSQL database implementations.[6] Here are some projects that implement it or were inspired by it.

See also

References

  1. ^ Dynamo: Amazon’s Highly Available Key-value Store, SOSP 2007
  2. ^
  3. ^ http://www.allthingsdistributed.com/2012/01/amazon-dynamodb.html
  4. ^ http://readwrite.com/2012/01/18/amazon-enters-the-nosql-market
  5. ^ http://www.allthingsdistributed.com/2007/10/amazons_dynamo.html
  6. ^ http://readwrite.com/2012/01/18/amazon-enters-the-nosql-market "launched a thousand NoSQL databases"

External links

  • Amazon's Dynamo (2007)
  • Amazon reveals its distributed storage: Dynamo (2007)


This article was sourced from Creative Commons Attribution-ShareAlike License; additional terms may apply. World Heritage Encyclopedia content is assembled from numerous content providers, Open Access Publishing, and in compliance with The Fair Access to Science and Technology Research Act (FASTR), Wikimedia Foundation, Inc., Public Library of Science, The Encyclopedia of Life, Open Book Publishers (OBP), PubMed, U.S. National Library of Medicine, National Center for Biotechnology Information, U.S. National Library of Medicine, National Institutes of Health (NIH), U.S. Department of Health & Human Services, and USA.gov, which sources content from all federal, state, local, tribal, and territorial government publication portals (.gov, .mil, .edu). Funding for USA.gov and content contributors is made possible from the U.S. Congress, E-Government Act of 2002.
 
Crowd sourced content that is contributed to World Heritage Encyclopedia is peer reviewed and edited by our editorial staff to ensure quality scholarly research articles.
 
By using this site, you agree to the Terms of Use and Privacy Policy. World Heritage Encyclopedia™ is a registered trademark of the World Public Library Association, a non-profit organization.
 



Copyright © World Library Foundation. All rights reserved. eBooks from Hawaii eBook Library are sponsored by the World Library Foundation,
a 501c(4) Member's Support Non-Profit Organization, and is NOT affiliated with any governmental agency or department.