Skip to main content

Distributed Hash Table


The idea is to have a lightweight alternative to a database where all the data resides in main memory across multiple machines, rather than on disk.

Goal is to create a single logical web cache by implementing a shared cache at a large scale requires spreading the cache over multiple machines.

DHTs characteristically emphasize the following properties: „ 

1. Decentralization: the nodes collectively form the system without any central coordination. „ 

2. Scalability: the system should function efficiently even with thousands or millions of nodes. „ 

3. Fault tolerance: the system should be reliable (in some sense) even with nodes continuously joining, leaving, and failing. 

A key technique used to achieve these goals is that any one node needs to coordinate with only a few other nodes in the system – most commonly, Θ(logn) of the n participants (see below) – so that only a limited amount of work needs to be done for each change in membership.

The structure of a DHT can be decomposed into several main components.[2][3] The foundation is an abstract keyspace, such as the set of 160-bit strings. A keyspace partitioning scheme splits ownership of this keyspace among the participating nodes. An overlay network then connects the nodes, allowing them to find the owner of any given key in the keyspace.



An overlay network and the underlay network on top of which the overlay network is built. Messages between the nodes in the overlay network logically follow the ring topology of the overlay network, but physically pass through the links and routers that form the underlay network.

DHT dynamically decide which node is responsible for which items. If the nodes currently responsible for certain items are removed from the system, the DHT self-manages by giving other nodes the responsibility over those items. Thus, nodes can continuously join and leave the system. The DHT will ensure that the routing tables are updated, and items are redistributed, such that the basic operations still work. This joining or leaving of nodes is referred to as churn or network dynamism

Another key feature of DHTs is that they are fault-tolerant. This implies that lookups should be possible even if some nodes fail. This is typically achieved by replicating items. Hence failures can be tolerated to a certain degree as long as there are some replicas of the items on some alive nodes. faulttolerance and the accompanying replication are self-managed by the system. This means that the system will automatically ensure that whenever a node fails, some other node actively starts replicating the items of the failed node to restore the replication degree.

Functionality of DHTs

Range Queries In some applications, it might be useful to ask the DHT to find values associated to all keys in a numerical or an alphabetical range. F

 Group Communication The routing information which exists in DHTs can be used for group communication. This is a dual use of DHTs, whereby they are not really used to do lookups for items, but rather just used to facilitate group communication among many hosts. For instance, the routing tables in the DHT can be used to broadcast a message from one node to every other node in the overlay network. The advantage of this is that every node gets the message in few time steps, while every node only needs to forward the message to a few other nodes.


Comments

Post a Comment

Popular posts from this blog

OWASP API Security Top 10 vulnerabilities 2023

API Security Do's and Don'ts Don't trust input data for an API and do validate all inputs. Ensure you understand security and keep sensitive data out of code. Don't hardcode keys/tokens Don't reveal useful info in error messages. Don't have hidden/unadvertised features. Don't filter data in UI - control at app level Don't confuse authentication and authorization Always use API gateways to control access and traffic Do require API documentation. Do expect users/hackers to find and use undocumented endpoints Do continuous testing - attack simulation, test configs, fuzzing, injections OWASP API Security Top 10 Vulnerabilities 2023 API-1:2023 - Broken Object Level Authorization: BOLA is still the leading vulnerability that plagues APIs. When data objects do not have sufficient access controls in place, resources can be accessed by unauthorized users. API-2:2023 - Broken Authentication  Broken Authentication contains all vulnerabilities associated with auth...
Post a Comment
Read more

Load Balancer Routing Algorithms - Draft version

The main purpose of the load balancer is to distribute the traffic evenly across multiple servers. The Load Balancer promises the efficient usage of the back-end servers without overloading and not keeping the server in busy times. Avoiding the server overload will reduce the downtime. Table of Content: Different types of LB Algorithms used by ALB ALB Features  Questions Configuring ALB in AWS  Creating K8S config files Factors to consider to ALB design There are different types of load balancers: Application Load Balancer (Layer 7) Network Load Balancer Global Server Load Balancer Let's narrow our discussion to Application Load Balancer(ALB). The ALB operates on layer 7 of OSI model.   Algorithms used by ALB are: Round Robin   Least connections Weighted Round Robin IP hash Least Connections Response time The ALB evaluates incoming requests to ensure efficient and reliable traffic distribution. LB has to ensure. High availability Scalability Performance opt...
Post a Comment
Read more

React JS Basics

  What are side effects in React? In React, side effects are operations that interact with external systems or cause changes outside the component's rendering process. These can include: Data fetching: Retrieving data from APIs or other sources. Subscriptions: Setting up listeners for events or data changes. Timers: Creating timers for delayed actions or animations. DOM manipulation: Directly modifying the DOM (rarely used in modern React with declarative approach). Why use useEffect ? In class-based components, you would typically use lifecycle methods like componentDidMount , componentDidUpdate , and componentWillUnmount to handle side effects. Functional components don't have these methods directly. The useEffect Hook provides a way to manage side effects in functional components. It allows you to run a function after a component renders (or re-renders) and optionally clean up any resources created by that function before the component unmounts. How does useEffect wor...
Post a Comment
Read more