Subscribe:
RSS feed
Articles on Big Data, HPC, and Highly Scalable Software Engineering
Browsing All Posts filed under »Big Data«
Fast Intersection of Sorted Lists Using SSE Instructions
Intersection of sorted lists is a cornerstone operation in many applications including search engines and databases because indexes are often implemented using different types of sorted structures. At GridDynamics, we recently worked on a custom database for realtime web analytics where fast intersection of very large lists of IDs was a must for good performance. From a functional […]
Probabilistic Data Structures for Web Analytics and Data Mining
May 1, 2012
Statistical analysis and mining of huge multi-terabyte data sets is a common task nowadays, especially in the areas like web analytics and Internet advertising. Analysis of such large data sets often requires powerful distributed data stores like Hadoop and heavy data processing with techniques like MapReduce. This approach often leads to heavyweight high-latency analytical processes and […]
NoSQL Data Modeling Techniques
March 1, 2012
NoSQL databases are often compared by various non-functional criteria, such as scalability, performance, and consistency. This aspect of NoSQL is well-studied both in practice and theory because specific non-functional properties are often the main justification for NoSQL usage and fundamental results on distributed systems like the CAP theorem apply well to NoSQL systems. At the same time, NoSQL […]
MapReduce Patterns, Algorithms, and Use Cases
February 1, 2012
In this article I digested a number of MapReduce patterns and algorithms to give a systematic view of the different techniques that can be found on the web or scientific articles. Several practical case studies are also provided. All descriptions and code snippets use the standard Hadoop’s MapReduce model with Mappers, Reduces, Combiners, Partitioners, and sorting. This […]
-
Ilya Katsov
- Distributed Algorithms in NoSQL Databases
- Speeding Up Hadoop Builds Using Distributed Unit Tests
- Fast Intersection of Sorted Lists Using SSE Instructions
- Probabilistic Data Structures for Web Analytics and Data Mining
- Hierarchical Navigation and Faceted Search on Top of Oracle Coherence
- NoSQL Data Modeling Techniques
- Tricks with Direct Memory Access in Java
- MapReduce Patterns, Algorithms, and Use Cases
- Implementation of MVCC Transactions for Key-Value Stores
- Performance of Priority Queue Sorting with Pagination
June 5, 2012
15