Awesome Hadoop

---

title: Awesome Hadoop slug: awesome-hadoop brand: youngwookim brand_logo: https://avatars.githubusercontent.com/u/1323384 category: themed-directories tags:

• awesome-lists
• big-data
• curated-lists source_url: https://github.com/youngwookim/awesome-hadoop#readme featured: false

---

Overview

Awesome Hadoop is a curated, community-maintained directory of Hadoop and Hadoop ecosystem resources. It organizes tools, libraries, and references for distributed storage and processing of very large datasets, following the popular "Awesome List" format on GitHub.

Features

• Hadoop Core Ecosystem

  • Resources focused on Apache Hadoop itself and its core components.

• YARN

  • Tools, references, and libraries related to Hadoop YARN (Yet Another Resource Negotiator).

• NoSQL Databases

  • Next-generation, non-relational, distributed, open‑source, horizontally scalable databases used in Big Data contexts.

• SQL on Hadoop

  • Engines and tools that provide SQL querying capabilities on top of Hadoop-based storage.

• Data Management

  • Projects and resources for managing datasets, schemas, and metadata in Hadoop environments.

• Workflow, Lifecycle, and Governance

  • Workflow schedulers, orchestration tools, and governance frameworks for data pipelines and Hadoop jobs.

• Data Ingestion and Integration

  • Tools for ingesting, integrating, and moving data into and within the Hadoop ecosystem.

• DSL (Domain-Specific Languages)

  • DSLs designed for data processing, querying, or pipeline definition on Hadoop.

• Libraries and Tools

  • General-purpose libraries, utilities, and tooling that support development and operations around Hadoop.

• Realtime Data Processing

  • Frameworks and systems for streaming and near-real-time data processing on or alongside Hadoop.

• Distributed Computing and Programming

  • Programming frameworks and models for large-scale distributed computation.

• Packaging, Provisioning, and Monitoring

  • Tools for deploying, configuring, and monitoring Hadoop clusters and related services.

• Search and Search Engine Frameworks

  • Search systems and frameworks that integrate with or complement Hadoop-based architectures.

• Security

  • Solutions and references for securing Hadoop ecosystems, including access control and data protection.

• Benchmark

  • Benchmarking tools and resources for evaluating performance of Hadoop and related components.

• Machine Learning and Big Data Analytics

  • Libraries, frameworks, and tools for analytics and machine learning on large datasets.

• Miscellaneous

  • Additional Hadoop and Big Data related tools and resources that don’t fit other categories.

Resources

• Websites

  • Curated selection of useful Hadoop and Big Data websites and articles.

• Presentations

  • Talks, slide decks, and presentations related to Hadoop and its ecosystem.

• Books

  • Reference books and reading materials covering Hadoop and Big Data topics.

• Hadoop and Big Data Events

  • Conferences, meetups, and other events focused on Hadoop and broader Big Data technologies.

Pricing

• Not applicable. Awesome Hadoop is an open, GitHub-hosted curated list with free access.