• Home
  • Categories
  • Pricing
  • Submit
    Built with
    Ever Works
    Ever Works

    Connect with us

    Stay Updated

    Get the latest updates and exclusive content delivered to your inbox.

    Product

    • Categories
    • Pricing
    • Help

    Clients

    • Sign In
    • Register
    • Forgot password?

    Company

    • About Us
    • Admin
    • Sitemap

    Resources

    • Blog
    • Submit
    • API Documentation
    All product names, logos, and brands are the property of their respective owners. All company, product, and service names used in this repository, related repositories, and associated websites are for identification purposes only. The use of these names, logos, and brands does not imply endorsement, affiliation, or sponsorship. This directory may include content generated by artificial intelligence.
    Copyright © 2025 Ever. All rights reserved.·Terms of Service·Privacy Policy·Cookies
    Decorative pattern
    Decorative pattern
    1. Home
    2. Data Engineering
    3. Awesome Data Pipeline

    Awesome Data Pipeline

    A curated list of data pipeline tools and frameworks organized by category including workflow management, data ingestion, data lakes, query engines, streaming, and data transformation.

    Surveys

    Loading more......

    Information

    Websitegithub.com
    PublishedMar 7, 2026

    Categories

    1 Item
    Data Engineering

    Tags

    3 Items
    #data-pipelines#workflow-management#stream-processing

    Similar Products

    6 result(s)

    Awesome Open-Source Data Engineering

    A comprehensive collection of open-source projects for data engineering, covering workflow orchestration, data processing, streaming, storage, and quality tools used in modern data platforms.

    Featured

    Awesome Kafka

    A curated list of awesome Apache Kafka resources, tools, libraries, and applications for building event-driven architectures and real-time data streaming platforms.

    Awesome ETL

    A curated list of notable ETL (Extract, Transform, Load) frameworks, libraries, and software for building robust data pipelines and managing data integration workflows.

    Awesome Apache Airflow

    A curated collection of resources, plugins, tools, and best practices for Apache Airflow, the leading open-source platform for orchestrating complex data workflows and ETL pipelines.

    Awesome Database Tools

    A comprehensive collection of tools that make working with databases easier, including migration tools, GUI clients, CLI utilities, monitoring solutions, and database management platforms.

    Awesome Database Design

    A collection of resources and tutorials to design better database schemas, covering normalization, indexing strategies, performance optimization, and best practices.

    Overview

    Awesome Data Pipeline is a well-organized collection of tools and frameworks for building modern data pipelines, from ingestion to transformation.

    Features

    • Workflow Management: Apache Airflow, Apache NiFi, and Luigi from Spotify
    • Data Ingestion: Apache Flume and Stitch from Talend for ETL
    • Data Lake Solutions: Tools for building scalable data lakes
    • Data Warehouse: Platforms for analytical data storage
    • Query Engines: High-performance query engines for big data
    • Streaming: Real-time data streaming platforms
    • Data Transformation: Tools for cleaning, enriching, and transforming data

    Categories

    Organized into clear categories:

    • Workflow Management
    • Data Ingestion
    • Data Lake
    • Data Warehouse
    • Query Engine
    • Streaming
    • Data Transformation

    Use Cases

    • Building end-to-end data platforms
    • Real-time analytics
    • Batch processing workflows
    • Data orchestration
    • ETL/ELT operations

    Pricing

    Free and open-source repository on GitHub.