Prefect was founded in 2018 by Jeremiah Lowin in Washington, DC. Lowin had worked in quantitative finance and data science, where he experienced firsthand how painful it was to manage complex data pipelines with tools like Apache Airflow and cron jobs.
Prefect’s core product is an open-source Python framework for building, scheduling, and monitoring data workflows. Unlike Airflow, which requires defining pipelines as DAGs (Directed Acyclic Graphs) with rigid structure, Prefect lets developers write standard Python code and add orchestration with simple decorators. Any Python function can become a task or flow with minimal changes.
Prefect Cloud is the commercial managed service. It handles scheduling, monitoring, alerting, and collaboration without requiring users to manage infrastructure. The dashboard shows flow run status, logs, and performance metrics across an organization’s entire pipeline landscape.
Prefect 2.0, released in 2022, was a major rewrite that simplified the framework dramatically. The company later evolved the product further with Prefect 3.0, focusing on async support, better error handling, and integration with the broader Python data ecosystem including pandas, Polars, and various ML frameworks.
The company raised $32 million in Series B funding in 2022 from investors including Insight Partners and Lightspeed Venture Partners. Thousands of organizations use Prefect, including data teams at major enterprises and fast-growing startups.
Lowin remains CEO and is active in the data engineering community. Prefect competes with tools like Airflow, Dagster, and Temporal in the workflow orchestration space. The company employs around 100 people.
