Hopsworks

Hopsworks is an operational machine learning platform with the industry’s leading feature store for machine learning (ML). Feature stores manage data throughout the entire ML lifecycle, from training to production. Hopsworks is an open platform that enables data to be either mounted as external tables from existing data stores or ingested from any data platform that has APIs for Python, Spark, or Flink. Hopsworks provides secure, centralized discovery and governance for features and models. As a Data Warehouse for data for ML with scalable Spark/Flink/SQL support, Hopsworks has pioneered the development of Python-native APIs that bring the value of feature stores to users with reasonably sized data. Hopsworks enables data scientists to use Python to create, use, and reuse features, all with the highest performance. 

Python-Centric

Using Hopsworks, data scientists can work, share, and interact with production and prototype environments in a Python-centric manner. Using transpilation, we bring the power of SQL to our Python SDK and seamlessly transfer data from any warehouse to Python for feature engineering and model training. 

Hopsworks provides the shortest feedback loop for developers when reading and writing features from/to a feature store. You can read and write DataFrames (in Python or Spark) in seconds, providing an interactive experience that enables you to build and test your machine learning pipelines faster. You can easily select features for models using a Pandas-like API, while behind the scene Hopsworks ensures no future data leakage by implementing an efficient point-in-time correct Join. Hopsworks also integrates with Great Expectations for validating feature data, helping ensure the correctness of data used for training and inference.

Operational Performance and High Availability

Hopsworks’ operational feature store is powered by the industry’s most performant and highly available database, RonDB. RonDB is built for feature store workloads such as personalized search and recommendation with fan-out, low latency feature access. 

Seamless Collaboration

Hopsworks is a collaborative platform, where Data Scientists, ML Engineers, and Data Engineers can, together, organize and use data for training and inference. Hopsworks includes role-based access control, project-based multi-tenancy, custom metadata for discovery and governance, provenance for features and models, and feature monitoring.

All Clouds and On-Premises Hopsworks is available as a SaaS platform (app.hopsworks.ai) with a generous free-forever tier, as a managed platform that runs in your cloud account (AWS, Azure, and GCP), and can be installed on-premises in your data center (including in air-gapped environments).