rcmnd

RobinhoodCompany

Commission-free brokerage and crypto trading app

10 things

DebeziumSoftware

recommended

For the first stage, we chose Debezium as a Change Data Capture (CDC) provider. Debezium is an open source distributed change data capture platform built on top of Kafka Connect. Debezium comes with a well-documented first class Postgres CDC connector that’s battle-tested.

Robinhood choosing Debezium for change data capture into its data lake

2022-02-18·robinhood.com ↗

Apache HudiSoftware

recommended

For the second stage, we are using Apache Hudi to incrementally ingest changelogs from Kafka to create data-lake tables.

Robinhood choosing Hudi for incremental data lake ingestion

2022-02-18·robinhood.com ↗

KubernetesSoftware

recommended

In this post, we’ll talk a little about why we’re embracing Kubernetes to tackle these challenges, share some stories from our experience onboarding applications onto Kubernetes, and discuss the platform we built to manage and standardize our Kubernetes-powered applications, called the Archetype Framework.

Robinhood moves its microservice deployments onto Kubernetes

2019-11-13·robinhood.com ↗

Apache SparkSoftware

uses

We run production batch processing pipelines predominantly using Apache Spark. Our dashboards are powered by Trino distributed SQL query engine.

Robinhood's data lake compute stack

2022-02-18·robinhood.com ↗

TrinoSoftware

uses

We run production batch processing pipelines predominantly using Apache Spark. Our dashboards are powered by Trino distributed SQL query engine.

Robinhood's data lake query engine

2022-02-18·robinhood.com ↗

KafkaSoftware

uses

We use Kafka as our databus to stream data, which usually comes from external feeds, or from internal Faust apps.

Robinhood's data lake ingestion layer

2019-09-09·robinhood.com ↗

PostgreSQLSoftware

uses

We use PostgreSQL and AWS Aurora as our relational databases, Elasticsearch as a document storage and indexing solution, AWS S3 for object storage, and InfluxDB as a time series database.

Robinhood's data stores ahead of building its data lake

2019-09-09·robinhood.com ↗

SaltStackSoftware

mixed

Historically, we have used a combination of Terraform and SaltStack to manage our AWS infrastructure. While this combination of technologies has carried us quite far

Robinhood on the Terraform and SaltStack setup it was moving away from

2019-11-13·robinhood.com ↗

RedshiftSoftware

disliked

We initially used Elasticsearch and Redshift as our analytics and data warehousing solutions. After a couple of years of maintaining these systems, we determined that scaling these systems to support our increasing data workload was neither efficient nor economical.

Why Robinhood built a data lake instead of scaling Elasticsearch and Redshift

2019-09-09·robinhood.com ↗

ElasticsearchSoftware

disliked

We initially used Elasticsearch and Redshift as our analytics and data warehousing solutions. After a couple of years of maintaining these systems, we determined that scaling these systems to support our increasing data workload was neither efficient nor economical.

2019-09-09·robinhood.com ↗

Every entry is a verbatim quote and a link to the public post it came from. Nothing is paraphrased. If it is not on record somewhere public, it is not here.

The verdict is what they actually said: loved and liked are explicit; recommended means they told others to get it; uses means they only say they use it; read is a book on their public shelf with no verdict; mixed and disliked are kept too.

A dashed affiliate link, sponsored or their own label means the post disclosed a relationship. It is shown, never hidden.

The shading behind a date shows how dated a recommendation is for its kind of thing.

The shading behind a date shows how dated a recommendation is, for its kind of thing. Fully green is recent. It drains a quarter at a time as the post gets older, and is empty once it reads as dated.

What counts as old depends on the thing. A laptop praised four years ago has been replaced twice; a book loved twenty years ago is probably still loved. The shading is gone after about:

Stripes mean the post carries no date and the date shown is an estimate. No date at all, no shading.