27-29 November

Conference about Big Data, High Load, Data Science, Machine Learning & AI

Day(s)

:

Hour(s)

:

Minute(s)

:

Second(s)

GERARD TOONSTRA

Coolblue B.V., The Netherlands

GERARD TOONSTRA

Coolblue B.V., The Netherlands

Biography

Gerard is a team lead of the data systems team at Coolblue in The Netherlands. He’s a kaggle master, ran a drone company in Brazil and worked as a senior developer on unmanned aerial vehicle management systems, autopilots, telecom billing systems and HDMI automated testing systems.

Talk

Design philosophy of Apache Airflow ETL Pipelines

Apache Airflow is attracting a lot of attention over the past couple of years. This session explains very important principles that should be maintained in your ETL pipelines to make them scalable and restartable; many of these principles have been known for years in functional programming communities. Apache Airflow is designed around that philosophy and naturally guides the developer towards better and more scalable pipelines.