Conference about Big Data, High Load, Data Science, Machine Learning & AI
Gerard is a team lead of the data systems team at Coolblue in The Netherlands. He’s a kaggle master, ran a drone company in Brazil and worked as a senior developer on unmanned aerial vehicle management systems, autopilots, telecom billing systems and HDMI automated testing systems.
Design philosophy of Apache Airflow ETL P
Apache Airflow is attracting a lot of attention over the past couple of years. This session explains very important principles that should be maintained in your ETL pipelines to make them scalable and restartable; many of these principles have been known for years in functional programming communities. Apache Airflow is designed around that philosophy and naturally guides the developer towards better and more scalable pipelines.